CN111273704A - Method and device for automatically inserting external equipment hole - Google Patents
Method and device for automatically inserting external equipment hole Download PDFInfo
- Publication number
- CN111273704A CN111273704A CN202010072741.6A CN202010072741A CN111273704A CN 111273704 A CN111273704 A CN 111273704A CN 202010072741 A CN202010072741 A CN 202010072741A CN 111273704 A CN111273704 A CN 111273704A
- Authority
- CN
- China
- Prior art keywords
- external equipment
- hole
- coordinate position
- dimensional
- plug
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
- G05D3/20—Control of position or direction using feedback using a digital comparing device
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
Abstract
The invention aims to provide a method and a device for automatically inserting an external equipment hole.A three-dimensional coordinate position of the external equipment hole in a photo is identified from the photo by shooting the photo of the side surface of the external equipment hole on the local equipment, and the three-dimensional coordinate position of the external equipment hole is converted into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula; and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the external equipment hole, thereby realizing the accurate positioning of the position of the external equipment hole of the equipment, and further reliably controlling the plug of the external equipment of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and a device for automatically inserting an external device hole.
Background
The existing mode for detecting the performance of intelligent equipment such as a mobile phone needs to manually insert an external equipment plug of detection equipment into an external equipment hole of the intelligent equipment such as the mobile phone, so that time and labor are wasted, and the labor cost is higher.
Disclosure of Invention
The invention aims to provide a method and a device for automatically inserting into a hole of external equipment.
According to one aspect of the present invention, there is provided a method for automatically inserting a peripheral device hole, the method comprising:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
Further, in the above method, taking a picture of a side surface where an external device hole on the local device is located, recognizing a three-dimensional coordinate position of the external device hole in the picture from the picture, and converting the three-dimensional coordinate position of the external device hole into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole includes:
scanning the side surface of the external equipment hole of the local equipment by using line laser;
shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process;
splicing all the imaging lines to form a three-dimensional point cloud outline of the side face where an external equipment hole of the equipment is located;
obtaining a three-dimensional coordinate position in the picture of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
controlling the external device plug to move to the three-dimensional target coordinate position to insert the external device plug into the external device hole, including:
and controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC (programmable logic controller) control module based on the three-dimensional target coordinate position (x ', y ', z ').
Further, in the above method, obtaining a three-dimensional coordinate position in the photo of the external device hole based on a three-dimensional point cloud contour of a side surface where the external device hole is located, where the three-dimensional coordinate position includes a plane coordinate position (x, y) and a depth coordinate position z, includes:
acquiring a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
performing normal vector filtering on the three-dimensional point cloud outline ps1, if an included angle between a normal vector of a certain point in the three-dimensional point cloud outline ps1 and the vertical direction of the three-dimensional point cloud outline ps1 exceeds a preset threshold value a1, discarding the point, otherwise, reserving the point;
finally, a filtered point cloud ps2 is obtained from all the retained points;
performing depth map projection on the point cloud ps2 to obtain a depth image d;
extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as a contour c1 of an external device hole;
traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1;
the three-dimensional coordinate values (x, y, z) of all points in the contour c1 are added and averaged to obtain the center position of the circumscribed device hole contour c 1.
Further, in the above method, based on a preset spatial coordinate conversion formula, converting the planar coordinate position (x, y) and the depth coordinate position z of the peripheral device hole into a three-dimensional target coordinate position (x ', y ', z ') of the peripheral device plug, including:
and converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula.
Further, in the above method, the controlling, by the PLC control module, the plug of the external device to be inserted into the external device hole based on the three-dimensional target coordinate position (x ', y ', z '), includes:
and controlling a first moving device to drive an external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') through a PLC (programmable logic controller) control module based on the three-dimensional target coordinate position (x', y ', z'), so that the external equipment plug is inserted into the external equipment hole.
Further, in the above method, capturing each imaging line of the line laser on the side where the external device hole is located in the scanning process includes:
and shooting each imaging line of the line laser on the side face where the external equipment hole is located and parallel to the short edge of the side face where the external equipment hole is located in the scanning process.
Further, in the above method, the scanning of the side surface where the external device hole of the device is located with the line laser includes:
and scanning the side surface of the equipment external connection hole by using blue line laser.
Further, in the above method, based on a preset spatial coordinate conversion formula, converting the three-dimensional coordinate position of the external device hole into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole, includes:
recognizing the shape of the external equipment hole from the photo;
determining a corresponding preset space coordinate conversion formula based on the shape;
and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
According to another aspect of the present invention, there is also provided an apparatus for automatically inserting into a hole of an external device, the apparatus comprising:
the identification module is used for shooting a picture of the side face where the external equipment hole on the local equipment is located, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and the moving module is used for controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
Furthermore, in the above apparatus, the identification module is configured to scan a side surface where an external device hole of the local device is located with line laser; shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process; splicing all the imaging lines to form a three-dimensional point cloud outline of the side face where an external equipment hole of the equipment is located; obtaining a three-dimensional coordinate position in the picture of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z; converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
the moving module is used for controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC control module based on the three-dimensional target coordinate position (x ', y ', z ').
Further, in the above apparatus, the identification module is configured to obtain a three-dimensional point cloud profile ps1 of a side surface where the external device hole is located; performing normal vector filtering on the three-dimensional point cloud profile ps1, if an included angle between a normal vector of a certain point in the three-dimensional point cloud profile ps1 and the vertical direction of the three-dimensional point cloud profile ps1 exceeds a preset threshold a1, discarding the point, otherwise, reserving the point, and finally obtaining a filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as a contour c1 of an external device hole; traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1; the three-dimensional coordinate values (x, y, z) of all points in the contour c1 are added and averaged to obtain the center position of the circumscribed device hole contour c 1.
Further, in the above apparatus, the identification module is configured to convert the center position of the profile c1 of the peripheral device hole into a three-dimensional target coordinate position (x ', y ', z ') of the peripheral device plug based on a preset spatial coordinate conversion formula.
Further, in the above apparatus, the moving module is configured to control, through the PLC control module, the first moving device to drive the external device plug to move to the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is inserted into the external device hole.
Furthermore, in the above apparatus, the recognition module is configured to capture each imaging line of the line laser on the side where the external device hole is located, the imaging line being parallel to the short side of the side where the external device hole is located.
Further, in the above apparatus, the identification module is configured to use a side surface where an external device hole of the blue-line laser scanning device is located.
Further, in the above apparatus, the recognition module is configured to recognize a shape of the external device hole from the photo; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
According to another aspect of the present invention, there is also provided a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
Compared with the prior art, the three-dimensional coordinate position of the external equipment hole in the photo is recognized from the photo by shooting the photo of the side face where the external equipment hole on the local equipment is located, and the three-dimensional coordinate position of the external equipment hole is converted into the three-dimensional target coordinate position of the external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula; and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the external equipment hole, thereby realizing the accurate positioning of the position of the external equipment hole of the equipment, and further reliably controlling the plug of the external equipment of the detection equipment to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 is a flow chart illustrating a method for inserting a peripheral device port according to an embodiment of the present invention;
FIG. 2 shows a schematic image line of an embodiment of the present invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output ports, network ports, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
As shown in fig. 1, the present invention provides a method for automatically inserting into a hole of an external device, the method comprising:
step S1, shooting a photo of the side surface where the external equipment hole on the local equipment is located, recognizing the three-dimensional coordinate position of the external equipment hole in the photo from the photo, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
here, the local device may be an intelligent terminal such as a mobile phone and a PAD;
the external device hole can be a USB hole or an earphone hole on an intelligent terminal such as a mobile phone and a PAD;
and step S2, controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
The external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in step S1, a photo of a side surface of the local device where the external device hole is located is taken, a three-dimensional coordinate position of the external device hole in the photo is identified from the photo, and the three-dimensional coordinate position of the external device hole is converted into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole based on a preset spatial coordinate conversion formula, including:
step S11, scanning the side surface of the external equipment hole of the local equipment by using line laser;
the local device can be various devices such as mobile phones, computers, PADs and the like with external device holes, the local device generally comprises 6 surfaces, wherein the top surface is a surface with a display screen, the bottom surface is the back surface of the display screen, the other four surfaces are four side surfaces respectively connected with the top surface and the bottom surface, and the external device holes are generally arranged on one side surface of the local device;
step S12, shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process;
the line laser scans the side face where the external equipment hole is located to form an imaging line reflecting the unevenness of the profile of the side face where the external equipment hole is located;
step S13, splicing the imaging lines to form a three-dimensional point cloud outline of the side face where the external equipment hole of the equipment is located;
step S14, obtaining a three-dimensional coordinate position in the photo of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
step S15, converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
step S2, controlling the external device plug to move to the three-dimensional target coordinate position, so as to insert the external device plug into the external device hole, including:
and step S21, controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC control module based on the three-dimensional target coordinate position (x ', y ', z ').
The external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in step S14, obtaining a three-dimensional coordinate position in the photo of the external device hole based on a three-dimensional point cloud profile of a side surface where the external device hole is located, where the three-dimensional coordinate position includes a plane coordinate position (x, y) and a depth coordinate position z, and the method includes:
step S141, acquiring a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
step S142, performing normal vector filtering on the three-dimensional point cloud profile ps1, if the included angle between the normal vector of a certain point in the three-dimensional point cloud profile ps1 and the vertical direction of the three-dimensional point cloud profile ps1 exceeds a preset threshold value a1, discarding the point, otherwise, reserving the point, and finally obtaining a filtered point cloud ps2 from the reserved point;
here, the point cloud may also be generated in the inner wall of the external device hole or in the smooth transition area between the bottom and the front of the mobile phone, but this part of the point cloud is not needed, and especially the point inside the external device hole may cause the USB center extraction to be inaccurate. Because the points inside the external equipment hole are all concave inwards, the points which are not needed are filtered by judging the included angle to exceed the preset threshold value a 1;
step S143, performing depth map projection on the point cloud ps2 to obtain a depth image d;
step S144, extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as the contour c1 of the external device hole;
step S145, traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1;
step S146, adding the three-dimensional coordinate values (x, y, z) of all points in the profile c1, and averaging to obtain the center position of the profile c1 of the circumscribed equipment hole.
Here, the present embodiment can efficiently and accurately obtain the center position of the profile c1 of the peripheral device hole, so as to accurately locate the position of the peripheral device hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S15, based on a preset spatial coordinate transformation formula, transforms a planar coordinate position (x, y) and a depth coordinate position z of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of an external device plug, including:
and converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula.
Here, by converting the center position of the contour c1 of the peripheral device hole into a three-dimensional target coordinate position (x ', y ', z ') of the peripheral device plug, it is convenient to subsequently control the accurate insertion of the peripheral device plug into the peripheral device hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in step S16, the controlling, by the PLC control module, the plug of the external device to be inserted into the external device hole based on the three-dimensional target coordinate position (x ', y ', z '), includes:
and controlling a first moving device to drive an external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') through a PLC (programmable logic controller) control module based on the three-dimensional target coordinate position (x', y ', z'), so that the external equipment plug is inserted into the external equipment hole.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in step S12, capturing each imaging line of the line laser on the side where the external device hole is located during scanning includes:
and shooting each imaging line of the line laser on the side face where the external equipment hole is located and parallel to the short edge of the side face where the external equipment hole is located in the scanning process.
Here, as shown in fig. 2, the side where the external device hole is located is a rectangular frame surrounded by long sides and short sides, and the line laser is parallel to the short side 1 of the side where the external device hole 4 is located instead of each imaging line 3 parallel to the long side 2 in the shooting and scanning process, so that each obtained imaging line is as short as possible, and the accuracy of the three-dimensional point cloud profile of the side where the external device hole of the device is located is ensured by subsequently splicing each imaging line.
In an embodiment of the method for automatically inserting an external device hole of the present invention, step S11, the scanning the side surface of the external device hole of the device with the line laser includes:
and scanning the side surface of the equipment external connection hole by using blue line laser.
The blue line laser has the characteristics of short wavelength and strong energy, and each imaging line of the line laser on the equipment can be efficiently and reliably obtained by using the blue line laser.
In an embodiment of the method for automatically inserting an external device hole of the present invention, in step S1, based on a preset spatial coordinate conversion formula, the converting the three-dimensional coordinate position of the external device hole into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole includes:
step S101, recognizing the shape of the external equipment hole from the photo;
step S102, determining a corresponding preset space coordinate conversion formula based on the shape;
and step S103, converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
Here, for example, in the case where the external device hole is a USB hole, the USB hole may include different shapes such as apple USB hole, TYPEC, and MICRO, and the different shapes may correspond to different preset spatial coordinate conversion formulas, respectively, so that the three-dimensional coordinate position of the external device hole may be more reliably and accurately converted into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole.
The invention provides a device for automatically inserting into a hole of external equipment, which comprises:
the identification module is used for shooting a picture of the side face where the external equipment hole on the local equipment is located, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and the moving module is used for controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
The external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the apparatus for automatically inserting an external device hole of the present invention, the identification module is configured to use a side surface of a line laser scanning device where the external device hole is located;
the device can be various devices such as mobile phones, computers, PADs and the like with external device holes, the device generally comprises 6 surfaces, wherein the top surface is a surface with a display screen, the bottom surface is the back surface of the display screen, the other four surfaces are four side surfaces respectively connected with the top surface and the bottom surface, and the external device holes are generally arranged on one side surface of the device;
the identification module is used for shooting each imaging line of the line laser on the side face where the external equipment hole is located in the scanning process;
scanning the side surface of the line laser equipment where the external equipment hole is located to form an imaging line reflecting the unevenness of the profile of the side surface where the external equipment hole is located;
the identification module is used for splicing the imaging lines to form a three-dimensional point cloud outline of the side face where the external equipment hole of the equipment is located;
the identification module is used for obtaining a three-dimensional coordinate position in the picture of the external equipment hole based on a three-dimensional point cloud outline of the side face where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
the identification module is used for converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (values x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
the moving module is used for controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC control module based on the three-dimensional target coordinate position (x ', y ', z ').
The external equipment plug of the detection equipment can be reliably controlled to be automatically inserted into the external equipment hole based on the position of the external equipment hole of the equipment.
In an embodiment of the apparatus for automatically inserting an external device hole, the identification module is configured to obtain a three-dimensional point cloud profile ps1 of a side surface where the external device hole is located; performing normal vector filtering on the three-dimensional point cloud profile ps1, if an included angle between a normal vector of a certain point in the three-dimensional point cloud profile ps1 and the vertical direction of the three-dimensional point cloud profile ps1 exceeds a preset threshold a1, discarding the point, otherwise, reserving the point, and finally obtaining a filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as a contour c1 of an external device hole; traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1; the three-dimensional coordinate values (x, y, z) of all points in the contour c1 are added and averaged to obtain the center position of the circumscribed device hole contour c 1.
Here, the point cloud may also be generated in the inner wall of the external device hole or in the smooth transition area between the bottom and the front of the mobile phone, but this part of the point cloud is not needed, and especially the point inside the external device hole may cause the USB center extraction to be inaccurate. Since the points inside the peripheral device hole are all inwardly concave, the undesired points are filtered by judging the included angle to exceed the preset threshold a 1.
Here, the present embodiment can efficiently and accurately obtain the center position of the profile c1 of the peripheral device hole, so as to accurately locate the position of the peripheral device hole.
In an embodiment of the apparatus for automatically inserting an external device hole of the present invention, the recognition module is configured to convert a center position of the outline c1 of the external device hole into a three-dimensional target coordinate position (x ', y ', z ') of the external device plug based on a preset spatial coordinate conversion formula.
Here, by converting the center position of the profile c1 of the peripheral device hole into the target coordinate values (x ', y ', z ') of the PLC control module, it is convenient to subsequently control the plug of the peripheral device to be accurately inserted into the peripheral device hole.
In an embodiment of the apparatus for automatically inserting an external device hole, the moving module is configured to control, through the PLC control module, the first moving device to drive the external device plug to move to the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is inserted into the external device hole and moves to a position corresponding to the external device hole, so that the external device plug is inserted into the external device hole.
In an embodiment of the apparatus for automatically inserting an external device hole of the present invention, the recognition module is configured to capture each imaging line of the line laser on the side surface where the external device hole is located in the scanning process, the imaging line being parallel to the short side of the side surface where the external device hole is located.
Here, as shown in fig. 2, the side where the external device hole is located is a rectangular frame surrounded by long sides and short sides, and the line laser is parallel to the short side 1 of the side where the external device hole 4 is located instead of each imaging line 3 parallel to the long side 2 in the shooting and scanning process, so that each obtained imaging line is as short as possible, and the accuracy of the three-dimensional point cloud profile of the side where the external device hole of the device is located is ensured by subsequently splicing each imaging line.
In an embodiment of the apparatus for automatically inserting an external device hole of the present invention, the scanning module is configured to scan a side surface of the external device hole of the device using a blue-ray laser.
The blue line laser has the characteristics of short wavelength and strong energy, and each imaging line of the line laser on the equipment can be efficiently and reliably obtained by using the blue line laser.
In an embodiment of the apparatus for automatically inserting a hole of an external device, the recognition module is configured to recognize a shape of the hole of the external device from the photograph; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
Here, for example, in the case where the external device hole is a USB hole, the USB hole may include different shapes such as apple USB hole, TYPEC, and MICRO, and the different shapes may correspond to different preset spatial coordinate conversion formulas, respectively, so that the three-dimensional coordinate position of the external device hole may be more reliably and accurately converted into a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole.
According to another aspect of the present invention, there is also provided a computing-based device comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
For details of embodiments of each device and storage medium of the present invention, reference may be made to corresponding parts of each method embodiment, and details are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (18)
1. A method for automatically inserting a peripheral device port, wherein the method comprises:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
2. The method of claim 1, wherein taking a photograph of a side of a native device on which an external device hole is located, identifying a three-dimensional coordinate position of the external device hole in the photograph from the photograph, and converting the three-dimensional coordinate position of the external device hole to a three-dimensional target coordinate position of an external device plug to be inserted into the external device hole comprises:
scanning the side surface of the external equipment hole of the local equipment by using line laser;
shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process;
splicing all the imaging lines to form a three-dimensional point cloud outline of the side face where an external equipment hole of the equipment is located;
obtaining a three-dimensional coordinate position in the picture of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z;
converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
controlling the external device plug to move to the three-dimensional target coordinate position to insert the external device plug into the external device hole, including:
and controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC (programmable logic controller) control module based on the three-dimensional target coordinate position (x ', y ', z ').
3. The method of claim 2, wherein deriving three-dimensional coordinate locations in the photograph of the peripheral device hole based on a three-dimensional point cloud profile of a side of the peripheral device hole, including a planar coordinate location (x, y) and a depth coordinate location z, comprises:
acquiring a three-dimensional point cloud profile ps1 of the side surface where the external equipment hole is located;
performing normal vector filtering on the three-dimensional point cloud outline ps1, if an included angle between a normal vector of a certain point in the three-dimensional point cloud outline ps1 and the vertical direction of the three-dimensional point cloud outline ps1 exceeds a preset threshold value a1, discarding the point, otherwise, reserving the point;
finally, a filtered point cloud ps2 is obtained from all the retained points;
performing depth map projection on the point cloud ps2 to obtain a depth image d;
extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as a contour c1 of an external device hole;
traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1;
the three-dimensional coordinate values (x, y, z) of all points in the contour c1 are added and averaged to obtain the center position of the circumscribed device hole contour c 1.
4. The method of claim 3, wherein translating the planar coordinate position (x, y) and the depth coordinate position z of the peripheral device aperture to a three-dimensional target coordinate position (x ', y ', z ') of the peripheral device plug based on a preset spatial coordinate translation formula comprises:
and converting the central position of the outline c1 of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula.
5. The method of claim 4, wherein controlling, by the PLC control module, insertion of the add-on plug into the add-on hole based on the three-dimensional target coordinate position (x ', y ', z '), comprises:
and controlling a first moving device to drive an external equipment plug to move towards the three-dimensional target coordinate position (x ', y', z ') through a PLC (programmable logic controller) control module based on the three-dimensional target coordinate position (x', y ', z'), so that the external equipment plug is inserted into the external equipment hole.
6. The method of claim 2, wherein capturing each imaged line of the line laser on the side where the peripheral device hole is located during scanning comprises:
and shooting each imaging line of the line laser on the side face where the external equipment hole is located and parallel to the short edge of the side face where the external equipment hole is located in the scanning process.
7. The method of claim 2, wherein scanning the side of the device on which the device aperture is located with a line laser comprises:
and scanning the side surface of the equipment external connection hole by using blue line laser.
8. The method of claim 1, wherein converting the three-dimensional coordinate position of the peripheral device hole to a three-dimensional target coordinate position of a peripheral device plug to be inserted into the peripheral device hole based on a preset spatial coordinate conversion formula comprises:
recognizing the shape of the external equipment hole from the photo;
determining a corresponding preset space coordinate conversion formula based on the shape;
and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
9. An apparatus for automatically inserting into a hole of an external device, wherein the apparatus comprises:
the identification module is used for shooting a picture of the side face where the external equipment hole on the local equipment is located, identifying the three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and the moving module is used for controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
10. The apparatus of claim 9, wherein the identification module is configured to scan a side surface of the local device where the peripheral device hole is located with a line laser; shooting each imaging line of the line laser on the side surface where the external equipment hole is located in the scanning process; splicing all the imaging lines to form a three-dimensional point cloud outline of the side face where an external equipment hole of the equipment is located; obtaining a three-dimensional coordinate position in the picture of the external equipment hole based on the three-dimensional point cloud outline of the side surface where the external equipment hole is located, wherein the three-dimensional coordinate position comprises a plane coordinate position (x, y) and a depth coordinate position z; converting the plane coordinate position (x, y) and the depth coordinate position z of the external equipment hole into a three-dimensional target coordinate position (x ', y ', z ') of the external equipment plug based on a preset space coordinate conversion formula;
the moving module is used for controlling the plug of the external equipment to be inserted into the external equipment hole through a PLC control module based on the three-dimensional target coordinate position (x ', y ', z ').
11. The device of claim 10, wherein the identification module is configured to obtain a three-dimensional point cloud profile ps1 of a side surface where the peripheral device hole is located; performing normal vector filtering on the three-dimensional point cloud profile ps1, if an included angle between a normal vector of a certain point in the three-dimensional point cloud profile ps1 and the vertical direction of the three-dimensional point cloud profile ps1 exceeds a preset threshold a1, discarding the point, otherwise, reserving the point, and finally obtaining a filtered point cloud ps2 from all reserved points; performing depth map projection on the point cloud ps2 to obtain a depth image d; extracting each cavity region contour from the depth image d, and taking the maximum cavity region contour extracted from the depth image d as a contour c1 of an external device hole; traversing the coordinates of each point in the contour c1, and acquiring the corresponding three-dimensional coordinate value (x, y, z) of each point in the point cloud ps2 according to the coordinates of each point in the contour c 1; the three-dimensional coordinate values (x, y, z) of all points in the contour c1 are added and averaged to obtain the center position of the circumscribed device hole contour c 1.
12. The apparatus of claim 11, wherein the recognition module is configured to convert the center position of the profile c1 of the peripheral device hole into a three-dimensional target coordinate position (x ', y ', z ') of the peripheral device plug based on a preset spatial coordinate conversion formula.
13. The apparatus according to claim 12, wherein the moving module is configured to control, by the PLC control module, the first moving device to move the external device plug to the three-dimensional target coordinate position (x ', y', z ') based on the three-dimensional target coordinate position (x', y ', z'), so that the external device plug is inserted into the external device hole.
14. The device of claim 10, wherein the recognition module is configured to capture each imaging line of the line laser on the side where the peripheral device hole is located, the imaging line being parallel to the short side of the side where the peripheral device hole is located in the scanning process.
15. The apparatus of claim 10, wherein the identification module is configured to scan a side of the device-external hole of the device using a blue-line laser.
16. The apparatus of claim 9, wherein the recognition module is configured to recognize a shape of the peripheral device hole from the photograph; determining a corresponding preset space coordinate conversion formula based on the shape; and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on the determined preset space coordinate conversion formula.
17. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
18. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
shooting a picture of a side face where an external equipment hole on the local equipment is located, identifying a three-dimensional coordinate position of the external equipment hole in the picture from the picture, and converting the three-dimensional coordinate position of the external equipment hole into a three-dimensional target coordinate position of an external equipment plug to be inserted into the external equipment hole based on a preset space coordinate conversion formula;
and controlling the plug of the external equipment to move to the three-dimensional target coordinate position so as to insert the plug of the external equipment into the hole of the external equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010072741.6A CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010072741.6A CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111273704A true CN111273704A (en) | 2020-06-12 |
| CN111273704B CN111273704B (en) | 2023-09-15 |
Family
ID=70999099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010072741.6A Active CN111273704B (en) | 2020-01-21 | 2020-01-21 | Method and device for automatically inserting external equipment hole |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111273704B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069420A (en) * | 2015-07-27 | 2015-11-18 | 上海振华重工(集团)股份有限公司 | Automatic container lockhole identifying-positioning method and system |
| WO2016185637A1 (en) * | 2015-05-20 | 2016-11-24 | 三菱電機株式会社 | Point-cloud-image generation device and display system |
| CN107578464A (en) * | 2017-06-30 | 2018-01-12 | 长沙湘计海盾科技有限公司 | A kind of conveyor belt workpieces measuring three-dimensional profile method based on line laser structured light |
| CN107834614A (en) * | 2017-07-28 | 2018-03-23 | 张涛 | A kind of charging method for adapting to different model mobile phone automatic charging |
| CN108981604A (en) * | 2018-07-11 | 2018-12-11 | 天津工业大学 | A kind of precision component three-dimensional overall picture measurement method based on line laser |
| CN109977954A (en) * | 2019-04-01 | 2019-07-05 | 上海电气集团股份有限公司 | The identification of electric vehicle charge interface and localization method and system |
-
2020
- 2020-01-21 CN CN202010072741.6A patent/CN111273704B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016185637A1 (en) * | 2015-05-20 | 2016-11-24 | 三菱電機株式会社 | Point-cloud-image generation device and display system |
| CN105069420A (en) * | 2015-07-27 | 2015-11-18 | 上海振华重工(集团)股份有限公司 | Automatic container lockhole identifying-positioning method and system |
| CN107578464A (en) * | 2017-06-30 | 2018-01-12 | 长沙湘计海盾科技有限公司 | A kind of conveyor belt workpieces measuring three-dimensional profile method based on line laser structured light |
| CN107834614A (en) * | 2017-07-28 | 2018-03-23 | 张涛 | A kind of charging method for adapting to different model mobile phone automatic charging |
| CN108981604A (en) * | 2018-07-11 | 2018-12-11 | 天津工业大学 | A kind of precision component three-dimensional overall picture measurement method based on line laser |
| CN109977954A (en) * | 2019-04-01 | 2019-07-05 | 上海电气集团股份有限公司 | The identification of electric vehicle charge interface and localization method and system |
Non-Patent Citations (1)
| Title |
|---|
| 谭小群;唐婧仪;于薇薇;李希;孙智金;: "基于线激光扫描和图像处理的基准孔检测技术研究", 现代制造工程, no. 04 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111273704B (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111273704B (en) | Method and device for automatically inserting external equipment hole | |
| TWI740087B (en) | Two-dimensional code reading method, device and equipment | |
| CN111311749B (en) | Method and device for automatically inserting USB hole of equipment | |
| CN100502471C (en) | Image processing device, image processing method and imaging device | |
| CN109190539A (en) | Face identification method and device | |
| CN114663618B (en) | Three-dimensional reconstruction and correction method, device, equipment and storage medium | |
| US20190385285A1 (en) | Image Processing Method and Device | |
| JP2013062803A (en) | Method for determining distance information, method for determining distance map, computer device, imaging system, and computer program | |
| US10839599B2 (en) | Method and device for three-dimensional modeling | |
| CN105578023A (en) | Image quick photographing method and device | |
| CN111325715A (en) | Camera color spot detection method and device | |
| CN111210473A (en) | Mobile phone contour positioning method and equipment | |
| CN111272067A (en) | Mobile phone detection method and equipment | |
| CN110855979A (en) | Camera module detection method and device | |
| US20200151855A1 (en) | Noise processing method and apparatus | |
| JP5857712B2 (en) | Stereo image generation apparatus, stereo image generation method, and computer program for stereo image generation | |
| CN111311678B (en) | USB position positioning method and device | |
| CN110555874B (en) | Image processing method and device | |
| JP6563084B1 (en) | Card number recognition apparatus and card number recognition method | |
| CN116311135A (en) | Data dimension reduction method, data dimension reduction system and controller for semantic information | |
| CN111294453B (en) | Mobile phone key test method and equipment | |
| CN111327745B (en) | Method and equipment for testing silencing key of mobile terminal | |
| CN114140771A (en) | Automatic annotation method and system for image depth data set | |
| KR101741501B1 (en) | Apparatus and Method for Estimation of Distance between Camera and Object | |
| CN115439550A (en) | Camera calibration method, distance measurement method, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant after: Shanghai wanwansheng Environmental Protection Technology Group Co.,Ltd. Address before: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant before: SHANGHAI YUEYI NETWORK INFORMATION TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |