CN110047140B - Unmanned on duty system and intelligent stock ground monitored control system in stock ground - Google Patents
Unmanned on duty system and intelligent stock ground monitored control system in stock ground Download PDFInfo
- Publication number
- CN110047140B CN110047140B CN201910462392.6A CN201910462392A CN110047140B CN 110047140 B CN110047140 B CN 110047140B CN 201910462392 A CN201910462392 A CN 201910462392A CN 110047140 B CN110047140 B CN 110047140B
- Authority
- CN
- China
- Prior art keywords
- point
- point cloud
- cloud data
- stock ground
- subsystem
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/2433—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures for measuring outlines by shadow casting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention relates to an unattended system of a stock ground, which comprises a three-dimensional laser scanner, an automatic stacking and reclaiming subsystem and a material coiling subsystem; the three-dimensional laser scanner is fixedly arranged above the stock yard and scans materials in the stock yard to form a three-dimensional model of the stock yard; the three-dimensional laser scanner is respectively connected with the automatic stacking subsystem, the material taking subsystem and the material checking subsystem; the automatic stacking and reclaiming subsystem completes stacking operation or reclaiming operation based on the three-dimensional model of the stock ground; the material checking subsystem is used for checking the materials in the stock ground based on the three-dimensional model of the stock ground. The invention also discloses an intelligent stock ground monitoring system. The stock yard unattended system and the intelligent stock yard monitoring system can work in full automation and real time.
Description
Technical Field
The invention belongs to the technical field of three-dimensional laser scanning, and particularly relates to an unattended system of a stock ground and an intelligent stock ground monitoring method and system.
Background
At present, the automation degree of a monitoring system of a stock ground is low, the stock ground cannot be accurately counted in real time, and in addition, when materials are piled and taken, the operation precision and the efficiency are low; in addition, the existing stock ground monitoring system does not organically combine systems such as material checking, man-machine position, environmental data of the stock ground, material weight and mixed burning, so that the working efficiency is low and a large amount of manpower and material resources are consumed.
Therefore, in view of the problems existing in the existing stockyard monitoring system, it is necessary to establish an unattended system and an intelligent stockyard monitoring system for a stockyard to solve the problems existing in the prior art.
Disclosure of Invention
In view of the above, the present invention provides an unattended system of a stock ground and an intelligent stock ground monitoring method and system, so that the unattended system and the intelligent stock ground monitoring system can fully automatically operate.
In a first aspect, an embodiment of the present invention provides an unattended system for a stock ground, where the system includes a three-dimensional laser scanner, an automatic stacking and reclaiming subsystem, and a material checking subsystem;
the three-dimensional laser scanner is fixedly arranged above the stock yard and scans materials in the stock yard to form first point cloud data; the three-dimensional laser scanner is respectively connected with the automatic stacking subsystem, the material taking subsystem and the material checking subsystem;
the automatic stacking and taking subsystem comprises a cleaning module, a splicing module, a removing module, a filling module, a modeling module, an angle alignment module, a starting point acquisition module, an end point acquisition module, a stacker and a reclaimer;
the cleaning module is used for cleaning the first point cloud data to form second point cloud data;
the splicing module is used for splicing the second point cloud data to obtain third point cloud data;
the three-dimensional laser scanner acquires point cloud data of a material stacking arm on a blanking accumulation surface of a material;
the removing module is used for removing the point cloud data of the stacking arm from the third point cloud data to obtain fourth point cloud data;
the filling module is used for performing point cloud filling on a blank area in the second point cloud data to obtain filling point cloud data, and combining the fourth point cloud data with the filling point cloud data to obtain fifth point cloud data;
the modeling module is used for carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point;
the angle alignment module is used for acquiring an alignment angle of operation of the stacker according to the three-dimensional model of the stock ground, and the stacker completes stacking operation based on the alignment angle;
when the stacking operation is carried out, the angle alignment module is used for acquiring the alignment angle of the operation of the stacker according to the three-dimensional model of the stock ground, and the stacker finishes the stacking operation based on the alignment angle;
when material taking operation is carried out, the starting point acquisition module and the end point acquisition module respectively acquire a starting point and an end point of the operation of the material taking machine according to the three-dimensional model of the stock ground, and the material taking machine finishes the material taking operation according to the starting point and the end point of the operation.
The material checking subsystem comprises a holder and a second scanner, wherein the second scanner checks materials in the stock ground according to the three-dimensional model of the stock ground.
In a second aspect, an embodiment of the present invention provides an intelligent stockyard monitoring system, which includes the above unattended system.
The invention scans the point cloud data of the stock ground materials through the three-dimensional laser scanner fixed above the stock ground to establish a three-dimensional model of the stock ground materials in real time, no material piling and taking machine participates in the scanning process, the three-dimensional laser scanner can independently run in real time, the working time is saved, and the piling and taking operation can be more accurately carried out by acquiring the alignment angle of the material piling machine or the starting point and the ending point of the material taking machine, so that the unattended system and the intelligent stock ground monitoring system of the stock ground can fully automatically work.
Drawings
Fig. 1 is a schematic flow chart of an automatic stacking method for a material yard according to an embodiment of the present invention;
FIG. 2 is a schematic view of the highest point of material in the working area of the stocker during the stacking process according to the embodiment of the present invention;
FIG. 3 is a side schematic view of the stacker of an embodiment of the present invention having been stacker chopped;
fig. 4 is a schematic flow chart of an automatic material taking method for a material yard according to an embodiment of the present invention;
FIG. 5 is a schematic view of the highest point of the material in the operation area of the reclaimer during the reclaiming process according to the embodiment of the present invention;
fig. 6A is a schematic top view of a circular stockyard according to an embodiment of the present invention;
fig. 6B is a schematic cross-sectional view of a circular stockyard according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an intelligent stock ground monitoring system according to an embodiment of the present invention.
Detailed Description
The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and that functional, methodological, or structural equivalents thereof, which are equivalent or substituted by those of ordinary skill in the art, are within the scope of the present invention.
1. Unmanned on duty system in stock ground
The unattended system of the stock ground in the embodiment comprises a three-dimensional laser scanner, an automatic stacking and taking subsystem and a material coiling subsystem;
the three-dimensional laser scanner is fixedly installed above a stock ground and scans materials in the stock ground to form first point cloud data; the three-dimensional laser scanner is respectively connected with the automatic stacking subsystem, the material taking subsystem and the material checking subsystem;
the automatic stacking and taking subsystem comprises a cleaning module, a splicing module, a removing module, a filling module, a modeling module, an angle alignment module, a starting point acquisition module, an end point acquisition module, a stacker and a reclaimer;
the cleaning module is used for cleaning the first point cloud data to form second point cloud data;
the splicing module is used for splicing the second point cloud data to obtain third point cloud data;
the three-dimensional laser scanner acquires point cloud data of a material stacking arm on a blanking accumulation surface of a material;
the removing module is used for removing the point cloud data of the material piling arm from the third point cloud data to obtain fourth point cloud data;
the filling module is used for performing point cloud filling on a blank area in the second point cloud data to obtain filling point cloud data, and combining the fourth point cloud data with the filling point cloud data to obtain fifth point cloud data;
the modeling module is used for carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point;
the angle alignment module is used for acquiring an alignment angle of operation of the stacker according to the three-dimensional model of the stock ground, and the stacker completes stacking operation based on the alignment angle;
when the stacking operation is carried out, the angle alignment module is used for acquiring the alignment angle of the operation of the stacker according to the three-dimensional model of the stock ground, and the stacker finishes the stacking operation based on the alignment angle;
when material taking operation is carried out, the starting point acquisition module and the end point acquisition module respectively acquire a starting point and an end point of the operation of the material taking machine according to the three-dimensional model of the stock ground, and the material taking machine finishes the material taking operation according to the starting point and the end point of the operation.
The material checking subsystem comprises a holder and a second scanner, wherein the second scanner checks materials in the stock ground according to the three-dimensional model of the stock ground; namely, the volume and the weight of the material are obtained according to the three-dimensional model of the stock ground.
The second scanner may be a three-dimensional laser scanner, or may be another type of scanner.
Further, the unattended system of the stock ground also comprises a coordinate system establishing module, which is used for establishing a space three-dimensional coordinate system of the stock ground before the stock ground materials are scanned, and dividing the space of the space three-dimensional coordinate system of the stock ground into a plurality of cubic grids; the three-dimensional laser scanner scans a three-dimensional coordinate system in a three-dimensional space, wherein each scanning area of the three-dimensional laser scanner corresponds to a group of cubic grids in the three-dimensional coordinate system; and
and the cleaning module detects outliers in the first point cloud data in the space three-dimensional coordinate system of the stock ground, and removes the outliers from the first point cloud data to obtain the second point cloud data.
2. Concrete working method of automatic stacking and reclaiming subsystem
The following will mainly describe a specific working method of an automatic stacking and reclaiming subsystem in an unattended system of the stockyard, and the method comprises the following steps:
scanning the material in the stock ground by using a three-dimensional laser scanner fixedly arranged above the stock ground to form first point cloud data of the material;
carrying out three-dimensional modeling on the stock ground material according to the first point cloud data and a preset reference characteristic point to form a three-dimensional model of the stock ground;
according to the three-dimensional model of the stock ground, the stacking and reclaiming operation is completed;
wherein the three-dimensional modeling comprises the following sub-steps:
a cleaning step: cleaning the first point cloud data to form second point cloud data;
splicing: splicing the second point cloud data to obtain third point cloud data;
removing: acquiring point cloud data of the material arm on a blanking accumulation surface of the material, and removing the point cloud data of the material arm from the third point cloud data to obtain fourth point cloud data;
filling: point cloud filling is carried out on blank areas in the second point cloud data to obtain filling point cloud data, and the fourth point cloud data and the filling point cloud data are combined to obtain fifth point cloud data;
modeling: carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point;
when the stacking operation is carried out, the angle alignment module is used for acquiring the alignment angle of the operation of the stacker according to the three-dimensional model of the stock ground, and the stacker finishes the stacking operation based on the alignment angle;
when material taking operation is carried out, the starting point acquisition module and the end point acquisition module respectively acquire a starting point and an end point of the operation of the material taking machine according to the three-dimensional model of the stock ground, and the material taking machine finishes the material taking operation according to the starting point and the end point of the operation.
The material arm includes a stacking arm of a stacker and a material taking arm of a reclaimer.
Since the operation processes of the stacker and the reclaimer are not completely the same in the stacking process and the reclaiming process, the different method steps of stacking and reclaiming will be separately described below.
(one) automatic stockpiling method for material yard
1. As shown in fig. 1, fig. 1 is a schematic flow chart of an automatic stacking method for a material yard according to an embodiment of the present invention; the automatic stacking method comprises the following steps:
scanning the material in the stock yard by using a three-dimensional laser scanner fixedly arranged above the stock yard to form first point cloud data of the material;
carrying out three-dimensional modeling on the stock ground material according to the first point cloud data and a preset reference characteristic point to form a three-dimensional model of the stock ground;
according to the three-dimensional model of the stock ground, acquiring the alignment angle of operation to complete stockpiling operation;
wherein the three-dimensional modeling comprises the following sub-steps:
a cleaning step: cleaning the first point cloud data to form second point cloud data;
splicing: splicing the second point cloud data to obtain third point cloud data;
removing: acquiring point cloud data of the material stacking arm on a blanking accumulation surface of the material, and removing the point cloud data of the material stacking arm from the third point cloud data to obtain fourth point cloud data;
filling: point cloud filling is carried out on blank areas in the second point cloud data to obtain filling point cloud data, and the fourth point cloud data and the filling point cloud data are combined to obtain fifth point cloud data;
modeling: and carrying out three-dimensional modeling on the material yard material according to the fifth point cloud data and a preset reference characteristic point.
2. The following will specifically describe a specific process of three-dimensional modeling of the stock ground material.
(1) A scanning step: scanning the material in the stock yard by using a three-dimensional laser scanner fixedly arranged above the stock yard to form first point cloud data;
specifically, the stock yard can be divided into a plurality of stock yard areas according to the coal types, and the number of the stock yard areas is assigned; one or more three-dimensional laser scanners are fixedly installed above the stock yard, the three-dimensional laser scanners scan the stock yard materials to form first point cloud data, and the first point cloud data and the stock yard area numbers are stored in a database.
Compared with the method for scanning the stock yard in a segmented manner by utilizing mechanical walking to drive the scanners, the method has the advantages that the plurality of three-dimensional laser scanners are fixedly installed above the stock yard, one or more three-dimensional laser scanners can be started to scan the materials locally, all the three-dimensional laser scanners can be started to scan the materials completely, the machine is prevented from running freely, the scanning mode in the embodiment is diversified, the scanning time is saved, and the scanning work efficiency is improved.
(2) A cleaning step: cleaning the first point cloud data to form second point cloud data;
the method for cleaning the first point cloud data may be that outliers in the first point cloud data are detected in a first three-dimensional space coordinate system of the stock yard based on a gaussian distribution method, and the outliers are removed from the first point cloud data to obtain second point cloud data; it is understood that, in other embodiments, other methods may be used to detect outliers in the first point cloud data, and are not limited in this regard.
(3) Splicing: splicing the second point cloud data to obtain third point cloud data;
specifically, the second point cloud data may be placed in a corresponding cubic grid, and the second point cloud data may be spliced to obtain third point cloud data.
(4) Removing: acquiring point cloud data of the material stacking arm on a blanking accumulation surface of the material, and removing the point cloud data of the material stacking arm from the third point cloud data to obtain fourth point cloud data;
wherein, the point cloud data of the material piling arm obtained in the removing step comprises the following steps:
for each first point N on the blanking accumulation plane of the material 0 (x 1 ，y 1 ，z 1 ) Finding the target point N 1 (x 1 ，y 1 ，z 2 ) The target point N 1 Should satisfy point N 1 A distance d from the origin 0 exceeds a preset distance and z 2 -z 1 Whether the height difference is greater than the preset height difference or not;
obtaining N 1 Angle alpha between point and x-axis 1 ：
judgment of alpha 1 Whether it is smaller than a predetermined angle, if alpha 1 If the angle is smaller than the preset angle, the point N is determined 1 As point cloud data on the stacking arm; if α is 1 If the angle is not smaller than the preset angle, searching a next target point; in this embodiment, the preset angle range may be, for example, 3 to 5 degrees, and in other embodiments, the preset angle range may also be other angle ranges, which is not limited herein.
Because the three-dimensional laser scanner can also scan the point cloud data of the stacking arm in the scanning process, the point cloud data of the stacking arm can be obtained through the three-dimensional laser scanner, namely, the point cloud data can be obtained according to any point N on the blanking accumulation surface of the material 1 And judging whether the data on the blanking accumulation surface is the point cloud data of the stacking arm or not at an included angle of 0 of the original point, and removing the point cloud data of the stacking arm from the spliced point cloud data, so that the interference of the point cloud data on the stacking arm on the finally obtained stock ground point cloud data can be avoided.
(5) Filling: performing point cloud filling on a blank area in the second point cloud data by utilizing a trilinear interpolation algorithm or a growing algorithm to obtain filling point cloud data, and combining the fourth point cloud data with the filling point cloud data to obtain fifth point cloud data;
specifically, the cubic grid is traversed to identify a blank area of the second point cloud data in the cubic grid, the blank area is filled with the point cloud within the range of the blanking accumulation surface of the material, and the specific steps of filling the blank area by using a trilinear interpolation algorithm or a growing algorithm are respectively described in detail below.
In this embodiment, the filling step includes the following substeps:
s100, counting the number of data points of second point cloud data contained in each cubic grid corresponding to each cubic grid;
the source of the data point can be used for splicing the second point cloud data, the source of the data point corresponds to a three-dimensional laser scanner fixedly installed above the stock yard, and the three-dimensional laser scanner transmits and displays the source of the data point to equipment of an external computer;
if the data point quantity =1, directly obtaining the coordinates of the unique data point of the cubic grid;
if the data point quantity is greater than 1, obtaining the coordinates of a plurality of data points of the cubic grid through an Inverse Distance Weighted method (IDW-Inverse Distance Weighted), and taking the coordinates of the plurality of data points as a characteristic operator in the cubic grid;
if the number of data points =0, the cubic grid is added to the blank sequence group BKArray to form blank areas in the second point cloud data, which may be understood as including one or more blank areas.
S200, for the cubic grid with the data point number of more than 1 or =1, obtaining a corresponding local surface normal vector according to the coordinates (namely, characteristic operators) of the data points in the cubic grid;
specifically, an area range may be preset, for example, the radius of the area range is 0.5 m to 1 m, all data points of the cube grid belonging to the area range are obtained, and the corresponding local surface normal vector is obtained according to the coordinates of the data points.
S300, acquiring a change value of a normal vector of a data point according to the local surface normal vector; in this embodiment, a Poisson (Poisson) surface reconstruction algorithm may be used to obtain a variation value of a normal vector of the data point.
And the change values of the normal vectors of the data points are the change values of the normal vectors in the front, back, left and right directions of each data point in the area range.
And S400, sorting the data points in the second point cloud data according to the change value of the normal vector of the data points.
S500, acquiring a data point corresponding to the minimum value in the variation values of the normal vectors of the data points, and taking the data point corresponding to the minimum value as an initial seed point P; wherein, the region where the starting seed point P is located is the smoothest region.
And S600, filling an empty area based on the initial seed point P.
In one embodiment, the filling of the blank region based on the starting seed point P is implemented by a tri-linear interpolation algorithm.
In another embodiment, the filling of the blank region based on the starting seed point P is performed by filling the blank region by using a growth algorithm, and the sub-step of filling the blank region by using the growth algorithm will be specifically described below;
s610, for each blank area, acquiring an included angle between a normal vector of a data point in a cubic grid with the data volume being more than 1 or =1 and a normal vector of a starting seed point P, and if the included angle is smaller than a preset smooth threshold value, adding the starting seed point P into the blank area;
s620, the steps are executed until all blank areas (namely the corresponding blank sequence group BKArray) are empty, and filling point cloud data are obtained.
It should be noted that one or more neighborhood points may be used as a filling point cloud in a blank area.
Through the specific filling steps of the two embodiments, the point cloud of the finally obtained fifth point cloud data can be complete, no blank area is left, so that the finally obtained three-dimensional model of the stock ground is more accurate, the stacker and the reclaimer can be more accurately controlled through the data after three-dimensional scanning imaging, the operation arm of the stacker and the reclaimer directly contacts the operation surface, and the stacker and the reclaimer do not need to run empty.
(6) Modeling: and carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point.
The preset reference feature point may be set according to an empirical value, and the preset reference feature point may be, for example, a stock yard gate, an access way, or the like.
In this embodiment, a three-dimensional model of the stock ground can be obtained by performing three-dimensional modeling on the stock ground materials by using a Delaunay triangulation network method according to the fifth point cloud data and the preset reference feature points, which is not described herein again.
3. The specific steps of the stacker acquiring the alignment angle of the job during the stacking process will be described in detail below.
In this embodiment, a three-dimensional coordinate system using the material center as an origin may be pre-established and recorded as a second three-dimensional space coordinate system, and the second three-dimensional space coordinate system may be measured and determined by a theodolite.
(1) Obtaining the inclination angle value alpha of the highest point of the materials in the operation area of the stacker by a bubble sorting method based on the point cloud data of the materials 1 (ii) a The inclination angle alpha of the highest point of the material 1 ＝arc tan[(z c -z b )/(x c -x b )]；
Referring to fig. 2, point C is the highest point on the material accumulation plane, and the coordinate of point C in the second three-dimensional space coordinate system is (x) c ,y c ,z c ) (ii) a The B point is a projection point of the C point, and the coordinate of the B point in the second three-dimensional space coordinate system is (x) b ,y b ,z b ) (ii) a Point A is the origin of the second three-dimensional space coordinate system; alpha is alpha 1 The inclination angle of the highest point of the material is shown; wherein, the coordinate of the point C can be obtained by scanning according to a three-dimensional laser scanner.
(2) Reading the polar coordinate position (theta, beta) of the stacking arm in a first three-dimensional space coordinate system;
in this embodiment, a ControlLogix series PLC controller of AB company, for example, 1756; alternatively, a PLC controller such as S7-300 from Siemens may be used to read the polar position (θ, β) of the stacking arm in the first three-dimensional spatial coordinate system.
(3) By using the inclination angle alpha of the highest point of the material 1 And the coordinate (x) of the highest point C on the material accumulation area c ，y c ，z c ) And the polar coordinate position (theta, beta) of the stacking arm in the first three-dimensional space coordinate system, and the current position of the material machine in the second three-dimensional space coordinate system is obtained by using the stacking arm position picked from the point cloud data of the material, directly reading an encoder from a PLC (programmable logic controller) or through a UWB (Ultra Wideband-carrier-free communication) wireless positioning method.
(4) The stocker moves from the current position to the operation start position of the stocker, which is obtained according to the operation area of the stocker, the operation direction of the stocker (e.g., clockwise or counterclockwise), and according to the preset stack height value of the stack (e.g., the stack height value of the stack may be 16 meters) and the stack angle of the stack.
In this embodiment, the operation start position of the stacker is a conical stack surface in the process of stacking operation of the stacker, the highest point of the conical stack surface is the operation start position of the stacker, and the operation start position of the stacker is obtained to enable the stacking start point of the stacker to be more accurate, so that the empty running of the stacker on an operation site is prevented, and the work efficiency of the stacker is improved.
(5) And finishing the stacking operation according to the alignment angle and the operation initial position of the stacker.
During the stacking operation, the height of the material pile of the current blanking point and the repose angle of the stacked material pile can be obtained due to the real-time modeling of the stock ground by the three-dimensional laser scanner
The stacking arm of the stacker operates towards a preset direction from an operation starting position, the real-time height of the stack can be obtained after a real-time model of a stock ground is obtained through the three-dimensional laser scanner, and when the real-time height of the stack reaches a preset stacking height (the preset stacking height can be 15-20 meters for example), the stacking arm depends on the repose angle of the stack
Is based on the mean value->
Moving to the next material piling point until the material is fully piled to finish the material piling operation, and then combining the average value>
Marking as the alignment angle of the stacker; see fig. 3, in which a mean value is taken ≥>
The method comprises the step of acquiring the repose angle of all the points of the piled material stack in the local range of the piling arm (such as +/-10-15 degrees of the piling arm)>
According to the angle of repose->
The mean value of the angle of repose is obtained>
(II) automatic material taking method for material yard
1. Steps of automatic material taking method for material yard
As shown in fig. 4, fig. 4 is a schematic flow chart of an automatic material taking method for a material yard according to an embodiment of the present invention; the automatic material taking method comprises the following steps:
scanning the material in the stock ground by using a three-dimensional laser scanner fixedly arranged above the stock ground to form first point cloud data of the material;
carrying out three-dimensional modeling on the stock ground material according to the first point cloud data and a preset reference characteristic point to form a three-dimensional model of the stock ground;
according to the three-dimensional model of the stock ground, the material taking machine obtains a starting point and an end point of operation, and material taking operation is completed based on the starting point and the end point;
wherein the three-dimensional modeling comprises the sub-steps of:
a cleaning step: cleaning the first point cloud data to form second point cloud data;
splicing: splicing the second point cloud data to obtain third point cloud data;
removing: acquiring point cloud data of the material taking arm on a blanking accumulation surface of the material, and removing the point cloud data of the material taking arm from the third point cloud data to obtain fourth point cloud data;
filling: performing point cloud filling on a blank area in the second point cloud data to obtain filling point cloud data, and combining the fourth point cloud data with the filling point cloud data to obtain fifth point cloud data; in this embodiment, the blank area in the second point cloud data may be point cloud filled by using a trilinear interpolation algorithm or a growing algorithm to obtain the filled point cloud data;
modeling: and carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point.
In the automatic material taking method, the method for performing three-dimensional modeling on the material yard material is basically consistent with the automatic material taking method, and is not described herein again.
2. The following will specifically describe the specific steps of obtaining the start point and the end point of the operation in the material taking process.
(1) Obtaining the inclination angle value alpha of the highest point of the material in the operating area of the reclaimer by a bubbling sorting method based on the point cloud data of the material 1 '; the inclination angle alpha of the highest point of the material 1 ′＝arc tan[(z c ′-z b ′)/(x c ′-x b ′)]；
Referring to fig. 5, point C is the highest point on the material accumulation area, and point B is the projection point of point C on the horizontal plane; point a is the origin of the second coordinate system; alpha is alpha 1 ' is the inclination angle of the highest point of the material; the coordinate of the point C' in the second three-dimensional space coordinate system is (x) c ′,y c ′,z c '), and the coordinate of the point B' in the second three-dimensional space coordinate system is (x) b ′,y b ′,z b ') where the coordinates of point C' can be obtained from a three-dimensional laser scanner scan.
(2) Reading the polar coordinate position (theta ', beta') of the material taking arm in a first three-dimensional space coordinate system;
in this embodiment, a ControlLogix series PLC controller from AB company, for example, 1756; alternatively, a PLC controller such as Siemens S7-300 may be employed to read the polar position (θ ', β') of the take-off arm in the first coordinate system.
(3) According to the materialAngle of inclination alpha of the highest point 1 'and the highest point C' of the material accumulation plane c ′,y c ′,z c ') and the polar coordinate position (theta ', beta ') of the material taking arm in the first three-dimensional space coordinate system to obtain the current position of the material taking machine in the second three-dimensional space coordinate system;
(4) In this embodiment, the starting point of the operation of the reclaimer can be obtained according to the coordinate of the highest point C' on the material stacking area:
taking out a point D 'at the middle position of the point C' and the point B 'on the material accumulation surface in the vertical direction, and forming a material taking area of a material taking arm of the material taking machine by taking the point D' as a circle center and taking the distance between the point D 'and the point C' as a radius, wherein the material taking arm of the material taking machine is tangent to the material taking area;
setting a coordinate value z value of each point in the point cloud data of the material as 0, and storing the value into a two-dimensional coordinate system, wherein the two-dimensional coordinate system is the projection of a three-dimensional model of the material yard material to the direction of a plane parallel to the traveling direction of the reclaimer, and two clusters of intersection point groups are formed on the plane and are respectively a first intersection point group and a second intersection point group;
fig. 6A and 6B are schematic top view and schematic cross-sectional view of a circular stock ground according to an embodiment of the present invention, where an angle range of the circular stock ground is 0 to 360 °, the angle range corresponds to a rotation angle of a large arm of a reclaimer, for example, an angle of a point P is an initial rotation angle of reclaiming, and the reclaimer starts to perform reclaiming operation, and is configured with parameters such as a rotation speed of a scraper reclaimer, a rotation angle of the scraper reclaimer, a downward depression angle of the scraper reclaimer, and a material level detection switch signal, and according to these parameters, various reclaiming operation modes such as constant-flow reclaiming can be supported.
The first intersection point group is the closest intersection point group along the advancing direction of the reclaimer, namely, the D 'points to be taken out are projected into the two-dimensional coordinate system, and then the D' points form a closed ring (namely, the circumference of a gray area in fig. 6B) in the two-dimensional coordinate system, referring to fig. 6B, the advancing direction of the material taking arm intersects with the closed ring twice, and the intersection point group obtained by the intersection of the current advancing direction of the material taking arm and the closed ring is the first intersection point group;
and the second intersection point group is an intersection point group obtained by intersecting the next advancing direction of the material taking arm with the closed ring.
The coordinate values of the point clouds on the first intersection point group are averaged to obtain a coordinate (x) e ,y e ,z e ) The point E is taken as the material taking starting point of the material taking machine on the current material taking layer;
further, the coordinate values of the point clouds on the second intersection point group are averaged to obtain a coordinate (x) f ,y f ,z f ) And taking the point F as a material taking termination point of the material taking machine on the current material taking layer.
In the embodiment, the three-dimensional laser scanner is adopted for real-time modeling, and the specific stacking and taking operation parameters of the stacking and taking machine are controlled in real time, so that the efficiency of the stacking and taking machine during operation is improved;
in addition, in the prior art, most of the operating parameters of the stacker and reclaimer during operation are fixed parameters, and the formed material pile 'peak valley' is also fixed, while the unattended data processing method in the embodiment can configure the parameters of the material piling arm rotation speed, the material piling arm rotation angle, the material piling arm pitching angle, the material level detection switch signal and the like, so that the operation material pile is formed into various material piling operation modes such as a herringbone piling method, a diamond piling method, a Chinese character 'zhong' piling method and the like, and the material pile is formed extremely evenly and evenly.
1. Intelligent stock ground monitoring system
Referring to fig. 7, fig. 7 is a schematic structural diagram of an intelligent stock ground monitoring system according to an embodiment of the present invention, where the intelligent stock ground monitoring system includes the above unattended system.
Furthermore, the intelligent stock ground monitoring system also comprises a man-machine positioning subsystem, a disaster prevention monitoring subsystem, a material machine monitoring subsystem, a coal blending subsystem and an alarm device;
the man-machine positioning subsystem positions stockyard personnel, a stacker and a reclaimer by using a wireless communication module; the man-machine positioning subsystem is connected with the alarm device, and when the distances among the stacking machines, the material taking machines and the stacking/material taking machines and stockyard personnel are smaller than a preset distance value, the alarm device can remind of collision early warning, so that the safety of operation workers in the stockyard is greatly improved;
the disaster prevention monitoring subsystem comprises a temperature sensor and a gas sensor,
the temperature sensor is used for monitoring the temperature of the stock ground, and the gas sensor is used for monitoring harmful gas in the stock ground; the disaster prevention monitoring subsystem is connected with the alarm device; the system realizes the on-line detection of the stock yard environment by a disaster prevention monitoring subsystem, and the disaster prevention monitoring system acquires the environmental data of the stock yard in a bus mode and is used for monitoring the concentration and the temperature of toxic gas and dust in each area of the stock yard; when the monitored environment value exceeds the preset environment value, the alarm device starts to give an alarm, so that the safety of workers in the stock yard is greatly improved;
the material machine monitoring subsystem positions a material conveying device of a stock ground by using a wireless communication module; specifically, the material conveying device can be a material pushing machine;
the batching subsystem is used for proportioning materials in a stock ground according to data obtained by the three-dimensional laser scanner and the man-machine positioning system; the batching subsystem system is used for proportioning materials according to a preset blending proportion, namely, the materials are conveyed to a belt through a feeding machine and a belt conveyor respectively by the material taking machine according to real-time data after three-dimensional modeling and positions of personnel and the material machine obtained by a human-computer positioning system to blend the materials, so that the coal accuracy of the materials is high, and high-precision closed-loop control is realized in the whole batching process.
Further, this intelligence stock ground monitored control system still includes weighing device, weighing device is used for weighing the material, wherein, weighing device is belt weigher or track scale.
Furthermore, the intelligent stock ground monitoring system also comprises a transportation equipment identification device, wherein the transportation equipment identification device identifies the type of the transportation equipment by utilizing an RFID card or a wireless communication module.
Furthermore, the intelligent stock ground monitoring system also comprises a display device, wherein the display device is connected with the man-machine positioning subsystem, the disaster prevention monitoring subsystem, the material machine monitoring subsystem, the weighing device and the transportation equipment identification device; the display device is used for displaying stock ground material data, positions and working states of the stocker and the reclaimer, the temperature of the stock ground, names and concentrations of harmful gases, the position of the material conveying device, the weight of the material or the type of the conveying equipment; preferably, the display device is an LED or LCD display screen.
Furthermore, the intelligent stock ground monitoring system is connected with the stock ground management system through a data interface; the stock yard management system comprises MIS (information management system), SIS (plant-level information monitoring system) or DCS (distributed control system) of the stock yard.
The invention has the following beneficial effects:
(1) According to the invention, the three-dimensional laser fixed above the stock ground scans without the participation of a stacker and a reclaimer in the process of establishing a three-dimensional model of the stock ground material, and the three-dimensional laser scanner can independently run in real time, so that the idle running of the stacker/reclaimer is reduced, and the stacker/reclaimer can carry out full-automatic and high-precision operation;
(2) The stock yard can be accurately checked in real time by utilizing data obtained by three-dimensional laser scanning fixed above the stock yard;
(3) The system organically connects an unattended subsystem, a man-machine positioning subsystem, a disaster prevention monitoring subsystem, a material machine monitoring subsystem, a coal blending subsystem, a weighing device, a transportation equipment identification device and the like of a stock yard; the intelligent management and control of the stock ground are realized, the full-automatic unattended operation of the stock ground is realized, the operation efficiency and the safety of stock ground equipment are greatly improved, and the production cost such as the labor cost and the equipment maintenance cost can be reduced for the stock ground.
(4) The wireless positioning module is used for positioning equipment such as stockyard personnel, stacking and taking machines, material pushing machines and the like, so that real-time management and visual operation of the whole process of inspection work are realized.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. An unattended system of a stock ground is characterized by comprising a three-dimensional laser scanner, an automatic stacking and taking subsystem and a material checking subsystem;
the three-dimensional laser scanner is fixedly arranged above the stock yard and scans materials in the stock yard to form first point cloud data; the three-dimensional laser scanner is respectively connected with the automatic stacking subsystem, the material taking subsystem and the material checking subsystem;
the automatic stacking and taking subsystem comprises a cleaning module, a splicing module, a removing module, a filling module, a modeling module, an angle alignment module, a starting point acquisition module, an end point acquisition module, a stacker and a reclaimer;
the cleaning module is used for cleaning the first point cloud data to form second point cloud data;
the splicing module is used for splicing the second point cloud data to obtain third point cloud data;
the three-dimensional laser scanner acquires point cloud data of a material stacking arm on a blanking accumulation surface of a material;
the removing module is used for removing the point cloud data of the material piling arm from the third point cloud data to obtain fourth point cloud data;
the removal module performs the following operations:
for each first point N on the blanking accumulation surface of the material 0 (x 1 ,y 1 ,z 1 ) Finding a target point N 1 (x 1 ,y 1 ,z 2 ) The target point N 1 Should satisfy point N 1 A distance d from the origin O exceeds a preset distance and z 2 -z 1 Whether the height difference is greater than the preset height difference or not;
obtaining N 1 Angle alpha between point and x-axis 1 ：
judgment of alpha 1 Whether or not less than a predetermined angle, if alpha 1 If the angle is smaller than the preset angle, the point N is determined 1 As point cloud data on the material taking arm; if α is 1 If the angle is not smaller than the preset angle, searching a next target point;
the filling module is used for performing point cloud filling on a blank area in the second point cloud data to obtain filling point cloud data, and combining the fourth point cloud data with the filling point cloud data to obtain fifth point cloud data;
the modeling module is used for carrying out three-dimensional modeling on the material field material according to the fifth point cloud data and a preset reference characteristic point;
the angle alignment module is used for acquiring an alignment angle of operation of the stacker according to the three-dimensional model of the stock ground, and the stacker completes stacking operation based on the alignment angle;
when the stacking operation is carried out, the angle alignment module is used for acquiring the alignment angle of the stacker operation according to the three-dimensional model of the stock ground, and the stacker finishes the stacking operation based on the alignment angle;
when material taking operation is carried out, the starting point acquisition module and the end point acquisition module respectively acquire a starting point and an end point of the operation of the material taking machine according to the three-dimensional model of the stock ground, and the material taking machine finishes the material taking operation according to the starting point and the end point of the operation;
the material checking subsystem comprises a holder and a second scanner, wherein the second scanner checks materials in the stock ground according to the three-dimensional model of the stock ground.
2. The system of claim 1, wherein: the second scanner is a three-dimensional laser scanner.
3. The system of claim 1, wherein: the system comprises a material yard material scanning module, a coordinate system establishing module and a data processing module, wherein the material yard material scanning module is used for scanning material yard materials; the three-dimensional laser scanner scans an area each time, and the area corresponds to a group of cubic grids in a three-dimensional space coordinate system; and
the cleaning module detects outliers in the first point cloud data in the space three-dimensional coordinate system of the stock yard, and removes the outliers from the first point cloud data to obtain the second point cloud data.
4. The system of claim 1, wherein: the fill module performs the following operations:
counting the number of data points of the second point cloud data contained in each cubic grid,
if the data point quantity =1, directly obtaining the coordinates of the unique data point of the cubic grid;
if the number of the data points is greater than 1, obtaining the coordinates of a plurality of data points of the cubic grid through an inverse distance weight method;
if the number of the data points is =0, adding the cubic grid into a blank sequence group to form a blank area in the second point cloud data;
and the number of the first and second groups,
for a cubic grid with data point number >1 or =1, obtaining a corresponding local surface normal vector according to coordinates of the data points in the cubic grid;
acquiring a normal vector change value of the data point according to the local surface normal vector;
sorting according to the magnitude of the normal vector variation value of the data points;
acquiring a data point corresponding to the minimum value in the normal vector variation values of the data point, and taking the data point as an initial seed point P;
filling blank regions based on the starting seed point P.
5. An intelligent stock ground monitoring system, characterized in that the intelligent stock ground monitoring system comprises an unattended system according to any one of claims 1-4.
6. The intelligent stock ground monitoring system according to claim 5, further comprising a man-machine positioning subsystem, a disaster prevention monitoring subsystem, a stock machine monitoring subsystem, a batching subsystem and an alarm device;
the man-machine positioning subsystem positions stockyard personnel, a stacker and a reclaimer by using a wireless communication module and is connected with the alarm device;
the disaster prevention monitoring subsystem comprises a temperature sensor and a gas sensor, wherein the temperature sensor is used for monitoring the temperature of the stock yard, and the gas sensor is used for monitoring harmful gas in the stock yard; the disaster prevention monitoring subsystem is connected with the alarm device;
the material machine monitoring subsystem positions a material conveying device of a stock ground by using a wireless communication module;
and the batching subsystem is used for proportioning the materials in the stock ground according to the data obtained by the three-dimensional laser scanner and the man-machine positioning system.
7. The intelligent stock ground monitoring system as claimed in claim 5, further comprising a weighing device for weighing the material.
8. The intelligent stock ground monitoring system according to claim 5, further comprising a display device, wherein the display device is connected with the human-machine positioning subsystem, the disaster prevention monitoring subsystem, the stock machine monitoring subsystem, the weighing device and the transportation equipment identification device; the display device is used for displaying stock ground material data, positions and working states of the stockers and the reclaimers, the temperature of the stock ground, names and concentrations of harmful gases, the positions of the material conveying devices, the weight of the materials or the types of the conveying equipment.
9. The intelligent stock ground monitoring system according to any one of claims 5-8, characterized in that the intelligent stock ground monitoring system is connected with the stock ground management system through a data interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910462392.6A CN110047140B (en) | 2019-05-30 | 2019-05-30 | Unmanned on duty system and intelligent stock ground monitored control system in stock ground |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910462392.6A CN110047140B (en) | 2019-05-30 | 2019-05-30 | Unmanned on duty system and intelligent stock ground monitored control system in stock ground |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110047140A CN110047140A (en) | 2019-07-23 |
| CN110047140B true CN110047140B (en) | 2023-04-11 |
Family
ID=67284149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910462392.6A Active CN110047140B (en) | 2019-05-30 | 2019-05-30 | Unmanned on duty system and intelligent stock ground monitored control system in stock ground |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110047140B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112305940A (en) * | 2019-07-26 | 2021-02-02 | 华能国际电力股份有限公司海门电厂 | Intelligent monitoring system and method for stacker-reclaimer |
| CN111429504B (en) * | 2020-03-02 | 2022-02-01 | 武汉大学 | Automatic material pile extraction and volume measurement method and system based on three-dimensional point cloud |
| CN111915668B (en) * | 2020-07-28 | 2022-11-11 | 中冶宝钢技术服务有限公司 | Unmanned aerial vehicle auxiliary material pile operation monitoring method, device, terminal and medium |
| WO2022032465A1 (en) * | 2020-08-11 | 2022-02-17 | 武汉深能环保新沟垃圾发电有限公司 | Intelligent control method for waste pool in waste-to-energy plant |
| CN113942844A (en) * | 2021-11-10 | 2022-01-18 | 新疆天池能源有限责任公司 | Remote control system of stacker-reclaimer |
| CN114139793B (en) * | 2021-11-25 | 2023-12-22 | 北京众驰自动化设备有限公司 | Intelligent stock ground centralized management and control system and method |
| CN114803391B (en) * | 2022-05-12 | 2023-11-03 | 北京华能新锐控制技术有限公司 | Unmanned automatic material taking method for bucket wheel machine of intelligent fuel system |
| CN117057966B (en) * | 2023-10-10 | 2024-01-09 | 山东朝辉自动化科技有限责任公司 | Material yard stock management system and method based on digital analysis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103090791A (en) * | 2013-01-08 | 2013-05-08 | 中联重科股份有限公司 | Measurement system, method and device of scattered materials and material piling and taking control system |
| CN109102539A (en) * | 2018-08-15 | 2018-12-28 | 宝钢湛江钢铁有限公司 | A kind of stock yard based on 3 D laser scanning automatically generates stock ground drawing method |
| CN109650090A (en) * | 2019-02-27 | 2019-04-19 | 莱芜钢铁集团电子有限公司 | A kind of reclaimer feeding control method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6773503B2 (en) * | 2016-09-27 | 2020-10-21 | 株式会社トプコン | Laser scanner system and point cloud data registration method |
-
2019
- 2019-05-30 CN CN201910462392.6A patent/CN110047140B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103090791A (en) * | 2013-01-08 | 2013-05-08 | 中联重科股份有限公司 | Measurement system, method and device of scattered materials and material piling and taking control system |
| CN109102539A (en) * | 2018-08-15 | 2018-12-28 | 宝钢湛江钢铁有限公司 | A kind of stock yard based on 3 D laser scanning automatically generates stock ground drawing method |
| CN109650090A (en) * | 2019-02-27 | 2019-04-19 | 莱芜钢铁集团电子有限公司 | A kind of reclaimer feeding control method and device |
Non-Patent Citations (1)
| Title |
|---|
| 金闯 ; 滕树满 ; 周茂涛 ; .一种堆取料机智能化改造方案.金属世界.2018,(第04期),73-77. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110047140A (en) | 2019-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110047140B (en) | Unmanned on duty system and intelligent stock ground monitored control system in stock ground | |
| CN110194375B (en) | Automatic stacking, taking and stacking method and system for material yard | |
| CN103090791B (en) | Measurement system, method and device of scattered materials and material piling and taking control system | |
| CN112722873B (en) | Automatic control system of stacker-reclaimer | |
| US9637887B2 (en) | Reclaimer 3D volume rate controller | |
| CN108981842B (en) | Adhesive tape material pile volume calculation and unbalance loading and material blockage identification method based on laser linear light source assistance | |
| CN109650089B (en) | Anti-collision method for material piling and taking machine and material pile of storage yard | |
| CN110136264B (en) | Three-dimensional laser scanning-based stock ground material modeling method and system | |
| CN102402224A (en) | Automatic guided vehicle and method for drive control of the same | |
| CN109242226A (en) | A kind of intelligence coal yard monitoring method and monitoring system | |
| CN104880149A (en) | Large-size bulk material pile volume measurement method based on stereo image analysis, and equipment thereof | |
| CN108147147A (en) | A kind of material stacking and fetching system of automatic intelligent | |
| CN111830948A (en) | Data sharing system and method for real-time three-dimensional pile imaging among different stock yards | |
| CN108182715A (en) | A kind of material stacking and fetching system with statistics rickyard material information | |
| CN112906127A (en) | Vehicle modeling method, system, medium and equipment based on holder and scanner | |
| CN113885422A (en) | Unmanned intelligent system of bucket wheel machine | |
| CN109592433A (en) | A kind of cargo de-stacking method, apparatus and de-stacking system | |
| KR100672044B1 (en) | Apparatus for measuring box size automatically | |
| CN207585702U (en) | A kind of material stockyard material heap reserves on-line measurement device | |
| CN110940271A (en) | Method for detecting, monitoring and intelligently carrying and installing large-scale industrial manufacturing of ships and the like based on space three-dimensional measurement and control network | |
| CN108100687A (en) | A kind of lower hopper, material stocking device and windrow method | |
| CN115716603B (en) | Bucket wheel machine material taking path planning method and system | |
| CN108045973A (en) | A kind of automatic stockpiling method of windrow system | |
| CN203981109U (en) | The large-scale bulk cargo stockpile cubing equipment of analyzing based on stereopsis | |
| CN115482356A (en) | System and method for determining coal-fired material pile and material taking point |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |