CN112934518A - Automatic spraying device and method based on point cloud - Google Patents
Automatic spraying device and method based on point cloud Download PDFInfo
- Publication number
- CN112934518A CN112934518A CN202110097083.0A CN202110097083A CN112934518A CN 112934518 A CN112934518 A CN 112934518A CN 202110097083 A CN202110097083 A CN 202110097083A CN 112934518 A CN112934518 A CN 112934518A
- Authority
- CN
- China
- Prior art keywords
- spraying
- sprayed
- data
- module
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005507 spraying Methods 0.000 title claims abstract description 330
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000007246 mechanism Effects 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims description 47
- 238000004519 manufacturing process Methods 0.000 claims description 32
- 239000003973 paint Substances 0.000 claims description 21
- 238000004458 analytical method Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 17
- 230000009471 action Effects 0.000 claims description 12
- 238000005259 measurement Methods 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 230000008030 elimination Effects 0.000 claims description 5
- 238000003379 elimination reaction Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims 1
- 239000007921 spray Substances 0.000 description 23
- 230000002159 abnormal effect Effects 0.000 description 15
- 229910000831 Steel Inorganic materials 0.000 description 4
- 238000007592 spray painting technique Methods 0.000 description 4
- 239000010959 steel Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 238000010981 drying operation Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B05—SPRAYING OR ATOMISING IN GENERAL; APPLYING FLUENT MATERIALS TO SURFACES, IN GENERAL
- B05B—SPRAYING APPARATUS; ATOMISING APPARATUS; NOZZLES
- B05B12/00—Arrangements for controlling delivery; Arrangements for controlling the spray area
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B05—SPRAYING OR ATOMISING IN GENERAL; APPLYING FLUENT MATERIALS TO SURFACES, IN GENERAL
- B05B—SPRAYING APPARATUS; ATOMISING APPARATUS; NOZZLES
- B05B13/00—Machines or plants for applying liquids or other fluent materials to surfaces of objects or other work by spraying, not covered by groups B05B1/00 - B05B11/00
- B05B13/02—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work
- B05B13/04—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation
- B05B13/0431—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation with spray heads moved by robots or articulated arms, e.g. for applying liquid or other fluent material to 3D-surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
Abstract
The invention relates to the technical field of intelligent spraying, in particular to an automatic spraying device and method based on point cloud. The automatic spraying device and method based on the point cloud comprise an assembly line conveying belt, wherein an automatic scanning device, a position sensor and a spraying mechanism are arranged on the assembly line conveying belt, the automatic scanning device is connected with an embedded computing unit, and the position sensor is connected with the embedded computing unit and the spraying mechanism.
Description
Technical Field
The invention relates to the technical field of intelligent spraying, in particular to an automatic spraying device and method based on point cloud.
Background
As the industry is upgraded, the demand for steel structural members is gradually increased. The traditional automatic spraying robot adopting the modes of online teaching or offline programming and the like has the defects of poor flexible operation, higher cost, low efficiency, large programming workload, complex and fussy operation, higher requirement on relative positioning precision of workpieces and the like, and has higher professional requirement on workers. Workpieces of different models, different series and different sizes need to be customized with a set of spraying process, are extremely complicated to operate, and cannot adapt to assembly line type mixed spraying of various steel members or parallel spraying of single steel members.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the automatic spraying device and the automatic spraying method based on the point cloud have the advantages that the defects of the prior art are overcome, the operation efficiency is improved, the production cost is saved, and the uniformity of the spraying thickness is improved.
The technical scheme adopted by the invention for solving the technical problem is as follows: the automatic spraying device based on point cloud comprises a production line conveying belt, wherein an automatic scanning device, a position sensor and a spraying mechanism are arranged on the production line conveying belt, the automatic scanning device is connected with an embedded computing unit, and the position sensor is connected with the embedded computing unit and the spraying mechanism;
the assembly line conveyer belt is used for conveying a component to be sprayed and is connected with the embedded computing unit; the assembly line conveyer belt adopts chain drive's gusset form, and the gusset height is 50mm, gusset width 2mm, gusset interval 500mm, and the purpose reduces the conveyer belt as far as possible and treats the area of contact of spraying component bottom, reduces the spraying blind area. The assembly line conveying belt conveys a plurality of components to be sprayed side by side at the speed of 0.01m/s-0.05m/s, each row of the components to be sprayed form a group, the longitudinal distance between the components to be sprayed in each group is kept at about 0.1m, and the transverse distance between the components to be sprayed in each group is correspondingly set according to the height of the components to be sprayed, the position of the automatic scanning device and the like, so that the three-dimensional scanning blind area of the automatic scanning device is preferably as small as possible.
The automatic scanning device is used for collecting the outline data of the components to be sprayed on the assembly line conveyor belt and transmitting the data to the embedded computing unit to prepare the data for the subsequent spraying process computing, the number of the automatic scanning devices is set according to the field angle and the range to be scanned, and the automatic scanning device can adopt two modes of timing scanning or position triggering scanning to transmit the collected outline data of each group of the components to be sprayed on the assembly line conveyor belt to the embedded computing unit;
the position sensor is used for acquiring position and speed information to be sprayed and constructed on the assembly line conveying belt and sending the information to the embedded computing unit for assisting in 3D modeling and longitudinal positioning of the spraying mechanism, the position sensor can adopt an encoder which exceeds 1000 lines to perform high-frequency acquisition on longitudinal position and speed data of a component to be sprayed on the assembly line conveying belt, and the data are transmitted to the embedded computing unit and the spraying mechanical arm through a TTL/HTL communication interface for assisting in 3D modeling and longitudinal position computing of the spraying mechanical arm;
the embedded computing unit is used for receiving data acquired by the automatic scanning device and the position sensor, performing 3D model computation, such as data filtering, data smoothing, 3D modeling, spraying object identification, spraying path planning, spraying time sequence instructions and the like, and sending related spraying instructions to the spraying mechanism;
the spraying mechanism further comprises a cloud computing unit, and the cloud computing unit is used for depicting and recording data images in the spraying process of the spraying mechanism. And recording operator information, equipment on-off time, equipment running state, production resource consumption, production task statistics, information of a component to be sprayed and the like through the cloud computing unit. The cloud computing unit can be deployed locally or at the cloud end, and data images of the whole process of each spraying task of the spraying mechanism are recorded in a depicting mode, such as operator information, equipment on-off time, equipment running state, production resource consumption, production task statistics, component detailed information and the like. The spraying mechanism is suitable for H-shaped components, groove-shaped components, flat-plate components, square components, circular components and simple special profiles, such as end faces, pier plates, rib plates, web plates, mounting legs and the like.
The spraying mechanism comprises a spraying device connected with a spraying mechanical arm, and the spraying mechanical arm and the spraying device are both connected with the embedded computing unit;
the spraying mechanical arm is provided with two degrees of freedom which move in the direction vertical to the assembly line conveying belt and in the direction of the width of the assembly line conveying belt. And the spraying mechanical arm receives a spraying instruction of the embedded computing unit to complete the spraying operation of the component to be sprayed. The embodiment sets the spraying device as a spray gun. Two degrees of freedom that the spraying arm set up cooperate the assembly line conveyer belt degree of freedom of motion, form the 3 degrees of freedom's in space motion form. In the embodiment, 6 spray guns are arranged, wherein 2 spray guns are mainly used for spraying the longitudinal end surfaces of the component to be sprayed, 2 spray guns are used for spraying the transverse left side surfaces and the transverse top surfaces of the component to be sprayed, and 2 spray guns are used for spraying the transverse right side surfaces and the transverse top surfaces of the component to be sprayed.
The automatic scanning device comprises a 2D laser radar, a 3D camera and a 2D laser radar, wherein the scanning frequency of the 2D laser radar is more than 20Hz, the scanning frequency of the 3D laser radar is more than 10Hz, and the frame rate of the 3D camera is more than 5 Hz.
The embedded computing unit comprises a sensor data acquisition module, a spraying object identification and extraction module, a spraying process analysis module, a spraying strategy planning module, a spraying operation analysis module, an action execution module, a database module, a global positioning module, a monitoring service module and a data portrait module.
The sensor module is used for acquiring profile scanning data of a component to be sprayed of the automatic scanning device and measurement data of the position and the speed of the component to be sprayed of the position sensor by the acquisition module through a UDP protocol, unifying each frame of measurement data of the automatic scanning devices from respective position sensor coordinate systems to a spraying coordinate system after space transformation to form a frame of complete profile data, and splicing each frame of scanned profile data in the longitudinal direction by using the data of the position sensor to form a complete three-dimensional model of the component to be sprayed.
The spraying object identification and extraction module is used for carrying out filtering, down-sampling, smoothing, outlier elimination and other processing on the data according to the constructed contour data generated by the sensor data acquisition module, discharging interference on the data due to environment, equipment noise and the like, and increasing the stability and consistency of the scanning data. And then dividing the contour data to calculate the number and geometric characteristic information of the components to be sprayed.
The spraying process analysis module extracts all surfaces to be sprayed of the member to be sprayed according to the existing spraying process parameters, performs parametric representation, and discretizes a spraying path of each spraying period according to a parameter equation of each spraying surface;
the spraying strategy planning module is combined with relevant spraying process parameters to fit a spraying path in each spraying surface and a walking path among the members to be sprayed, so that a spraying track with time and speed characteristics is formed, and the area to be sprayed of each group of members to be sprayed is covered;
the spraying operation analysis module converts the spraying track into related control instructions of spraying equipment defined according to a time sequence, such as a production line conveyor belt control instruction, a spraying mechanical arm control instruction, a spray gun control instruction, a spray painting pressure control instruction, a spray painting type control instruction and the like;
the action execution module executes related control instructions according to a time sequence, controls the conveyer belt of the assembly line, the spraying mechanical arm and the spray gun to reach a certain coordinate position according to the time sequence to complete spraying operation, and controls the paint spraying pressure to change the paint pressure according to the time sequence.
The database module is prestored with a plurality of 3D point cloud model templates for spraying the components to be sprayed, spraying process parameters, such as paint film thickness, spraying speed, flow line speed, longitudinal spraying track interval, spraying pressure, spray gun type and the like, point cloud algorithm super parameters, such as template matching threshold, iteration times, search radius, unit grid number, exterior point threshold, sampling number and the like, employee account authority, employee operation records and system operation information, such as equipment startup and shutdown time, equipment operation state, production resource consumption, production task statistics and the like;
the global positioning module provides longitudinal position service for the reconstruction of the 3D model of the component to be sprayed, plays a role in aligning a 3D model coordinate system and a spraying mechanical arm coordinate system, and enables data to have consistency in different operation space states;
the monitoring service module always monitors the intermediate processing state information of each component of the equipment and the software algorithm; such as abnormal information of each sensor, abnormal information of a spraying mechanical arm, abnormal information of a production line conveying belt, misoperation information, abnormal information of algorithm processing and the like. When the abnormal information occurs, a warning notice is given to the operator.
The data image module dynamically displays each data in a UI form. Such as reconstruction of a 3D point cloud model of a component to be sprayed, materialization of a point cloud plane, discretization of a spraying track, marking of the spraying track, marking of key points of spraying actions, spraying area, predicted paint consumption, equipment operation information and the like.
An automatic spraying method based on point cloud comprises the following steps:
information entry: recording information such as a 3D point cloud model template, spraying process parameters, point cloud algorithm hyper-parameters and the like of a plurality of components to be sprayed into a database module;
conveying a component: placing the components to be sprayed of the same type on a conveyer belt of a production line side by side at a certain interval, and controlling the conveyer belt of the production line to convey at a set speed;
data acquisition: the automatic scanning device scans and acquires profile data of a member to be sprayed passing through the assembly line conveyor belt, the position sensor acquires the position and the advancing speed data of the member to be sprayed, and data information acquired by the automatic scanning device and the position sensor is transmitted to the embedded computing unit;
data processing and component spraying: the embedded computing unit receives data information acquired by the automatic scanning device and the position sensor, performs 3D modeling, spraying object recognition, spraying process analysis, spraying path planning and spraying time sequence designation generation, sends spraying time sequence designation to the spraying mechanical arm and the spraying device, and performs spraying operation.
The data processing and component spraying comprises the following substeps:
s1: the method comprises the steps that a sensor data acquisition module acquires data information acquired by automatic scanning devices and position sensors, after space transformation is carried out, each frame of measurement data of a plurality of automatic scanning devices is unified to a spraying coordinate system from respective sensor coordinate systems to form a frame of complete contour data, the existence of a component to be sprayed is judged by utilizing the change of the characteristics of the frame of contour data, when the component to be sprayed is detected, the position service of a global positioning module is utilized to splice the scanned contour data of each frame in the longitudinal direction, the effect of aligning a 3D model coordinate system and a spraying mechanical arm coordinate system is achieved, a complete three-dimensional model of the component has consistency in different operation space states, and a three-dimensional model of the component to be sprayed is generated;
s2: the three-dimensional model generated at this time still has many noise points due to factors such as environment and equipment noise. The spraying object identification and extraction module firstly separates the profile data of a single component to be sprayed, and then carries out processing such as filtering, down-sampling, smoothing and outlier elimination on the data so as to increase the stability and consistency of the scanned data. Dividing the original overall contour data and then calculating the number and geometric characteristic information of the members to be sprayed;
s3: the spraying process analysis module retrieves the 3D point cloud model template in the matching database module after taking the contour data of a plurality of components to be sprayed, extracts corresponding process parameters after matching successfully, extracts all surfaces to be sprayed of a sprayed workpiece according to the spraying process parameters and carries out parametric representation, and discretizes a spraying path of each spraying period according to a parameter equation of each spraying surface by using a corresponding spraying process algorithm;
s4: the spraying strategy planning module is combined with relevant spraying process parameters to fit a spraying path in each spraying surface and a walking path among all spraying components to form a spraying track with time and speed characteristics and cover a to-be-sprayed area of each group of components;
s5: the spraying operation analysis module converts the spraying track into related control instructions defined according to a time sequence, such as a production line conveyor belt control instruction, a spraying mechanical arm control instruction, a spray gun control instruction, a paint spraying pressure control instruction, a paint spraying type control instruction and the like, and converts high-grade track data based on a space position into low-grade control instructions based on the time sequence;
s6: the assembly line conveyer belt conveys a plurality of scanned components to be sprayed to a spraying mechanical arm operation area and aligns the coordinate system, the action execution module controls the assembly line conveyer belt, the spraying mechanical arm and the spraying device to reach a certain coordinate position according to a time sequence to complete spraying operation according to a low-level control instruction based on the time sequence, and controls the paint spraying pressure to change the paint pressure according to the time sequence.
S7: the monitoring service module monitors relevant spraying information in the spraying process.
Compared with the prior art, the invention has the following beneficial effects:
(1) the flexible automatic spraying operation of the streamline steel member under the industrial environment is realized.
(2) A plurality of members of the same kind can be sprayed in parallel, and the spraying coverage rate, the spraying accuracy and the spraying efficiency are higher.
(3) The requirement on the placement position of the components on the conveyer belt of the assembly line is not high, and the transverse distance of each group of internal components is not too close.
(4) The operation is simplified, and the requirement on the professional quality of operators is not high, so that the operation efficiency is improved, the production cost is saved, and the uniformity of the spraying thickness is improved.
Drawings
FIG. 1 is a block diagram of an embedded computing unit of the present invention.
Fig. 2 is a schematic structural diagram of the present invention.
FIG. 3 is a first schematic view of the installation of the automatic scanning device of the present invention.
FIG. 4 is a second schematic view of the installation of the automatic scanner of the present invention.
Figure 5 is a schematic view of the spray robot of the present invention moving to the left.
Figure 6 is a schematic view of the spray robot of the present invention moving to the right.
FIG. 7 is a schematic view of the front end of the arm of the spray coating apparatus of the present invention.
Figure 8 is a rear end spray view of the spray arm of the present invention.
FIG. 9 is a schematic view of an H-shaped component 3D point cloud model.
In the figure: 1. a production line conveyor belt; 2. an automatic scanning device; 3. a member to be sprayed; 4. spraying mechanical arms; 5. a position sensor; 6. an embedded computing unit; 7. and a cloud computing unit.
Detailed Description
Embodiments of the invention are further described below with reference to the accompanying drawings:
examples
As shown in fig. 1 to 9, the device comprises a production line conveying belt 1, wherein an automatic scanning device 2, a position sensor 5 and a spraying mechanism are arranged on the production line conveying belt 1, the automatic scanning device 2 is connected with an embedded computing unit 6, and the position sensor 5 is connected with the embedded computing unit 6 and the spraying mechanism;
the assembly line conveyer belt 1 is used for conveying a component 3 to be sprayed, and the assembly line conveyer belt 1 is connected with the embedded computing unit 6; the assembly line conveyer belt 1 adopts a chain-driven rib plate form, the height of the rib plate is 50mm, the width of the rib plate is 2mm, and the distance between the rib plates is 500mm, so that the contact area between the conveyer belt and the bottom of the component to be sprayed 3 is reduced as far as possible, and the spraying blind area is reduced. The assembly line conveying belt 1 conveys a plurality of components to be sprayed side by side at a speed of 0.01m/s-0.05m/s, each row of the components to be sprayed form a group, the longitudinal distance between the components to be sprayed in each group is kept at about 0.1m, and the transverse distance between the components to be sprayed in each group is correspondingly set according to the height of the components to be sprayed, the position of the automatic scanning device 2 and the like, so that the three-dimensional scanning blind area of the automatic scanning device 2 is preferably as small as possible.
The automatic scanning device 2 is used for collecting the outline data of the components to be sprayed 3 on the assembly line conveyor belt 1 and transmitting the data to the embedded computing unit 6 to prepare the data for the subsequent spraying process calculation, the number of the automatic scanning devices 2 is set according to the field angle and the range to be scanned, and the automatic scanning device 2 can adopt two modes of timing scanning or position triggering scanning to transmit the collected outline data of each group of the components to be sprayed on the assembly line conveyor belt 1 to the embedded computing unit 6;
the position sensor 5 is used for acquiring position and speed information to be sprayed and constructed on the assembly line conveyor belt 1, sending the information to the embedded computing unit 6 and assisting in 3D modeling and longitudinal positioning of the spraying mechanism, and the position sensor 5 can adopt an encoder exceeding 1000 lines to perform high-frequency acquisition on longitudinal position and speed data of a component 3 to be sprayed on the assembly line conveyor belt 1 and transmits the data to the embedded computing unit 6 and the spraying mechanical arm 4 through a TTL/HTL communication interface and assists in 3D modeling and longitudinal position calculation of the spraying mechanical arm 4;
the embedded computing unit 6 is used for receiving data acquired by the automatic scanning device 2 and the position sensor 5, performing 3D model computation, such as data filtering, data smoothing, 3D modeling, spraying object identification, spraying path planning, spraying time sequence instructions and the like, and sending related spraying instructions to the spraying mechanism;
the spraying mechanism further comprises a cloud computing unit 7, and the cloud computing unit 7 is used for depicting and recording data images in the spraying process of the spraying mechanism. The cloud computing unit 7 records operator information, equipment on-off time, equipment running state, production resource consumption, production task statistics, information of the component to be sprayed and the like. The cloud computing unit can be deployed locally or at the cloud end, and data images of the whole process of each spraying task of the spraying mechanism are recorded in a depicting mode, such as operator information, equipment on-off time, equipment running state, production resource consumption, production task statistics, component detailed information and the like. The spraying mechanism is suitable for H-shaped components, groove-shaped components, flat-plate components, square components, circular components and simple special profiles, such as end faces, pier plates, rib plates, web plates, mounting legs and the like.
The spraying mechanism comprises a spraying device connected with a spraying mechanical arm 4, and the spraying mechanical arm 4 and the spraying device are both connected with an embedded computing unit 6;
the spraying arm 4 is provided with two degrees of freedom that move with assembly line conveyer belt 1 vertical direction and assembly line conveyer belt 1 width direction, and its concrete structure adopts prior art, no longer gives unnecessary details. The spraying mechanical arm 4 receives a spraying instruction of the embedded computing unit 6 to complete the spraying operation of the component to be sprayed. The embodiment sets the spraying device as a spray gun. Two degrees of freedom that the spraying arm 4 set up cooperate assembly line conveyer belt 1 degree of freedom of motion, form the motion form of 3 degrees of freedom in space. In the embodiment, 6 spray guns are arranged, wherein 2 spray guns are mainly used for spraying the longitudinal end surfaces of the component 3 to be sprayed, 2 spray guns are used for spraying the transverse left side surfaces and the transverse top surfaces of the component 3 to be sprayed, and 2 spray guns are used for spraying the transverse right side surfaces and the transverse top surfaces of the component 3 to be sprayed.
The automatic scanning device 2 includes a 2D laser radar whose scanning frequency is 20Hz or more, a 3D laser radar whose scanning frequency is 10Hz or more, and a 3D camera whose frame rate is 5Hz or more.
The embedded computing unit 6 comprises a sensor data acquisition module, a spraying object identification and extraction module, a spraying process analysis module, a spraying strategy planning module, a spraying operation analysis module, an action execution module, a database module, a global positioning module, a monitoring service module and a data portrait module.
The sensor module is used for acquiring profile scanning data of the automatic scanning device 2 and measurement data of the position and the speed of the component to be sprayed by the position sensor 5 through a UDP protocol, unifying each frame of measurement data of the automatic scanning devices 2 from respective position sensor 5 coordinate systems to a spraying coordinate system after space transformation to form a frame of complete profile data, and splicing each frame of scanned profile data in the longitudinal direction by using the data of the position sensor 5 to form a complete three-dimensional model of the component to be sprayed.
The spraying object identification and extraction module is used for carrying out filtering, down-sampling, smoothing, outlier elimination and other processing on the data according to the constructed contour data generated by the sensor data acquisition module, eliminating the interference on the data due to environment, equipment noise and the like, and improving the stability and consistency of the scanning data. And then dividing the contour data to calculate the number and geometric characteristic information of the components 3 to be sprayed.
The spraying process analysis module extracts all the surfaces to be sprayed of the component 3 to be sprayed according to the existing spraying process parameters and carries out parametric representation, and the spraying path of each spraying period is discretized according to the parameter equation of each spraying surface;
the spraying strategy planning module is combined with relevant spraying process parameters to fit a spraying path in each spraying surface and a walking path among the members to be sprayed, so that a spraying track with time and speed characteristics is formed, and the area to be sprayed of each group of members to be sprayed is covered;
the spraying operation analysis module converts the spraying track into related control instructions of spraying equipment defined according to a time sequence, such as a production line conveyor belt control instruction, a spraying mechanical arm 4 control instruction, a spray gun control instruction, a spray painting pressure control instruction, a spray painting type control instruction and the like;
the action execution module executes related control instructions according to a time sequence, controls the conveyor belt of the assembly line, the spraying mechanical arm 4 and the spray gun to reach a certain coordinate position according to the time sequence to complete spraying operation, and controls the paint spraying pressure to change the paint pressure according to the time sequence.
The database module is prestored with a plurality of 3D point cloud model templates for spraying the components to be sprayed, spraying process parameters, such as paint film thickness, spraying speed, flow line speed, longitudinal spraying track interval, spraying pressure, spray gun type and the like, point cloud algorithm super parameters, such as template matching threshold, iteration times, search radius, unit grid number, exterior point threshold, sampling number and the like, employee account authority, employee operation records and system operation information, such as equipment startup and shutdown time, equipment operation state, production resource consumption, production task statistics and the like;
the global positioning module provides longitudinal position service for the reconstruction of the 3D model of the component to be sprayed, plays a role in aligning a 3D model coordinate system and a spraying mechanical arm 4 coordinate system, and enables data to have consistency in different operation space states;
the monitoring service module always monitors the intermediate processing state information of each component of the equipment and the software algorithm; such as abnormal information of each sensor, abnormal information of a spraying mechanical arm, abnormal information of a production line conveying belt, misoperation information, abnormal information of algorithm processing and the like. When the abnormal information occurs, a warning notice is given to the operator.
The data image module dynamically displays each data in a UI form. Such as reconstruction of a 3D point cloud model of a component to be sprayed, materialization of a point cloud plane, discretization of a spraying track, marking of the spraying track, marking of key points of spraying actions, spraying area, predicted paint consumption, equipment operation information and the like.
Example 2
An automatic spraying method based on point cloud comprises the following steps:
information entry: inputting the 3D point cloud model templates of a plurality of components to be sprayed 3, spraying process parameters such as paint film thickness, spraying speed, flow line speed, longitudinal spraying track interval, spraying pressure, spray gun type and the like, and point cloud algorithm super parameters such as template matching threshold, iteration times, search radius, cell number, exterior point threshold, sampling number and the like into a database module;
conveying a component: the components 3 to be sprayed of the same type are placed on the assembly line conveyer belt 1 side by side at a certain interval, and the assembly line conveyer belt 1 is controlled to convey at a set speed;
data acquisition: the automatic scanning device 2 scans and acquires profile data of a component 3 to be sprayed passing through the assembly line conveyor belt 1, the position sensor 5 acquires data of the position and the advancing speed of the component 3 to be sprayed, and data information acquired by the automatic scanning device 2 and the position sensor 5 is transmitted to the embedded computing unit 6;
data processing and component spraying: the embedded computing unit 6 receives data information acquired by the automatic scanning device 2 and the position sensor 5, performs 3D modeling, spraying object recognition, spraying process analysis, spraying path planning and spraying time sequence designation generation, sends spraying time sequence designation to the spraying mechanical arm 4 and the spraying device, and performs spraying operation.
When a group of components 3 to be sprayed reach the detection area of the automatic scanning device 2, profile scanning data of the components 3 to be sprayed are transmitted to the sensor data acquisition module through a UDP protocol, and measurement data of the positions and the speeds of the components 3 to be sprayed of the position sensors are synchronously processed.
The data processing and component spraying comprises the following substeps:
s1: the sensor data acquisition module acquires data information acquired by the automatic scanning devices 2 and the position sensor 5, after space transformation, each frame of measurement data of the automatic scanning devices 2 is unified to a spraying coordinate system from respective sensor coordinate systems to form a frame of complete contour data, the existence of the component 3 to be sprayed is judged by utilizing the change of the characteristics of the frame of contour data, such as characteristic points, geometric attributes, normal properties and the like, when the component 3 to be sprayed is detected, the position service of the global positioning module is utilized to splice the scanned contour data of each frame in the longitudinal direction, the effect of aligning a 3D model coordinate system and a spraying mechanical arm coordinate system is achieved, the complete three-dimensional model of the component has consistency in different operation space states, and the three-dimensional model of the component 3 to be sprayed is generated;
s2: the three-dimensional model generated at this time still has many noise points due to factors such as environment and equipment noise. The spraying object identification and extraction module firstly separates the profile data of the single component 3 to be sprayed, and then carries out processing such as filtering, down-sampling, smoothing and outlier elimination on the data so as to increase the stability and consistency of the scanned data. Dividing the original overall contour data and then calculating the number and geometric characteristic information of the members 3 to be sprayed;
s3: the spraying process analysis module retrieves the 3D point cloud model template in the matching database module after taking the contour data of the plurality of components to be sprayed 3, extracts corresponding process parameters after matching successfully, extracts all surfaces to be sprayed of a sprayed workpiece according to the spraying process parameters and carries out parametric representation, and discretizes a spraying path of each spraying period according to a parameter equation of each spraying surface by using a corresponding spraying process algorithm;
s4: the spraying strategy planning module is combined with relevant spraying process parameters to fit a spraying path in each spraying surface and a walking path among all spraying components to form a spraying track with time and speed characteristics and cover a to-be-sprayed area of each group of components;
s5: the spraying operation analysis module converts the spraying track into related control instructions defined according to a time sequence, such as a production line conveyor belt control instruction, a spraying mechanical arm control instruction, a spray gun control instruction, a paint spraying pressure control instruction, a paint spraying type control instruction and the like, and converts high-grade track data based on a space position into low-grade control instructions based on the time sequence;
s6: the assembly line conveyer belt 1 conveys a plurality of scanned components 3 to be sprayed to the operation area of the spraying mechanical arm 4 and aligns the coordinate system, the action execution module controls the assembly line conveyer belt 1, the spraying mechanical arm 4 and the spraying device to reach a certain coordinate position according to a time sequence to complete spraying operation according to a low-level control instruction based on the time sequence, and controls the paint spraying pressure to change the paint pressure according to the time sequence and the like. The spraying mechanical arm 4 is arranged at the upper position and the lower position of the assembly line conveying belt 1 respectively, and the spraying mechanical arm is provided with one set. Since the automatic scanning device 2 can only reconstruct the contours of the top and side surfaces of the component, but not the contours of the end and bottom surfaces, the setup method 1: aiming at asymmetric components, such as groove-shaped components and the like, under the condition that the data of the side surface and the top surface are known, the process algorithm can automatically fit the data of the bottom surface and the side surface, and the upper spraying mechanical arm and the lower spraying mechanical arm can independently move for spraying; the method 2 comprises the following steps: for symmetrical components, such as H-shaped components, besides the scheme of the method 1, the spraying track of the upper spraying mechanical arm can be planned, and the lower spraying mechanical arm performs follow-up spraying. After the spraying operation of a group of components is completed, the assembly line conveyer belt 1 conveys the sprayed components to a station for manual paint repair, and then the components enter a drying room for drying operation.
S7: the monitoring service module monitors relevant spraying information in the spraying process. The state information of the operation of the spraying device and the software algorithm is monitored by a monitoring service module, such as abnormal information of each sensor, abnormal information of a spraying mechanical arm, abnormal information of a production line conveying belt, misoperation information, abnormal information of algorithm processing, employee authority, employee operation records and system operation information: such as the on-off time of the equipment, the running state of the equipment, the consumption of production resources, the statistics of production tasks and the like, when abnormal information occurs, the alarm notification is given to the operator. The state data of the operation of the spraying device and the software algorithm can be dynamically displayed in a UI form, such as component 3D point cloud model reconstruction, point cloud plane materialization, spraying track discretization, spraying track marking, spraying action key point marking, spraying area, predicted paint consumption, equipment operation information and the like.
Claims (10)
1. The automatic spraying device based on the point cloud is characterized by comprising a production line conveying belt (1), wherein an automatic scanning device (2), a position sensor (5) and a spraying mechanism are arranged on the production line conveying belt (1), the automatic scanning device (2) is connected with an embedded computing unit (6), and the position sensor (5) is connected with the embedded computing unit (6) and the spraying mechanism;
the assembly line conveying belt (1) is used for conveying a component (3) to be sprayed, and the assembly line conveying belt (1) is connected with the embedded computing unit (6);
the automatic scanning device (2) is used for acquiring the outline data of the component (3) to be sprayed on the assembly line conveying belt (1) and transmitting the data to the embedded computing unit (6);
the position sensor (5) is used for acquiring position and speed information to be sprayed and constructed on the assembly line conveying belt (1) and sending the information to the embedded computing unit (6);
the embedded computing unit (6) is used for receiving data acquired by the automatic scanning device (2) and the position sensor (5), computing a 3D model and sending a relevant spraying instruction to the spraying mechanism;
the spraying mechanism further comprises a cloud computing unit (7), and the cloud computing unit (7) is used for depicting and recording data images in the spraying process of the spraying mechanism.
2. The automatic spraying device based on point cloud according to claim 1, characterized in that the spraying mechanism comprises a spraying device connected with a spraying mechanical arm (4), and the spraying mechanical arm (4) and the spraying device are both connected with the embedded computing unit (6);
the spraying mechanical arm (4) is provided with two degrees of freedom which move in the direction vertical to the assembly line conveying belt (1) and in the width direction of the assembly line conveying belt (1).
3. The point cloud based automatic spraying device according to claim 1, characterized in that the automatic scanning device (2) comprises a 2D lidar, a 3D lidar and a 3D camera.
4. The automatic spraying device based on point cloud of any one of claims 1-3, characterized in that the embedded computing unit (6) comprises a sensor data acquisition module, a spraying object identification and extraction module, a spraying process analysis module, a spraying strategy planning module, a spraying operation analysis module, an action execution module, a database module, a global positioning module, a monitoring service module, and a data portrait module.
5. The automatic spraying device based on point cloud of claim 4, wherein the sensor module is configured to obtain, by the acquisition module, profile scanning data of the component to be sprayed by the automatic scanning device (2) and measurement data of the position and the speed of the component to be sprayed by the position sensor (5) through a UDP protocol, unify each frame of measurement data of the plurality of automatic scanning devices (2) from respective coordinate systems of the position sensor (5) to a spraying coordinate system after spatial transformation, form a complete frame of profile data, and splice the profile data scanned by each frame in the longitudinal direction by using the data of the position sensor (5), so as to form a complete three-dimensional model of the component to be sprayed.
6. The automatic spraying device based on point cloud of claim 5, wherein the spraying object identification and extraction module is used for carrying out filtering, downsampling, smoothing and outlier removing on the data according to the constructed contour data generated by the sensor data acquisition module, and then carrying out segmentation on the contour data to calculate the number and geometric characteristic information of the components (3) to be sprayed.
7. The automatic spraying device based on point cloud of claim 6, characterized in that the spraying process analysis module extracts all the surfaces to be sprayed of the component (3) to be sprayed according to the existing spraying process parameters and represents them in a parameterized manner, and the spraying path of each spraying period is discretized according to the parameter equation of each spraying surface;
the spraying strategy planning module is combined with relevant spraying process parameters to fit a spraying path in each spraying surface and a walking path among the members to be sprayed, so that a spraying track with time and speed characteristics is formed, and the area to be sprayed of each group of members to be sprayed is covered;
the spraying operation analysis module converts the spraying track into a spraying equipment related control instruction defined according to a time sequence;
and the action execution module executes the related control instructions according to the time sequence.
8. The automatic spraying device based on the point cloud as claimed in claim 7, wherein the database module is pre-stored with a plurality of 3D point cloud model templates for spraying the components to be sprayed, spraying process parameters, point cloud algorithm hyper-parameters, employee account authority, employee operation records and system operation information;
the global positioning module provides longitudinal position service for the reconstruction of the 3D model of the component to be sprayed, plays a role in aligning a 3D model coordinate system and a spraying mechanical arm (4) coordinate system, and enables data to have consistency in different operation space states;
the monitoring service module always monitors the intermediate processing state information of each component of the equipment and the software algorithm;
the data image module dynamically displays each data in a UI form.
9. An automatic spraying method based on point cloud is characterized by comprising the following steps:
information entry: recording information such as a 3D point cloud model template, spraying process parameters, point cloud algorithm hyper-parameters and the like of a plurality of components (3) to be sprayed into a database module;
conveying a component: the components (3) to be sprayed of the same type are placed on the assembly line conveyer belt (1) side by side at a certain interval, and the assembly line conveyer belt (1) is controlled to convey at a set speed;
data acquisition: the automatic scanning device (2) scans and acquires profile data of a member (3) to be sprayed, the position sensor (5) collects the position and the traveling speed data of the member (3) to be sprayed, and data information collected by the automatic scanning device (2) and the position sensor (5) is sent to the embedded computing unit (6);
data processing and component spraying: the embedded computing unit (6) receives data information acquired by the automatic scanning device (2) and the position sensor (5), performs 3D modeling, spraying object identification, spraying process analysis, spraying path planning and spraying time sequence designation generation, sends spraying time sequence designation to the spraying mechanical arm (4) and the spraying device, and performs spraying operation.
10. The point cloud based automated spray coating method of claim 9, wherein the data processing and component spraying comprises the sub-steps of:
s1: acquiring data information acquired by automatic scanning devices (2) and position sensors (5), unifying each frame of measurement data of a plurality of automatic scanning devices (2) from respective sensor coordinate systems to a spraying coordinate system after spatial transformation to form a frame of complete contour data, judging whether a member to be sprayed (3) exists by using the change of the characteristics of the frame of contour data, and splicing the scanned contour data of each frame in the longitudinal direction by using the position service of a global positioning module when the member to be sprayed (3) is detected to generate a three-dimensional model of the member to be sprayed (3);
s2: firstly, separating profile data of a single component (3) to be sprayed, then carrying out processing such as filtering, down-sampling, smoothing and outlier elimination on the data, and calculating the quantity and geometric characteristic information of the component (3) to be sprayed after segmenting the original overall profile data;
s3: after contour data of a plurality of components (3) to be sprayed are obtained, a 3D point cloud model template in a matching database module is retrieved, corresponding process parameters are extracted after matching is successful, all surfaces to be sprayed of a sprayed workpiece are extracted according to the spraying process parameters and parameterized, and a spraying path of each spraying period is discretized according to a parameter equation of each spraying surface by using a corresponding spraying process algorithm;
s4: fitting the spraying paths in the spraying surfaces and the walking paths among the spraying components by combining related spraying process parameters to form spraying tracks with time and speed characteristics, and covering the to-be-sprayed area of each group of components;
s5: converting the spraying track into related control instructions defined according to a time sequence, and converting high-level track data based on a space position into low-level control instructions based on the time sequence;
s6: the assembly line conveying belt (1) conveys a plurality of scanned components (3) to be sprayed into an operation area of the spraying mechanical arm (4) and aligns a coordinate system, the action execution module controls the assembly line conveying belt (1), the spraying mechanical arm (4) and the spraying device to reach a certain coordinate position according to a time sequence to complete spraying operation according to a low-level control instruction based on the time sequence, and the paint spraying pressure is controlled to change paint pressure according to the time sequence;
s7: monitoring the spraying process for relevant spraying information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110097083.0A CN112934518A (en) | 2021-01-25 | 2021-01-25 | Automatic spraying device and method based on point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110097083.0A CN112934518A (en) | 2021-01-25 | 2021-01-25 | Automatic spraying device and method based on point cloud |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112934518A true CN112934518A (en) | 2021-06-11 |
Family
ID=76236437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110097083.0A Pending CN112934518A (en) | 2021-01-25 | 2021-01-25 | Automatic spraying device and method based on point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112934518A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113976354A (en) * | 2021-11-05 | 2022-01-28 | 深圳联钜自控科技有限公司 | UV three-proofing paint coating special equipment based on artificial intelligence technology |
| CN114749299A (en) * | 2022-04-26 | 2022-07-15 | 广东天太机器人有限公司 | Control system and method for spraying and repairing live working tool |
| CN115829963A (en) * | 2022-11-28 | 2023-03-21 | 江苏凯威特斯半导体科技有限公司 | Plasma spraying image data identification and analysis method and system |
| CN116030134A (en) * | 2023-02-14 | 2023-04-28 | 长沙智能驾驶研究院有限公司 | Positioning method, apparatus, device, readable storage medium and program product |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4334495A (en) * | 1978-07-11 | 1982-06-15 | Trw Inc. | Method and apparatus for use in making an object |
| CN205647741U (en) * | 2016-04-22 | 2016-10-12 | 天津庆通科技有限公司 | Laser rangefinder is assisted in highspeed camera focal length setting |
| CN107552289A (en) * | 2017-10-16 | 2018-01-09 | 佛山沃顿装备技术股份有限公司 | A kind of visual identity paint-spray robot system and its operation method |
| CN107899814A (en) * | 2017-12-20 | 2018-04-13 | 芜湖哈特机器人产业技术研究院有限公司 | A kind of robot spraying system and its control method |
| CN107908152A (en) * | 2017-12-26 | 2018-04-13 | 苏州瀚华智造智能技术有限公司 | A kind of movable robot automatic spray apparatus, control system and method |
| CN108225180A (en) * | 2017-12-31 | 2018-06-29 | 芜湖哈特机器人产业技术研究院有限公司 | A kind of application alignment system and method |
| JP2021092370A (en) * | 2019-12-12 | 2021-06-17 | オリオン機械株式会社 | Cooling device and cooling method for the same |
-
2021
- 2021-01-25 CN CN202110097083.0A patent/CN112934518A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4334495A (en) * | 1978-07-11 | 1982-06-15 | Trw Inc. | Method and apparatus for use in making an object |
| CN205647741U (en) * | 2016-04-22 | 2016-10-12 | 天津庆通科技有限公司 | Laser rangefinder is assisted in highspeed camera focal length setting |
| CN107552289A (en) * | 2017-10-16 | 2018-01-09 | 佛山沃顿装备技术股份有限公司 | A kind of visual identity paint-spray robot system and its operation method |
| CN107899814A (en) * | 2017-12-20 | 2018-04-13 | 芜湖哈特机器人产业技术研究院有限公司 | A kind of robot spraying system and its control method |
| CN107908152A (en) * | 2017-12-26 | 2018-04-13 | 苏州瀚华智造智能技术有限公司 | A kind of movable robot automatic spray apparatus, control system and method |
| CN108225180A (en) * | 2017-12-31 | 2018-06-29 | 芜湖哈特机器人产业技术研究院有限公司 | A kind of application alignment system and method |
| JP2021092370A (en) * | 2019-12-12 | 2021-06-17 | オリオン機械株式会社 | Cooling device and cooling method for the same |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113976354A (en) * | 2021-11-05 | 2022-01-28 | 深圳联钜自控科技有限公司 | UV three-proofing paint coating special equipment based on artificial intelligence technology |
| CN113976354B (en) * | 2021-11-05 | 2022-07-01 | 深圳联钜自控科技有限公司 | Special UV three-proofing paint coating equipment based on artificial intelligence technology |
| CN114749299A (en) * | 2022-04-26 | 2022-07-15 | 广东天太机器人有限公司 | Control system and method for spraying and repairing live working tool |
| CN114749299B (en) * | 2022-04-26 | 2022-11-04 | 广东天太机器人有限公司 | Control system and method for spray repairing live-wire tool |
| CN115829963A (en) * | 2022-11-28 | 2023-03-21 | 江苏凯威特斯半导体科技有限公司 | Plasma spraying image data identification and analysis method and system |
| CN115829963B (en) * | 2022-11-28 | 2023-11-03 | 江苏凯威特斯半导体科技有限公司 | Plasma spraying image data identification analysis method and system |
| CN116030134A (en) * | 2023-02-14 | 2023-04-28 | 长沙智能驾驶研究院有限公司 | Positioning method, apparatus, device, readable storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112934518A (en) | Automatic spraying device and method based on point cloud | |
| CN107899814A (en) | A kind of robot spraying system and its control method | |
| CN112934541B (en) | Automatic spraying device and method based on visual 3D reconstruction | |
| CN109095213B (en) | Intelligent chain bucket type continuous ship unloader control system and method | |
| CN106391366A (en) | Automatic coating system for large component of civil aircraft | |
| CN108182689A (en) | The plate workpiece three-dimensional recognition positioning method in polishing field is carried applied to robot | |
| CN104785958B (en) | A kind of cryogenic tank outer member robot welding workstation | |
| CN110341749B (en) | Track disease inspection robot system and control method | |
| CN109013405A (en) | It is a kind of independently detected with cast(ing) surface and substandard products sorting function robot system | |
| CN111634636B (en) | Full-automatic material taking control system of bucket wheel machine | |
| CN101161151A (en) | Method and system for automatic generating shoe sole photopolymer coating track based on linear structure optical sensor | |
| CN111013857A (en) | Spraying robot control system and control method | |
| CN112605033B (en) | Photovoltaic panel posture recognition and cleaning regulation and control device and method | |
| CN113190002B (en) | Method for realizing automatic inspection by high-speed rail box girder inspection robot | |
| CN112141734A (en) | Material taking control system and material taking method for scraper reclaimer | |
| CN104615150A (en) | Machine vision based adaptive precise mist spray device and method | |
| CN112282787A (en) | Multi-arm robot for automatic maintenance of tunnel and control method thereof | |
| CN114851195A (en) | Control method of visual welding process system | |
| He et al. | Intelligent logistics system of steel bar warehouse based on ubiquitous information | |
| Paul et al. | A robotic system for steel bridge maintenance: Field testing | |
| CN107633501A (en) | The image recognition localization method of machine vision | |
| CN117369460A (en) | Intelligent inspection method and system for loosening faults of vehicle bolts | |
| Kim et al. | Path planning for machine vision assisted, teleoperated pavement crack sealer | |
| CN112782686A (en) | Automatic alignment system and method for loading based on multidimensional identification and positioning technology | |
| CN117282580A (en) | Intelligent manipulator for spraying assembly line and control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211014 Address after: Room A01, room 418, East head, building 10, Dongcheng garden, No. 34, industrial South Road, high tech Zone, Jinan, Shandong Province Applicant after: Jinan lantushi Intelligent Technology Co.,Ltd. Address before: 255000 7-11, 2nd floor, No.3, Zibo science and Technology Industrial Park, Xishou, Sanying Road, Zhangdian District, Zibo City, Shandong Province Applicant before: Shandong Huarui Intelligent Technology Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210611 |