CN110580475A - line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium - Google Patents

line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium Download PDF

Info

Publication number
CN110580475A
CN110580475A CN201911097298.1A CN201911097298A CN110580475A CN 110580475 A CN110580475 A CN 110580475A CN 201911097298 A CN201911097298 A CN 201911097298A CN 110580475 A CN110580475 A CN 110580475A
Authority
CN
China
Prior art keywords
line image
line
network
training
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911097298.1A
Other languages
Chinese (zh)
Inventor
万慧建
于雪
万吨
杨娜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
JIANGXI BOWEI NEW TECHNOLOGY CO LTD
Original Assignee
JIANGXI BOWEI NEW TECHNOLOGY CO LTD
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by JIANGXI BOWEI NEW TECHNOLOGY CO LTD filed Critical JIANGXI BOWEI NEW TECHNOLOGY CO LTD
Priority to CN201911097298.1A priority Critical patent/CN110580475A/en
Publication of CN110580475A publication Critical patent/CN110580475A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06Energy or water supply
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/13Satellite images

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Economics (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Strategic Management (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Water Supply & Treatment (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Human Resources & Organizations (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Tourism & Hospitality (AREA)
  • Image Analysis (AREA)

Abstract

the invention relates to a data processing technology and provides a line diagnosis method based on unmanned aerial vehicle routing inspection, an electronic device and a storage medium. The method comprises the steps of obtaining a preset number of line images shot by an unmanned aerial vehicle in advance from a database, preprocessing the line images to be used as a training sample set, constructing a MobileNet V2 network, inputting the training sample set into a MobileNet V2 network, using output characteristic vectors of the MobileNet V2 network as characteristic vectors corresponding to all line images in the training sample set, marking preset labels on all line images in the training sample set, inputting the preset labels and the corresponding characteristic vectors of all line images into an SSD model for training to obtain a line image fault diagnosis model, receiving line images to be diagnosed shot when the unmanned aerial vehicle patrols, inputting the line images into the line image fault recognition model, and feeding output results back to a client. By using the method and the device, the accuracy of identifying and diagnosing the power line image can be improved.

Description

line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium
Technical Field
the invention relates to the field of data processing, in particular to a line diagnosis method based on unmanned aerial vehicle routing inspection, an electronic device and a storage medium.
background
overhead power lines are wide in coverage, complex in terrain passing through areas and bad in natural environment, and power departments spend huge manpower and material resources to conduct line patrol every year so as to master the running conditions of the lines and timely eliminate potential hidden dangers of the lines.
The power line is mostly under complicated natural environment, and the accuracy rate of utilizing the existing image recognition technology to carry out fault recognition through the power line image that unmanned aerial vehicle shot is very low. Therefore, how to perform efficient intelligent identification and diagnosis on the power line image acquired by the unmanned aerial vehicle becomes a problem to be solved urgently by technical personnel in the field.
Disclosure of Invention
In view of the above, the present invention provides a line diagnosis method, an electronic device, and a storage medium based on unmanned aerial vehicle inspection, and aims to improve the accuracy of identifying and diagnosing power line images.
in order to achieve the purpose, the invention provides a line diagnosis method based on unmanned aerial vehicle routing inspection, which comprises the following steps:
an acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
the construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
Training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
a feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
preferably, the network structure of the MobileNetV2 network comprises 53 convolutional layers, 1 pooling layer and 1 fully-connected layer which are connected in sequence.
preferably, the inputting the preset label of each line image and the corresponding feature vector into the SSD model for training to obtain the line image fault identification model includes:
generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion;
training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
and finishing training when the accuracy rate is greater than a preset threshold value, and obtaining the line image fault diagnosis model.
Preferably, the extracting step further comprises:
setting a loss function for the MobileNet V2 network in advance, inputting the training sample into the MobileNet V2 network, carrying out forward propagation on the input training sample to obtain actual output, substituting preset target output and the actual output into the loss function, calculating a loss value of the loss function, carrying out backward propagation, and optimizing parameters of the MobileNet V2 network by using the loss value to obtain the optimized MobileNet V2 network.
preferably, the labeling of the preset label to each line image in the training sample set includes:
Respectively marking the fault area of each fault line image in a rectangular frame form by using a preset marking tool, and generating a marking file in a preset format corresponding to each line image, wherein the marking file stores the coordinate data of the fault area of the line image.
to achieve the above object, the present invention also provides an electronic device, including: the unmanned aerial vehicle inspection system comprises a memory and a processor, and is characterized in that a line diagnosis program based on unmanned aerial vehicle inspection is stored in the memory, and is executed by the processor, so that the following steps are realized:
an acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
The construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
Training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
a feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
Preferably, the inputting the preset label of each line image and the corresponding feature vector into the SSD model for training to obtain the line image fault identification model includes:
Generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion;
Training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
And finishing training when the accuracy rate is greater than a preset threshold value, and obtaining the line image fault diagnosis model.
Preferably, the extracting step further comprises:
setting a loss function for the MobileNet V2 network in advance, inputting the training sample into the MobileNet V2 network, carrying out forward propagation on the input training sample to obtain actual output, substituting preset target output and the actual output into the loss function, calculating a loss value of the loss function, carrying out backward propagation, and optimizing parameters of the MobileNet V2 network by using the loss value to obtain the optimized MobileNet V2 network.
Preferably, the labeling of the preset label to each line image in the training sample set includes:
Respectively marking the fault area of each fault line image in a rectangular frame form by using a preset marking tool, and generating a marking file in a preset format corresponding to each line image, wherein the marking file stores the coordinate data of the fault area of the line image.
In order to achieve the above object, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a line diagnosis program based on unmanned aerial vehicle inspection, and when the line diagnosis program based on unmanned aerial vehicle inspection is executed by a processor, any step of the above line diagnosis method based on unmanned aerial vehicle inspection is implemented.
According to the line diagnosis method based on unmanned aerial vehicle inspection, the electronic device and the storage medium, the defects and hidden dangers of the power line are deeply analyzed, identified and diagnosed by receiving the power line image shot when the unmanned aerial vehicle inspects, and diagnosis and fault location of the power line are accurately achieved. The identification and diagnosis information of the defects and hidden dangers of the power line equipment is associated with the running state, the age limit, the model number, the external environment information and the like of the equipment, so that a solid and reliable original data base is provided for a command system for researching and judging the distribution network fault, and the risk of safe running of a power grid caused by personnel shortage is effectively reduced.
Drawings
FIG. 1 is a diagram of an electronic device according to a preferred embodiment of the present invention;
Fig. 2 is a schematic block diagram of a preferred embodiment of the route diagnosis procedure based on unmanned aerial vehicle routing inspection in fig. 1;
FIG. 3 is a flow chart of a preferred embodiment of the route diagnosis method based on unmanned aerial vehicle routing inspection according to the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a schematic diagram of an electronic device 1 according to a preferred embodiment of the invention is shown.
the electronic device 1 includes but is not limited to: memory 11, processor 12, display 13, and network interface 14. The electronic device 1 is connected to a network through a network interface 14 to obtain raw data. The network may be a wireless or wired network such as an Intranet (Internet), the Internet (Internet), a Global System for mobile communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), a call network, and the like.
the memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 1, such as a hard disk or a memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic apparatus 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided in the electronic apparatus 1. Of course, the memory 11 may also comprise both an internal memory unit of the electronic apparatus 1 and an external memory device thereof. In this embodiment, the memory 11 is generally used for storing an operating system installed in the electronic device 1 and various types of application software, such as program codes of the line diagnosis program 10 based on drone patrol. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is generally used for controlling the overall operation of the electronic device 1, such as performing data interaction or communication related control and processing. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run the program code of the route diagnosis program 10 based on the unmanned aerial vehicle inspection.
The display 13 may be referred to as a display screen or display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-emitting diode (OLED) touch screen, or the like. The display 13 is used for displaying information processed in the electronic apparatus 1 and for displaying a visual work interface.
The network interface 14 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), the network interface 14 typically being used for establishing a communication connection between the electronic apparatus 1 and other electronic devices.
Fig. 1 shows only the electronic device 1 with components 11-14 and the drone patrol based line diagnostics program 10, but it will be understood that not all of the shown components are required and that more or fewer components may be implemented instead.
optionally, the electronic device 1 may further comprise a user interface, the user interface may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic apparatus 1 and for displaying a visualized user interface.
The electronic device 1 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, and the like, which are not described in detail herein.
In the above embodiment, the processor 12, when executing the route diagnosis program 10 based on unmanned aerial vehicle inspection stored in the memory 11, may implement the following steps:
An acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
the construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
Training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
a feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
for detailed description of the above steps, please refer to the following description of fig. 2 regarding a program module diagram of an embodiment of the route diagnosis program 10 based on unmanned aerial vehicle inspection, and fig. 3 regarding a flowchart of an embodiment of the route diagnosis method based on unmanned aerial vehicle inspection.
in other embodiments, the drone patrol based line diagnostics program 10 may be partitioned into modules that are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The modules referred to herein are referred to as a series of computer program instruction segments capable of performing specified functions.
referring to fig. 2, a block diagram of an embodiment of the route diagnosis program 10 based on unmanned aerial vehicle inspection in fig. 1 is shown. In this embodiment, the route diagnosis program 10 based on unmanned aerial vehicle inspection may be divided into: an acquisition module 110, a construction module 120, a training module 130, and a feedback module 140.
The acquisition module 110 is configured to acquire a preset number of line images shot by the unmanned aerial vehicle in advance from a database, where the line images include normal line images and fault line images, and the line images are preprocessed to use the preprocessed images as a training sample set.
In this embodiment, a preset number of power transmission line images shot by the unmanned aerial vehicle in advance can be obtained from the database, the line images include images of normal lines and images of line faults, if the number of the obtained line images is too small, the problem of insufficient network training or the phenomenon of overfitting can easily occur during model training and recognition, and the model becomes very sensitive to image distortion due to overfitting of a small number of samples; if the number of images is too large, the training speed of the network will be very slow, and the convergence of the network will take a long time. In addition, in order to ensure the accuracy of the network model, the fault line image should select an image with an obvious fault point as much as possible. Therefore, the obtained line image can be cut at random positions of the original image by using squares with the original image size not less than 0.5 times, the cut image is randomly turned or mirrored horizontally or vertically, and the cut image is also used as a training sample. By increasing the number of images of the training samples and the diversity of the increased training samples, overfitting can be avoided, and the recognition performance of subsequent models can be improved. The preprocessing mode may be normalization processing, for example, the pixel values of the acquired line image area are mapped to a [ 0, 1 ] interval uniformly to eliminate the influence of uneven illumination on the line image, and the preprocessed image is used as a training sample set. The image can be subjected to graying processing, denoising processing and image segmentation processing.
And the construction module 120 is configured to construct a MobileNetV2 network, input the training sample set into the MobileNetV2 network, and use an output feature vector of the MobileNetV2 network as a feature vector corresponding to each line image in the training sample set.
In this embodiment, mobilonetv 2 is a lightweight convolutional neural network structure, and the mobilonetv 2 network can efficiently and quickly identify an image with low resolution, occupies a small bandwidth in calculation, and can be carried on a mobile device for use. The mobilenetV2 network comprises 53 convolutional layers, 1 pooling layer and 1 full-connection layer which are sequentially connected, wherein the 53 convolutional layers comprise 1 input layer, 17 bottleneck building blocks and 1 output layer which are sequentially connected, each bottleneck building block comprises 3 convolutional layers respectively, and the convolutional cores of the 53 convolutional layers are all 3 x 3. Since the features of the images only need to be extracted by using MobileNetV2, the present implementation uses the feature vector output after removing the convolutional layer that is finally used for classification by MobileNetV2 as the feature vector corresponding to each line image in the training sample set.
further, when the MobileNetV2 network is trained, a loss function can be set for the MobileNetV2 network in advance, a training sample is input into the MobileNetV2 network, forward propagation is performed on the input training sample to obtain an actual output, a preset target output and the actual output are substituted into the loss function, a loss value of the loss function is calculated, backward propagation is performed, parameters of the MobileNetV2 network are optimized by using the loss value, and the optimized MobileNetV2 network is obtained. Then, a training sample is selected and input into the optimized MobileNetV2, and the optimized MobileNetV2 is trained again by referring to the above operation until the condition of stopping training is reached.
Wherein, the preset loss function of the MobileNetV2 network is as follows:
The loss value of the actual output and the target output of the MobileNetV2 network,for the target output value in the kth sample,The actual output value of the kth sample in the MobileNetV2 network during training. When the loss value is larger than the preset learning error, the parameters of the MobileNet V2 network are optimized and adjusted until the loss value is smaller than the preset learning errorand learning errors.
the training module 130 is configured to label a preset label on each line image in the training sample set, and input the preset label of each line image and a corresponding feature vector into an SSD model for training to obtain a line image fault diagnosis model.
marking a preset label for a fault area of the line fault image in the training sample according to the type of the fault, for example: lightning stroke faults, windage yaw faults, pollution flashover faults, icing faults and bird damage faults are respectively marked as first-class faults, second-class faults, third-class faults, fourth-class faults and fifth-class faults. Specifically, a preset labeling tool is used for labeling the fault area of each fault line image in a rectangular frame form, and a labeling file in a preset format corresponding to each line image is generated, wherein the labeling file stores coordinate data of the fault area of the line image. The method comprises the following steps of taking a feature vector of a marked line fault image as the input of a model, taking a preset label marked by the line image as the output of the model, training an SSD model to obtain a line image fault diagnosis model, and specifically comprises the following training steps:
Generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion (for example, 4: 1);
Training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
And when the accuracy is greater than a preset threshold (for example, 95%), ending the training to obtain the line image fault diagnosis model.
The feedback module 140 is configured to receive a line image to be diagnosed, which is shot when the unmanned aerial vehicle patrols and examines, wherein the line image to be diagnosed includes the GPS location information and the shooting time of the line image, input the line image into the line image fault identification model, and feed back an output result, the GPS location information, and the shooting time to the client.
In this embodiment, the unmanned aerial vehicle has an RTK high-precision positioning module, and can accurately acquire GPS position information of a line photographed by the unmanned aerial vehicle based on the position of the unmanned aerial vehicle, and a line image photographed by the unmanned aerial vehicle includes the GPS position information and the photographing time of the line image, and inputs the line image into the MobileNetV2 network, and inputs a feature vector output by the MobileNetV2 network into the line image fault diagnosis model, so as to obtain a diagnosis result of each image, and feeds back the diagnosis result, the GPS position information, and the photographing time to the client.
Fig. 3 is a flowchart of a preferred embodiment of the route diagnosis method based on unmanned aerial vehicle routing inspection according to the present invention.
Step S10: the method comprises the steps of obtaining a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set.
in this embodiment, a preset number of power transmission line images shot by the unmanned aerial vehicle in advance can be obtained from the database, the line images include images of normal lines and images of line faults, if the number of the obtained line images is too small, the problem of insufficient network training or the phenomenon of overfitting can easily occur during model training and recognition, and the model becomes very sensitive to image distortion due to overfitting of a small number of samples; if the number of images is too large, the training speed of the network will be very slow, and the convergence of the network will take a long time. In addition, in order to ensure the accuracy of the network model, the fault line image should select an image with an obvious fault point as much as possible. Therefore, the obtained line image can be cut at random positions of the original image by using squares with the original image size not less than 0.5 times, the cut image is randomly turned or mirrored horizontally or vertically, and the cut image is also used as a training sample. By increasing the number of images of the training samples and the diversity of the increased training samples, overfitting can be avoided, and the recognition performance of subsequent models can be improved. The preprocessing mode may be normalization processing, for example, the pixel values of the acquired line image area are mapped to a [ 0, 1 ] interval uniformly to eliminate the influence of uneven illumination on the line image, and the preprocessed image is used as a training sample set. The image can be subjected to graying processing, denoising processing and image segmentation processing.
step S20: and constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set.
in this embodiment, mobilonetv 2 is a lightweight convolutional neural network structure, and the mobilonetv 2 network can efficiently and quickly identify an image with low resolution, occupies a small bandwidth in calculation, and can be carried on a mobile device for use. The mobilenetV2 network comprises 53 convolutional layers, 1 pooling layer and 1 full-connection layer which are sequentially connected, wherein the 53 convolutional layers comprise 1 input layer, 17 bottleneck building blocks and 1 output layer which are sequentially connected, each bottleneck building block comprises 3 convolutional layers respectively, and the convolutional cores of the 53 convolutional layers are all 3 x 3. Since the features of the images only need to be extracted by using MobileNetV2, the present implementation uses the feature vector output after removing the convolutional layer that is finally used for classification by MobileNetV2 as the feature vector corresponding to each line image in the training sample set.
Further, when the MobileNetV2 network is trained, a loss function can be set for the MobileNetV2 network in advance, a training sample is input into the MobileNetV2 network, forward propagation is performed on the input training sample to obtain an actual output, a preset target output and the actual output are substituted into the loss function, a loss value of the loss function is calculated, backward propagation is performed, parameters of the MobileNetV2 network are optimized by using the loss value, and the optimized MobileNetV2 network is obtained. Then, a training sample is selected and input into the optimized MobileNetV2, and the optimized MobileNetV2 is trained again by referring to the above operation until the condition of stopping training is reached.
Wherein, the preset loss function of the MobileNetV2 network is as follows:
The loss value of the actual output and the target output of the MobileNetV2 network,For the target output value in the kth sample,The actual output value of the kth sample in the MobileNetV2 network during training. And when the loss value is larger than the preset learning error, optimizing and adjusting the parameters of the MobileNet V2 network until the loss value is smaller than the preset learning error.
step S30: and labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model.
Marking a preset label for a fault area of the line fault image in the training sample according to the type of the fault, for example: lightning stroke faults, windage yaw faults, pollution flashover faults, icing faults and bird damage faults are respectively marked as first-class faults, second-class faults, third-class faults, fourth-class faults and fifth-class faults. Specifically, a preset labeling tool is used for labeling the fault area of each fault line image in a rectangular frame form, and a labeling file in a preset format corresponding to each line image is generated, wherein the labeling file stores coordinate data of the fault area of the line image. The method comprises the following steps of taking a feature vector of a marked line fault image as the input of a model, taking a preset label marked by the line image as the output of the model, training an SSD model to obtain a line image fault diagnosis model, and specifically comprises the following training steps:
Generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion (for example, 4: 1);
training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
and when the accuracy is greater than a preset threshold (for example, 95%), ending the training to obtain the line image fault diagnosis model.
Step S40: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
In this embodiment, the unmanned aerial vehicle has an RTK high-precision positioning module, and can accurately acquire GPS position information of a line photographed by the unmanned aerial vehicle based on the position of the unmanned aerial vehicle, and a line image photographed by the unmanned aerial vehicle includes the GPS position information and the photographing time of the line image, and inputs the line image into the MobileNetV2 network, and inputs a feature vector output by the MobileNetV2 network into the line image fault diagnosis model, so as to obtain a diagnosis result of each image, and feeds back the diagnosis result, the GPS position information, and the photographing time to the client.
Furthermore, the embodiment of the present invention also provides a computer-readable storage medium, which may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, and the like. The computer readable storage medium includes a drone patrol based line diagnostic program 10, which when executed by a processor, the drone patrol based line diagnostic program 10 performs the following operations:
An acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
the construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
Training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
A feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
the specific implementation of the computer-readable storage medium of the present invention is substantially the same as the above-mentioned line diagnosis method based on unmanned aerial vehicle inspection, and is not described herein again.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention essentially or contributing to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (such as a mobile phone, a computer, an electronic device, or a network device) to execute the method according to the embodiments of the present invention.
the above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. a line diagnosis method based on unmanned aerial vehicle routing inspection is applied to an electronic device and is characterized by comprising the following steps:
an acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
the construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
A feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
2. the unmanned aerial vehicle inspection tour based line diagnosis method of claim 1, wherein the network structure of the MobileNetV2 network includes 53 convolutional layers, 1 pooling layer, and 1 fully-connected layer, which are connected in sequence.
3. The unmanned aerial vehicle inspection-based line diagnosis method according to claim 1, wherein the step of inputting preset labels and corresponding feature vectors of the line images into an SSD model for training to obtain a line image fault recognition model comprises:
generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion;
Training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
and finishing training when the accuracy is verified to be larger than a preset threshold value, and obtaining the line image fault diagnosis model.
4. the unmanned aerial vehicle inspection tour based line diagnosis method of claim 1, wherein the extracting step further includes:
Setting a loss function for the MobileNet V2 network in advance, inputting the training sample into the MobileNet V2 network, carrying out forward propagation on the input training sample to obtain actual output, substituting preset target output and the actual output into the loss function, calculating a loss value of the loss function, carrying out backward propagation, and optimizing parameters of the MobileNet V2 network by using the loss value to obtain the optimized MobileNet V2 network.
5. The unmanned aerial vehicle inspection tour based line diagnosis method of claim 1, wherein the labeling of the preset label to each line image in the training sample set comprises:
respectively marking the fault area of each fault line image in a rectangular frame form by using a preset marking tool, and generating a marking file in a preset format corresponding to each line image, wherein the marking file stores the coordinate data of the fault area of the line image.
6. an electronic device, comprising a memory and a processor, wherein the memory stores a line diagnosis program based on unmanned aerial vehicle inspection, and the line diagnosis program based on unmanned aerial vehicle inspection is executed by the processor, so as to realize the following steps:
an acquisition step: acquiring a preset number of line images shot by an unmanned aerial vehicle in advance from a database, wherein the line images comprise normal line images and fault line images, preprocessing the line images, and taking the preprocessed images as a training sample set;
the construction steps are as follows: constructing a MobileNet V2 network, inputting the training sample set into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to each line image in the training sample set;
Training: labeling a preset label on each line image in the training sample set, inputting the preset label of each line image and the corresponding feature vector into an SSD model for training, and obtaining a line image fault diagnosis model; and
a feedback step: the method comprises the steps of receiving a line image to be diagnosed shot when an unmanned aerial vehicle patrols and examines, inputting the line image into a line image fault identification model, and feeding back an output result, GPS position information and shooting time to a client side, wherein the line image to be diagnosed comprises the GPS position information and the shooting time of the line image.
7. The electronic device of claim 6, wherein the inputting the preset label and the corresponding feature vector of each line image into the SSD model for training to obtain the line image fault recognition model comprises:
generating a target sample set by taking the feature vector of each line image as a variable X and taking a preset label of each line image as a dependent variable Y;
Dividing the target sample set into a training set and a verification set according to a preset proportion;
training the SSD model by using each variable X and each dependent variable Y in the training set, verifying the SSD model by using the verification set every preset period, and verifying the accuracy of the SSD model by using each variable X and each dependent variable Y in the verification set; and
and finishing training when the accuracy rate is greater than a preset threshold value, and obtaining the line image fault diagnosis model.
8. The electronic device of claim 6, wherein the extracting step further comprises:
Setting a loss function for the MobileNet V2 network in advance, inputting the training sample into the MobileNet V2 network, carrying out forward propagation on the input training sample to obtain actual output, substituting preset target output and the actual output into the loss function, calculating a loss value of the loss function, carrying out backward propagation, and optimizing parameters of the MobileNet V2 network by using the loss value to obtain the optimized MobileNet V2 network.
9. The electronic device of claim 6, wherein the labeling of the pre-set labels to the line images in the training sample set comprises:
Respectively marking the fault area of each fault line image in a rectangular frame form by using a preset marking tool, and generating a marking file in a preset format corresponding to each line image, wherein the marking file stores the coordinate data of the fault area of the line image.
10. A computer-readable storage medium, wherein the computer-readable storage medium includes a line diagnosis program based on unmanned aerial vehicle inspection, and when the line diagnosis program based on unmanned aerial vehicle inspection is executed by a processor, the steps of the line diagnosis method based on unmanned aerial vehicle inspection according to any one of claims 1 to 5 are implemented.
CN201911097298.1A 2019-11-12 2019-11-12 line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium Pending CN110580475A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911097298.1A CN110580475A (en) 2019-11-12 2019-11-12 line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911097298.1A CN110580475A (en) 2019-11-12 2019-11-12 line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium

Publications (1)

Publication Number Publication Date
CN110580475A true CN110580475A (en) 2019-12-17

Family

ID=68815542

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911097298.1A Pending CN110580475A (en) 2019-11-12 2019-11-12 line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN110580475A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111311967A (en) * 2020-03-31 2020-06-19 普宙飞行器科技(深圳)有限公司 Unmanned aerial vehicle-based power line inspection system and method
CN111738189A (en) * 2020-06-29 2020-10-02 广东电网有限责任公司 Transmission line crimping hardware inspection control method, device, terminal and medium
CN112233270A (en) * 2020-10-30 2021-01-15 国家电网有限公司 Unmanned aerial vehicle is intelligence around tower system of patrolling and examining independently
CN112884018A (en) * 2021-01-28 2021-06-01 广东电网有限责任公司 Power grid line fault recognition model training method and power grid line inspection method
CN112927220A (en) * 2021-03-25 2021-06-08 郑州大学 Method and device for detecting insulator fault based on counterstudy and terminal equipment
CN113139529A (en) * 2021-06-21 2021-07-20 北京科技大学 Linear cultural heritage exploration method and system, storage medium and electronic equipment
CN113191389A (en) * 2021-03-31 2021-07-30 中国石油大学(华东) Submarine pipeline autonomous inspection method and device based on optical vision technology
CN113759968A (en) * 2021-09-01 2021-12-07 中国南方电网有限责任公司超高压输电公司贵阳局 Unmanned aerial vehicle-based power grid line patrol planning method and system
WO2021243724A1 (en) * 2020-06-05 2021-12-09 北京嘀嘀无限科技发展有限公司 Image processing method, electronic device, vehicle traveling data recorder, and server
CN113822834A (en) * 2020-06-03 2021-12-21 富泰华工业(深圳)有限公司 Article detection device, article detection method, and storage medium
CN114724268A (en) * 2022-03-31 2022-07-08 武汉珞珈天铭电气科技有限公司 Power transmission line inspection method, device, equipment and storage medium
CN115061490A (en) * 2022-05-30 2022-09-16 广州中科云图智能科技有限公司 Reservoir inspection method, device and equipment based on unmanned aerial vehicle and storage medium
CN116721095A (en) * 2023-08-04 2023-09-08 杭州瑞琦信息技术有限公司 Aerial photographing road illumination fault detection method and device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103824340A (en) * 2014-03-07 2014-05-28 山东鲁能智能技术有限公司 Intelligent inspection system and inspection method for electric transmission line by unmanned aerial vehicle
CN103839194A (en) * 2014-03-07 2014-06-04 国家电网公司 Unmanned aerial vehicle routing inspection image retrieval system and method based on electric transmission line and GIS
CN108010030A (en) * 2018-01-24 2018-05-08 福州大学 A kind of Aerial Images insulator real-time detection method based on deep learning
CN109840502A (en) * 2019-01-31 2019-06-04 深兰科技(上海)有限公司 A kind of method and apparatus carrying out target detection based on SSD model
CN109886225A (en) * 2019-02-27 2019-06-14 浙江理工大学 A kind of image gesture motion on-line checking and recognition methods based on deep learning
CN109936080A (en) * 2019-03-28 2019-06-25 郑州大学 A kind of method of unmanned plane inspection transmission line of electricity
CN110069975A (en) * 2018-12-25 2019-07-30 国网河北省电力有限公司电力科学研究院 A kind of conducting wire neural network based opens burst recognition methods and system
CN110287720A (en) * 2019-07-01 2019-09-27 国网内蒙古东部电力有限公司 A kind of access control method based on image recognition and user gradation

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103824340A (en) * 2014-03-07 2014-05-28 山东鲁能智能技术有限公司 Intelligent inspection system and inspection method for electric transmission line by unmanned aerial vehicle
CN103839194A (en) * 2014-03-07 2014-06-04 国家电网公司 Unmanned aerial vehicle routing inspection image retrieval system and method based on electric transmission line and GIS
CN108010030A (en) * 2018-01-24 2018-05-08 福州大学 A kind of Aerial Images insulator real-time detection method based on deep learning
CN110069975A (en) * 2018-12-25 2019-07-30 国网河北省电力有限公司电力科学研究院 A kind of conducting wire neural network based opens burst recognition methods and system
CN109840502A (en) * 2019-01-31 2019-06-04 深兰科技(上海)有限公司 A kind of method and apparatus carrying out target detection based on SSD model
CN109886225A (en) * 2019-02-27 2019-06-14 浙江理工大学 A kind of image gesture motion on-line checking and recognition methods based on deep learning
CN109936080A (en) * 2019-03-28 2019-06-25 郑州大学 A kind of method of unmanned plane inspection transmission line of electricity
CN110287720A (en) * 2019-07-01 2019-09-27 国网内蒙古东部电力有限公司 A kind of access control method based on image recognition and user gradation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
孙翠英 等: "《基于神经网络的输电线路故障识别方法》", 《科学技术与工程》 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111311967A (en) * 2020-03-31 2020-06-19 普宙飞行器科技(深圳)有限公司 Unmanned aerial vehicle-based power line inspection system and method
CN113822834A (en) * 2020-06-03 2021-12-21 富泰华工业(深圳)有限公司 Article detection device, article detection method, and storage medium
WO2021243724A1 (en) * 2020-06-05 2021-12-09 北京嘀嘀无限科技发展有限公司 Image processing method, electronic device, vehicle traveling data recorder, and server
CN111738189A (en) * 2020-06-29 2020-10-02 广东电网有限责任公司 Transmission line crimping hardware inspection control method, device, terminal and medium
CN112233270A (en) * 2020-10-30 2021-01-15 国家电网有限公司 Unmanned aerial vehicle is intelligence around tower system of patrolling and examining independently
CN112884018A (en) * 2021-01-28 2021-06-01 广东电网有限责任公司 Power grid line fault recognition model training method and power grid line inspection method
CN112927220A (en) * 2021-03-25 2021-06-08 郑州大学 Method and device for detecting insulator fault based on counterstudy and terminal equipment
CN113191389A (en) * 2021-03-31 2021-07-30 中国石油大学(华东) Submarine pipeline autonomous inspection method and device based on optical vision technology
CN113139529A (en) * 2021-06-21 2021-07-20 北京科技大学 Linear cultural heritage exploration method and system, storage medium and electronic equipment
CN113759968A (en) * 2021-09-01 2021-12-07 中国南方电网有限责任公司超高压输电公司贵阳局 Unmanned aerial vehicle-based power grid line patrol planning method and system
CN114724268A (en) * 2022-03-31 2022-07-08 武汉珞珈天铭电气科技有限公司 Power transmission line inspection method, device, equipment and storage medium
CN115061490A (en) * 2022-05-30 2022-09-16 广州中科云图智能科技有限公司 Reservoir inspection method, device and equipment based on unmanned aerial vehicle and storage medium
CN115061490B (en) * 2022-05-30 2024-04-05 广州中科云图智能科技有限公司 Unmanned aerial vehicle-based reservoir inspection method, unmanned aerial vehicle-based reservoir inspection device, unmanned aerial vehicle-based reservoir inspection equipment and storage medium
CN116721095A (en) * 2023-08-04 2023-09-08 杭州瑞琦信息技术有限公司 Aerial photographing road illumination fault detection method and device
CN116721095B (en) * 2023-08-04 2023-11-03 杭州瑞琦信息技术有限公司 Aerial photographing road illumination fault detection method and device

Similar Documents

Publication Publication Date Title
CN110580475A (en) line diagnosis method based on unmanned aerial vehicle inspection, electronic device and storage medium
CN108734283B (en) Neural network system
CN108022235B (en) Method for identifying defects of key components of high-voltage transmission iron tower
CN111784685A (en) Power transmission line defect image identification method based on cloud edge cooperative detection
CN111914813A (en) Power transmission line inspection image naming method and system based on image classification
CN110020653B (en) Image semantic segmentation method, device and computer readable storage medium
CN111401418A (en) Employee dressing specification detection method based on improved Faster r-cnn
CN110969719A (en) Automatic inspection method, system, terminal equipment and storage medium
CN110910360B (en) Positioning method of power grid image and training method of image positioning model
CN112367400B (en) Intelligent inspection method and system for power internet of things with edge cloud coordination
CN112381092B (en) Tracking method, tracking device and computer readable storage medium
CN111767915A (en) License plate detection method, device, equipment and storage medium
CN115512238A (en) Method and device for determining damaged area, storage medium and electronic device
Tomaszewski et al. Detection of power line insulators on digital images with the use of laser spots
CN115984158A (en) Defect analysis method and device, electronic equipment and computer readable storage medium
CN111860641A (en) Power grid image identification method, electronic device and storage medium
CN113408630A (en) Transformer substation indicator lamp state identification method
CN116503807A (en) Equipment inspection method, device, electronic equipment and computer program product
CN111104965A (en) Vehicle target identification method and device
CN116152685A (en) Pedestrian detection method and system based on unmanned aerial vehicle visual field
CN116151323A (en) Model generation method, device, electronic equipment and storage medium
CN110503685B (en) Data processing method and equipment
CN111859370B (en) Method, apparatus, electronic device and computer readable storage medium for identifying service
CN114821309A (en) Indoor transformer substation switch and indicator lamp state identification method and system
CN114663751A (en) Power transmission line defect identification method and system based on incremental learning technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20191217

RJ01 Rejection of invention patent application after publication