CN112733710A - Method for training a neural network for irrigation water pressure control of an irrigation device - Google Patents
Method for training a neural network for irrigation water pressure control of an irrigation device Download PDFInfo
- Publication number
- CN112733710A CN112733710A CN202110024413.3A CN202110024413A CN112733710A CN 112733710 A CN112733710 A CN 112733710A CN 202110024413 A CN202110024413 A CN 202110024413A CN 112733710 A CN112733710 A CN 112733710A
- Authority
- CN
- China
- Prior art keywords
- training
- loss function
- neural network
- water pressure
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/182—Network patterns, e.g. roads or rivers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a training method of a neural network for irrigation water pressure control of an irrigation device, which comprises the following steps: acquiring a training image, wherein the training image is an image of an area to be irrigated; passing the training image through an encoder to obtain a training feature vector, wherein the encoder comprises a deep convolutional neural network; calculating an orthogonalization loss function value of the training feature vector, the orthogonalization loss function value being a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix; passing the training feature vector through a decoder to obtain decoded values, the decoder comprising a plurality of fully-connected layers, the output bit of the last fully-connected layer in the decoder being one; calculating a difference loss function value between the decoded value and the real value; and updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value.
Description
Technical Field
The present invention relates to the field of deep learning and neural network technology, and more particularly, to a training method of a neural network for irrigation water pressure control of an irrigation device, an irrigation water pressure control method based on a deep neural network, a training system of a neural network for irrigation water pressure control of an irrigation device, an irrigation water pressure control system based on a deep neural network, and an electronic apparatus.
Background
Water is an essential precious resource in human production and life, but its naturally occurring state does not completely meet the needs of human beings. Through building hydraulic engineering, can control rivers, prevent flood disasters to adjust and the distribution of water yield, in order to satisfy people's life and production to the needs of water resource.
In the aspect of agricultural application, irrigation equipment for hydraulic engineering is used for irrigating the operation to the crop, but, current irrigation equipment for hydraulic engineering can not adapt to many topography well and irrigate the operation, and when irrigating the crop that has the uniform height, also appear the improper problem of water pressure control easily. If the water pressure is insufficient, the irrigation range is reduced, the water resource is not fully utilized, and if the water pressure is too high, the waste of the water resource and the efficiency reduction are caused.
Therefore, a solution for irrigation water pressure control of irrigation devices is desired.
At present, deep learning and neural networks have been widely applied in the fields of computer vision, natural language processing, text signal processing, and the like. In addition, deep learning and neural networks also exhibit a level close to or even exceeding that of humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.
The deep learning and the development of the neural network provide new solutions and schemes for the irrigation water pressure control of the irrigation device.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. Embodiments of the present application provide a training method of a neural network for irrigation water pressure control of an irrigation device, an irrigation water pressure control method based on a deep neural network, a training system of a neural network for irrigation water pressure control of an irrigation device, an irrigation water pressure control system based on a deep neural network, and an electronic apparatus, which perform control of irrigation water pressure of an irrigation device based on a deep learning computer vision method. Specifically, in the training process of the neural network for irrigation water pressure control of the irrigation device, in order to remove useless information in the feature vector as much as possible, the parameters of the encoder are updated by constructing an orthogonalization loss function when the encoder is trained, and therefore the accuracy of the model is improved.
According to one aspect of the present application, there is provided a method of training a neural network for irrigation water pressure control of an irrigation device, comprising:
acquiring a training image, wherein the training image is an image of an area to be irrigated;
passing the training image through an encoder to obtain a training feature vector, wherein the encoder comprises a deep convolutional neural network, the last layer of the deep convolutional neural network is a fully-connected layer, and the training feature vector is output by the fully-connected layer;
calculating an orthogonalization loss function value of the training feature vector, the orthogonalization loss function value being a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix;
passing the training feature vector through a decoder to obtain decoded values, the decoder comprising a plurality of fully-connected layers, the output bit of the last fully-connected layer in the decoder being one;
calculating a difference loss function value between the decoded value and the real value; and
updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value.
In the above method for training a neural network for irrigation water pressure control of an irrigation device, passing the training image through an encoder to obtain a training feature vector, the method includes: passing the training image through multiple convolutional layers, multiple pooling layers and multiple activation layers in the deep convolutional neural network to obtain a convolutional feature map; and passing the convolved feature map through the fully connected layer in the deep convolutional neural network to obtain the training feature vector.
In the above method for training a neural network for irrigation water pressure control of an irrigation device, calculating a difference loss function value between the decoded value and the true value includes: an L1 difference loss function value is calculated between the decoded value and the true value.
In the above method for training a neural network for irrigation water pressure control of an irrigation device, calculating a difference loss function value between the decoded value and the true value includes: an L2 difference loss function value is calculated between the decoded value and the true value.
In the above method for training a neural network for irrigation water pressure control of an irrigation device, updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value includes: in each iteration, updating parameters of the encoder based on the orthogonalization loss function values; and updating parameters of the encoder and the decoder with the difference loss function values.
In the above training method for a neural network for irrigation water pressure control of an irrigation device, acquiring a training image, which is an image of an area to be irrigated, comprising: collecting an image of an area to be irrigated through a camera deployed on an unmanned aerial vehicle; and adjusting the training image into an image with a uniform scale based on the depth of field value of the unmanned aerial vehicle camera in the process of collecting the area to be irrigated.
According to another aspect of the present application, there is provided a deep neural network-based irrigation water pressure control method, including:
acquiring an image of an irrigation area to be detected;
inputting the image into an encoder and a decoder trained according to the training method for the neural network for irrigation water pressure control of the irrigation device as described above, the output of the last fully-connected layer in the decoder representing a parameter of irrigation water pressure of the irrigation device; and
adjusting the irrigation water pressure of the irrigation device based on the parameter of the irrigation water pressure.
According to yet another aspect of the present application, there is provided a training system for an irrigation water pressure controlled neural network for an irrigation device, comprising:
the training image acquisition unit is used for acquiring a training image, wherein the training image is an image of an area to be irrigated;
a training feature vector generation unit, configured to pass the training image obtained by the training image obtaining unit through an encoder to obtain a training feature vector, where the encoder includes a deep convolutional neural network, a last layer of the deep convolutional neural network is a fully connected layer, and the training feature vector is output by the fully connected layer;
an orthogonalization loss function value calculating unit configured to calculate an orthogonalization loss function value of the training feature vector obtained by the training feature vector generating unit, where the orthogonalization loss function value is a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix;
a decoded value generating unit, configured to pass the training eigenvector obtained by the training eigenvector generating unit through a decoder to obtain a decoded value, where the decoder includes multiple fully-connected layers, and an output bit of a last fully-connected layer in the decoder is one;
a difference loss function value calculation unit configured to calculate a difference loss function value between the decoded value obtained by the decoded value generation unit and a true value; and
a parameter updating unit configured to update parameters of the encoder and the decoder based on the difference loss function value obtained by the difference loss function value calculating unit and the orthogonalization loss function value obtained by the orthogonalization loss function value calculating unit.
In the above training system for an irrigation water pressure controlled neural network of an irrigation device, the training feature vector generation unit includes: a convolution feature map generation subunit, configured to pass the training image through a multilayer convolution layer, a multilayer pooling layer, and a multilayer activation layer in the deep convolutional neural network to obtain a convolution feature map; and the convolutional neural network processing subunit is configured to pass the convolutional feature map obtained by the convolutional feature map generating subunit through the fully-connected layer in the deep convolutional neural network to obtain the training feature vector.
In the above training system for a neural network for irrigation water pressure control of an irrigation device, the difference loss function value calculating unit is further configured to: an L1 difference loss function value is calculated between the decoded value and the true value.
In the above training system for a neural network for irrigation water pressure control of an irrigation device, the difference loss function value calculating unit is further configured to: an L2 difference loss function value is calculated between the decoded value and the true value.
In the above training system for an irrigation water pressure controlled neural network of an irrigation device, the parameter updating unit includes: an encoder parameter updating subunit for updating a parameter of the encoder based on the orthogonalization loss function value; and a decoder parameter updating subunit for updating parameters of the encoder and the decoder with the difference loss function values.
In the above training system for an irrigation water pressure controlled neural network of an irrigation device, the training image obtaining unit includes: the image acquisition subunit is used for acquiring an image of the area to be irrigated through a camera deployed on the unmanned aerial vehicle; and the image adjusting subunit is used for adjusting the training image obtained by the image collecting subunit into an image with a uniform scale based on the depth of field value of the unmanned aerial vehicle camera in the process of collecting the area to be irrigated.
According to yet another aspect of the present application, there is provided an irrigation water pressure control system based on a deep neural network, comprising:
the to-be-detected image acquisition unit is used for acquiring an image of an irrigation area to be detected;
the parameter generating unit is used for inputting the images acquired by the image acquiring unit to be detected into the encoder and the decoder trained according to the training method of the neural network for controlling the irrigation water pressure of the irrigation device, and the output of the last fully-connected layer in the decoder represents the parameters of the irrigation water pressure of the irrigation device; and
and the irrigation water pressure adjusting unit is used for adjusting the irrigation water pressure of the irrigation device based on the parameters of the irrigation water pressure obtained by the parameter generating unit.
According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory in which computer program instructions are stored, which computer program instructions, when executed by the processor, cause the processor to perform a method of training a neural network for irrigation water pressure control of an irrigation device, or a method of irrigation water pressure control based on a deep neural network, as described above.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform a method of training a neural network for irrigation water pressure control of an irrigation device, or a method of irrigation water pressure control based on a deep neural network, as described above.
Compared with the prior art, the training method of the neural network for controlling the irrigation water pressure of the irrigation device, the irrigation water pressure control method based on the deep neural network, the training system of the neural network for controlling the irrigation water pressure of the irrigation device, the irrigation water pressure control system based on the deep neural network and the electronic equipment are based on the deep learning computer vision method to control the irrigation water pressure of the irrigation device. Specifically, in the training process of the neural network for irrigation water pressure control of the irrigation device, in order to remove useless information in the feature vector as much as possible, the parameters of the encoder are updated by constructing an orthogonalization loss function when the encoder is trained, and therefore the accuracy of the model is improved.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 illustrates an application scenario diagram of a training method of a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application;
FIG. 2 illustrates a flow chart of a method of training a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application;
FIG. 3 illustrates a system architecture diagram of a training method for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application;
FIG. 4 illustrates a flow chart for obtaining training images in a training method for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application;
FIG. 5 illustrates a flow chart of passing the training image through an encoder to obtain training feature vectors in a training method for an irrigation water pressure controlled neural network of an irrigation device according to an embodiment of the present application;
FIG. 6 illustrates a flow chart for updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value in a training method for a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application;
FIG. 7 illustrates a flow chart of a method of irrigation water pressure control based on a deep neural network according to an embodiment of the present application;
FIG. 8 illustrates a block diagram of a training system for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application.
Fig. 9 illustrates a block diagram of a training feature vector generation unit in a training system for an irrigation water pressure controlled neural network of an irrigation device according to an embodiment of the present application.
Fig. 10 illustrates a block diagram of a parameter update unit in a training system for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application.
Fig. 11 illustrates a block diagram of a training image acquisition unit in a training system for an irrigation water pressure controlled neural network of an irrigation device according to an embodiment of the present application.
FIG. 12 illustrates a block diagram of a deep neural network based irrigation water pressure control system according to an embodiment of the present application.
FIG. 13 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.
Overview of a scene
As described above, in the agricultural application, irrigation devices for hydraulic engineering are used for irrigation of crops, but current irrigation devices for hydraulic engineering cannot be well adapted to irrigation of multiple terrains, and when crops with a certain height are irrigated, the problem of improper water pressure control is easily caused. If the water pressure is insufficient, the irrigation range is reduced, the water resource is not fully utilized, and if the water pressure is too high, the waste of the water resource and the efficiency reduction are caused.
Considering that both the terrain condition and the height condition of the work can be relatively easily recognized from the image of the area to be irrigated, the inventors of the present application desire to perform control of the irrigation water pressure of the irrigation device by a computer vision method based on deep learning.
Specifically, in the solution of the present application, the above problem is solved with a two-stage encoder-decoder configuration, first, the feature map of the area to be irrigated is extracted by an encoder, i.e. a convolutional neural network, and the last layer of the convolutional neural network is set as a fully-connected layer, so as to encode the feature map of the area to be irrigated into a training feature vector of a predetermined length. In order to remove unnecessary information in the training feature vector as much as possible, when the encoder is trained, the encoder parameters are updated by constructing an orthogonalization loss function, which is the difference between the unit matrix and the product of the training feature vector and the transpose of the training feature vector, so that the encoder can extract and encode the features of the region to be irrigated into feature values orthogonal to each other as much as possible.
Next, the training feature vector is passed through a decoder composed of a plurality of fully-connected layers, wherein the number of output bits of the last fully-connected layer in the decoder is one, and is used for representing the parameters of the irrigation water pressure of the irrigation device, and in the training process of the decoder, the difference between the output value and the real value of the decoder is used for calculating a difference loss function, and the parameters of the encoder and the decoder are updated through back propagation. Here, during each iteration, the encoder may be trained first by an orthogonalization loss function and then by a difference loss function together with the decoder, so that the parameters of the decoder are also suitable for decoding operations based on eigenvalues that are orthogonal to each other.
Based on this, the present application proposes a training method of a neural network for irrigation water pressure control of an irrigation device, comprising: acquiring a training image, wherein the training image is an image of an area to be irrigated; passing the training image through an encoder to obtain a training feature vector, wherein the encoder comprises a deep convolutional neural network, the last layer of the deep convolutional neural network is a fully-connected layer, and the training feature vector is output by the fully-connected layer; calculating an orthogonalization loss function value of the training feature vector, the orthogonalization loss function value being a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix; passing the training feature vector through a decoder to obtain decoded values, the decoder comprising a plurality of fully-connected layers, the output bit of the last fully-connected layer in the decoder being one; calculating a difference loss function value between the decoded value and the real value; and updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value.
Based on this, the present application also proposes an irrigation water pressure control method based on a deep neural network, which includes: acquiring an image of an irrigation area to be detected; inputting the image into an encoder and a decoder trained according to the training method for the neural network for irrigation water pressure control of the irrigation device as described above, the output of the last fully-connected layer in the decoder representing a parameter of irrigation water pressure of the irrigation device; and adjusting the irrigation water pressure of the irrigation device based on the parameter of the irrigation water pressure.
Fig. 1 illustrates an application scenario diagram of a training method of a neural network for irrigation water pressure control of an irrigation device and an irrigation water pressure control method based on a deep neural network according to an embodiment of the application.
As shown in fig. 1, in the training phase of the application scenario, an image of an area to be watered is acquired as a training image through a camera (e.g., C as illustrated in fig. 1), for example, a camera deployed in an unmanned aerial vehicle; the training images are then input into a server (e.g., S as illustrated in fig. 1) deployed with a training algorithm for an irrigation water pressure controlled neural network for an irrigation device, wherein the server is capable of training the irrigation water pressure controlled neural network for the irrigation device with the training images based on the training algorithm for the irrigation water pressure controlled neural network for the irrigation device.
After training the neural network by the training algorithm of the neural network for irrigation water pressure control of the irrigation device as described above, irrigation water pressure control of the irrigation device may be detected based on the deep neural network.
Further, as shown in fig. 1, in the application stage of the application scenario, an image of the irrigation area to be detected is acquired through a camera (e.g., as indicated by C in fig. 1); the image is then input into a server (e.g., S as illustrated in fig. 1) deployed with a deep neural network-based irrigation water pressure control algorithm, wherein the server is capable of processing the image based on the deep neural network-based irrigation water pressure control algorithm to generate irrigation water pressure parameters and adjusting irrigation water pressure of an irrigation device based on the irrigation water pressure parameters.
Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary method
Fig. 2 illustrates a flow chart of a training method for an irrigation water pressure controlled neural network for an irrigation device. As shown in fig. 2, a training method of an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application includes: s110, acquiring a training image, wherein the training image is an image of an area to be irrigated; s120, passing the training image through an encoder to obtain a training feature vector, wherein the encoder comprises a deep convolutional neural network, the last layer of the deep convolutional neural network is a full-connection layer, and the training feature vector is output by the full-connection layer; s130, calculating an orthogonalization loss function value of the training feature vector, wherein the orthogonalization loss function value is a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix; s140, passing the training feature vector through a decoder to obtain a decoded value, wherein the decoder comprises a plurality of fully-connected layers, and the output bit of the last fully-connected layer in the decoder is one; s150, calculating a difference loss function value between the decoded value and the real value; s160, updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value.
Fig. 3 illustrates an architectural schematic of a training method for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application. As shown IN fig. 3, IN the network architecture of the training method of the neural network for irrigation water pressure control of irrigation devices, first, a training image (e.g., IN1 as illustrated IN fig. 3) of an area to be irrigated acquired by a camera is passed through an encoder (e.g., CNN as illustrated IN fig. 3) to obtain a training feature vector (e.g., V1 as illustrated IN fig. 3); then, calculating an orthogonalization loss function value of the training feature vector (e.g., N1 as illustrated in fig. 3); then, passing the training feature vector through a decoder (e.g., a decoder as illustrated in fig. 3) to obtain a decoded value (e.g., N2 as illustrated in fig. 3); then, a difference loss function value (e.g., circle L as illustrated in fig. 3) between the decoded value and the real value is calculated; then, parameters of the encoder and the decoder are updated based on the difference loss function value and the orthogonalization loss function value.
In step S110, a training image is obtained, where the training image is an image of an area to be irrigated. As described above, considering that both the topographic condition and the height condition of the work can be relatively easily recognized from the image of the area to be irrigated, the inventors of the present application have desired to perform control of the irrigation water pressure of the irrigation device by a computer vision method based on deep learning. That is, the camera is used to collect the image of the area to be irrigated as the training image.
In particular implementations, images of the area to be watered may be acquired by a camera deployed on the drone. It should be appreciated that because aerial cameras have different capture heights when captured in the air, the captured images have different dimensions. Accordingly, adjustments may be made based on the depth of field. Accordingly, in the embodiment of the present application, a process of acquiring a training image, which is an image of an area to be irrigated, includes: firstly, acquiring an image of an area to be irrigated through a camera arranged on an unmanned aerial vehicle, namely acquiring the image of the area to be irrigated through an aerial photography device; then, based on the depth of field value of the unmanned aerial vehicle camera when collecting the area to be irrigated, the training image is adjusted to be an image with a uniform scale.
As will be appreciated by those of ordinary skill in the art, depth of field (DOF), refers to the range of distances before and after a subject measured at the front of a camera lens or other imager where a sharp image can be taken. The farther away the distance, the deeper the depth of field; the closer the distance (which cannot be smaller than the minimum shooting distance), the shallower the depth of field.
Fig. 4 illustrates a flow chart of acquiring a training image of an area to be irrigated in a training method of a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application. As shown in fig. 4, a training image is obtained, where the training image is an image of an area to be irrigated, and includes: s210, collecting an image of an area to be irrigated through a camera deployed on an unmanned aerial vehicle; and S220, based on the depth of field value of the unmanned aerial vehicle camera in the process of collecting the area to be irrigated, adjusting the training image into an image with a uniform scale.
In step S120, the training image is passed through an encoder to obtain a training feature vector, where the encoder includes a deep convolutional neural network, a last layer of the deep convolutional neural network is a fully-connected layer, and the training feature vector is output by the fully-connected layer. That is, the encoder extracts the high-dimensional features in the training image. The feature map of the area to be irrigated is extracted by an encoder, i.e. a convolutional neural network, and the last layer of the convolutional neural network is set as a fully connected layer, so as to encode the feature map of the area to be irrigated into a training feature vector of a predetermined length.
Specifically, in the embodiment of the present application, the process of passing the training image through an encoder to obtain a training feature vector includes: firstly, the training image is passed through a plurality of convolutional layers, a plurality of pooling layers and a plurality of activation layers in the deep convolutional neural network to obtain a convolutional feature map, it should be understood that the convolutional layers perform dimension reduction and feature extraction on the input training image through convolutional operation, the pooling layers perform pooling operation on the feature image, for example, mean pooling or maximum pooling, so that the image size is further reduced, the calculation speed is increased, the activation layers map the features to a high-dimensional nonlinear interval for interpretation, and the problem that cannot be solved by a linear model is solved. The convolved feature map is then passed through the fully-connected layer in the deep convolutional neural network to obtain the training feature vector, i.e., the learned "distributed feature representation" is mapped to the sample label space through the fully-connected layer, it being understood that the last layer of the convolutional neural network is set for the purpose of the fully-connected layer: and encoding the feature map of the area to be irrigated into a training feature vector with a preset length.
In particular, the convolutional neural network may employ a deep residual neural network, e.g., ResNet 50. It should be known to those skilled in the art that, compared to the conventional convolutional neural network, the deep residual network is an optimized network structure proposed on the basis of the conventional convolutional neural network, which mainly solves the problem of gradient disappearance during the training process. The depth residual error network introduces a residual error network structure, the network layer can be made deeper through the residual error network structure, and the problem of gradient disappearance can not occur. The residual error network uses the cross-layer link thought of a high-speed network for reference, breaks through the convention that the traditional neural network only can provide N layers as input from the input layer of the N-1 layer, enables the output of a certain layer to directly cross several layers as the input of the later layer, and has the significance of providing a new direction for the difficult problem that the error rate of the whole learning model is not reduced and inversely increased by superposing multiple layers of networks.
Fig. 5 illustrates a flow chart of passing the training image through an encoder to obtain a training feature vector in a training method of a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application. As shown in fig. 5, passing the training image through an encoder to obtain a training feature vector includes: s310, enabling the training image to pass through a multilayer convolution layer, a multilayer pooling layer and a multilayer activation layer in the deep convolutional neural network to obtain a convolution characteristic diagram; and S320, passing the convolution feature map through the full-connection layer in the deep convolution neural network to obtain the training feature vector.
In step S130, an orthogonalization loss function value of the training feature vector is calculated, the orthogonalization loss function value being a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix. As described above, in order to remove unnecessary information in the training eigenvector as much as possible, when the encoder is trained, the parameters of the encoder are updated by constructing an orthogonalization loss function, which is the difference between the unit matrix and the product of the training eigenvector and the transpose of the training eigenvector, so that the encoder can extract and encode the features of the region to be irrigated as much as possible into eigenvalues orthogonal to each other.
It should be appreciated that the orthogonalization loss function value represents the probability that the training feature vector fits its orthogonalization matrix. Here, the orthogonalization matrix of the training feature vector means that the product of the training feature vector and its transpose is obtained by subtracting an identity matrix. Therefore, the encoder is updated through the orthogonalization loss function, so that the encoder focuses on extracting the features which are orthogonal to each other in the image to be irrigated, the feature characterization capability is improved, and the calculation amount is reduced.
In step S140, the training feature vector is passed through a decoder to obtain a decoded value, the decoder includes a plurality of fully-connected layers, and an output bit of a last fully-connected layer in the decoder is one. That is, the training feature vector is passed through a decoder consisting of a plurality of fully-connected layers, wherein the number of output bits of the last fully-connected layer in the decoder is one for a parameter representing the irrigation water pressure of the irrigation device.
In step S150, a difference loss function value between the decoded value and the true value is calculated. I.e. using the difference between the output value of the decoder and the true value as a difference loss function value representing the probability that the output value of the decoder will correspond to the true value, it will be appreciated that updating the parameters of the encoder and the decoder by back-propagating the difference loss function value results in a more accurate output result for the model.
Specifically, in the embodiment of the present application, calculating the difference loss function value between the decoded value and the real value includes: an L1 difference loss function value is calculated between the decoded value and the true value. Those of ordinary skill in the art will appreciate that the L1 loss function, also known as the minimum absolute deviation (LAD), is the sum of the absolute differences of the target value and the estimated value. By calculating the L1 difference loss function value between the decoded value and the real value, the characteristic difference between the decoded value and the real value can be reflected from the numerical dimension, so that the part with the overlarge difference in the numerical dimension can be excluded in the training process.
It is worth mentioning that in other examples of the present application, the difference loss function value between the decoded value and the true value may also be calculated in other ways. For example, in another example of the present application, calculating a difference loss function value between the decoded value and a true value comprises: an L2 difference loss function value is calculated between the decoded value and the true value. Therein, the L2 loss function, also called Least Squares Error (LSE), is the sum of the squares of the differences between the target and estimated values, also called euclidean distance. It should be appreciated that the L2 loss function has a stable solution compared to the L1 loss function.
In another example, the characteristic difference between the decoded value and the real value can be reflected from the spatial distance dimension by calculating the L2 distance between the decoded value and the real value, so that the part with too large difference in the spatial distance dimension is excluded from the training process.
In step S160, parameters of the encoder and the decoder are updated based on the difference loss function value and the orthogonalization loss function value. That is, an encoder and a decoder are trained based on the difference loss function value and the orthogonalization loss function value.
Specifically, in this embodiment of the present application, the process of updating the parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value includes: in each iteration, the parameters of the encoder are first updated based on the orthogonalization loss function values, that is, the parameters of the encoder are updated by constructing the orthogonalization loss function, so that the encoder can extract and encode the features of the region to be irrigated into feature values orthogonal to each other as much as possible. Then, parameters of the encoder and the decoder are updated with the difference loss function values so that the parameters of the decoder are also suitable for a decoding operation based on feature values orthogonal to each other.
Fig. 6 illustrates a flow chart for updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value in a training method of a neural network for irrigation water pressure control of an irrigation device according to an embodiment of the present application. As shown in fig. 6, updating the parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value includes: s410, updating parameters of the encoder based on the orthogonalization loss function values; and, S420, updating parameters of the encoder and the decoder with the difference loss function values.
According to another aspect of the application, an irrigation water pressure control method based on a deep neural network is also provided.
FIG. 7 illustrates a flow chart of a method of irrigation water pressure control based on a deep neural network according to an embodiment of the present application. As shown in fig. 7, a method for controlling irrigation water pressure based on a deep neural network according to an embodiment of the present application includes: s510, acquiring an image of an irrigation area to be detected; s520, inputting the image into an encoder and a decoder which are trained according to the training method of the neural network for irrigation water pressure control of the irrigation device, wherein the output of the last fully-connected layer in the decoder represents parameters of irrigation water pressure of the irrigation device; and S530, adjusting the irrigation water pressure of the irrigation device based on the irrigation water pressure parameter.
In summary, a training method of a neural network for irrigation water pressure control of an irrigation device and an irrigation water pressure control method based on a deep neural network are explained, which control irrigation water pressure of the irrigation device based on a deep learning computer vision method. Specifically, in the training process of the neural network for irrigation water pressure control of the irrigation device, in order to remove useless information in the feature vector as much as possible, the parameters of the encoder are updated by constructing an orthogonalization loss function when the encoder is trained, and therefore the accuracy of the model is improved.
Exemplary System
FIG. 8 illustrates a block diagram of a training system for an irrigation water pressure controlled neural network for an irrigation device according to an embodiment of the present application.
As shown in fig. 8, a training system 800 for an irrigation water pressure controlled neural network of an irrigation device according to an embodiment of the present application includes: a training image obtaining unit 810, configured to obtain a training image, where the training image is an image of an area to be irrigated; a training feature vector generating unit 820, configured to pass the training image obtained by the training image obtaining unit 810 through an encoder to obtain a training feature vector, where the encoder includes a deep convolutional neural network, a last layer of the deep convolutional neural network is a fully-connected layer, and the training feature vector is output by the fully-connected layer; an orthogonalization loss function value calculating unit 830, configured to calculate an orthogonalization loss function value of the training feature vector obtained by the training feature vector generating unit 820, where the orthogonalization loss function value is a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix; a decoded value generating unit 840, configured to pass the training eigenvector obtained by the training eigenvector generating unit 820 through a decoder to obtain a decoded value, where the decoder includes multiple fully-connected layers, and an output bit of a last fully-connected layer in the decoder is one; a difference loss function value calculation unit 850 for calculating a difference loss function value between the decoded value obtained by the decoded value generation unit 840 and a true value; a parameter updating unit 860 for updating parameters of the encoder and the decoder based on the difference loss function value obtained by the difference loss function value calculating unit 850 and the orthogonalizing loss function value obtained by the orthogonalizing loss function value calculating unit 830.
In an example, in the training system 800 of the neural network for irrigation water pressure control of an irrigation device, as shown in fig. 9, the training feature vector generating unit 820 includes: a convolution feature map generation subunit 821, configured to pass the training image through a plurality of convolution layers, a plurality of pooling layers, and a plurality of activation layers in the deep convolutional neural network to obtain a convolution feature map; and a convolutional neural network processing subunit 822, configured to pass the convolutional feature map obtained by the convolutional feature map generating subunit 821 through the fully-connected layer in the deep convolutional neural network to obtain the training feature vector.
In an example, in the training system 800 of the neural network for irrigation water pressure control of an irrigation device, the difference loss function value calculating unit 850 is further configured to: an L1 difference loss function value is calculated between the decoded value and the true value.
In an example, in the training system 800 of the neural network for irrigation water pressure control of an irrigation device, the difference loss function value calculating unit 850 is further configured to: an L2 difference loss function value is calculated between the decoded value and the true value.
In an example, in the training system 800 of the neural network for irrigation water pressure control of an irrigation device, as shown in fig. 10, the parameter updating unit 860 includes: an encoder parameter update subunit 861 configured to update a parameter of the encoder based on the orthogonalization loss function value; and a decoder parameter updating subunit 862 for updating parameters of the encoder and the decoder with the difference loss function values.
In an example, in the training system 800 of the neural network for irrigation water pressure control of an irrigation device, as shown in fig. 11, the training image obtaining unit 810 includes: the image acquisition subunit 811 is used for acquiring an image of the area to be irrigated by a camera deployed on the unmanned aerial vehicle; and an image adjusting subunit 812, configured to adjust the training image obtained by the training image obtaining unit 810 into an image with a uniform scale based on a depth of field value of the camera of the unmanned aerial vehicle when acquiring the area to be irrigated.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the training system 800 described above have been described in detail in the description of the training method of the neural network for irrigation water pressure control of an irrigation device with reference to fig. 1 to 6, and thus, a repetitive description thereof will be omitted.
As described above, the training system 800 according to the embodiment of the present application may be implemented in various terminal devices, such as a server for irrigation water pressure control of an irrigation device, and the like. In one example, the training system 800 according to embodiments of the present application may be integrated into a terminal device as a software module and/or a hardware module. For example, the training system 800 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the training system 800 may also be one of many hardware modules of the terminal device.
Alternatively, in another example, the training system 800 and the terminal device may be separate devices, and the training system 800 may be connected to the terminal device through a wired and/or wireless network and transmit the interaction information according to an agreed data format.
According to another aspect of the present application, there is also provided an irrigation water pressure control system based on a deep neural network.
FIG. 12 illustrates a block diagram of a deep neural network based irrigation water pressure control system according to an embodiment of the present application. As shown in fig. 12, the irrigation water pressure control system 900 based on the deep neural network according to the embodiment of the present application includes: an image acquiring unit 910 to be detected, configured to acquire an image of an irrigation area to be detected; a parameter generating unit 920, configured to input the image obtained by the image obtaining unit 910 to be detected into an encoder and a decoder trained according to the above-mentioned training method for the neural network for controlling the irrigation water pressure of the irrigation device, where an output of a last fully-connected layer in the decoder represents a parameter of the irrigation water pressure of the irrigation device; and an irrigation water pressure adjusting unit 930 for adjusting irrigation water pressure of the irrigation device based on the parameter of the irrigation water pressure obtained by the parameter generating unit 920.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described control system 900 have been described in detail in the above-described irrigation water pressure control method based on the deep neural network with reference to fig. 7, and thus, a repetitive description thereof will be omitted.
As described above, the control system 900 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for irrigation water pressure control of irrigation equipment, and the like. In one example, the control system 900 according to the embodiments of the present application may be integrated into the terminal device as one software module and/or hardware module. For example, the control system 900 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the control system 900 may also be one of many hardware modules of the terminal device.
Alternatively, in another example, the control system 900 and the terminal device may be separate devices, and the control system 900 may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information according to an agreed data format.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 13.
FIG. 13 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
As shown in fig. 13, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
In one example, the electronic device 10 may further include: an input system 13 and an output system 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input system 13 may comprise, for example, a keyboard, a mouse, etc.
The output system 14 may output various information including decoded values and the like to the outside. The output system 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 13, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Claims (10)
1. A method of training an irrigation water pressure controlled neural network for an irrigation device, comprising:
acquiring a training image, wherein the training image is an image of an area to be irrigated;
passing the training image through an encoder to obtain a training feature vector, wherein the encoder comprises a deep convolutional neural network, the last layer of the deep convolutional neural network is a fully-connected layer, and the training feature vector is output by the fully-connected layer;
calculating an orthogonalization loss function value of the training feature vector, the orthogonalization loss function value being a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix;
passing the training feature vector through a decoder to obtain decoded values, the decoder comprising a plurality of fully-connected layers, the output bit of the last fully-connected layer in the decoder being one;
calculating a difference loss function value between the decoded value and the real value; and
updating parameters of the encoder and the decoder based on the difference loss function value and the orthogonalization loss function value.
2. The method of training a neural network for irrigation water pressure control of an irrigation device of claim 1, wherein passing the training image through an encoder to obtain a training feature vector comprises:
passing the training image through multiple convolutional layers, multiple pooling layers and multiple activation layers in the deep convolutional neural network to obtain a convolutional feature map; and
passing the convolved feature map through the fully-connected layer in the deep convolutional neural network to obtain the training feature vector.
3. The method of claim 1, wherein calculating the difference loss function value between the decoded value and the true value comprises:
an L1 difference loss function value is calculated between the decoded value and the true value.
4. The method of claim 1, wherein calculating the difference loss function value between the decoded value and the true value comprises:
an L2 difference loss function value is calculated between the decoded value and the true value.
5. The method of training a neural network for irrigation water pressure control of an irrigation device of claim 1, wherein updating parameters of the encoder and the decoder based on the difference loss function values and the orthogonalization loss function values comprises: in each of the rounds of the iteration(s),
updating parameters of the encoder based on the orthogonalization loss function values; and
updating parameters of the encoder and the decoder with the difference loss function values.
6. The method of training an irrigation water pressure controlled neural network for an irrigation device of claim 1, wherein obtaining a training image, the training image being an image of an area to be irrigated, comprises:
collecting an image of an area to be irrigated through a camera deployed on an unmanned aerial vehicle; and
and adjusting the training image into an image with a uniform scale based on the depth of field value of the unmanned aerial vehicle camera in the process of collecting the area to be irrigated.
7. An irrigation water pressure control method based on a deep neural network is characterized by comprising the following steps:
acquiring an image of an irrigation area to be detected;
inputting said image into an encoder and decoder trained according to the method of any one of claims 1 to 6 for training a neural network for irrigation water pressure control of an irrigation device, the output of the last fully-connected layer in the decoder representing a parameter of irrigation water pressure of the irrigation device; and
adjusting the irrigation water pressure of the irrigation device based on the parameter of the irrigation water pressure.
8. A training system for an irrigation water pressure controlled neural network for an irrigation device, comprising:
the training image acquisition unit is used for acquiring a training image, wherein the training image is an image of an area to be irrigated;
a training feature vector generation unit, configured to pass the training image obtained by the training image obtaining unit through an encoder to obtain a training feature vector, where the encoder includes a deep convolutional neural network, a last layer of the deep convolutional neural network is a fully connected layer, and the training feature vector is output by the fully connected layer;
an orthogonalization loss function value calculating unit configured to calculate an orthogonalization loss function value of the training feature vector obtained by the training feature vector generating unit, where the orthogonalization loss function value is a mean square error between a product of the training feature vector and a transpose of the training feature vector and an identity matrix;
a decoded value generating unit, configured to pass the training eigenvector obtained by the training eigenvector generating unit through a decoder to obtain a decoded value, where the decoder includes multiple fully-connected layers, and an output bit of a last fully-connected layer in the decoder is one;
a difference loss function value calculation unit configured to calculate a difference loss function value between the decoded value obtained by the decoded value generation unit and a true value; and
a parameter updating unit configured to update parameters of the encoder and the decoder based on the difference loss function value obtained by the difference loss function value calculating unit and the orthogonalization loss function value obtained by the orthogonalization loss function value calculating unit.
9. An irrigation water pressure control system based on a deep neural network, comprising:
the to-be-detected image acquisition unit is used for acquiring an image of an irrigation area to be detected;
a parameter generating unit, for inputting the image obtained by the image obtaining unit to be detected into an encoder and a decoder trained according to the training method of the neural network for irrigation water pressure control of the irrigation device according to any one of claims 1 to 6, wherein the output of the last fully-connected layer in the decoder represents a parameter of irrigation water pressure of the irrigation device; and
and the irrigation water pressure adjusting unit is used for adjusting the irrigation water pressure of the irrigation device based on the irrigation water pressure parameters.
10. An electronic device, comprising:
a processor; and
a memory in which computer program instructions are stored, which, when executed by the processor, cause the processor to carry out the method of training of a neural network for irrigation water pressure control of an irrigation device according to any one of claims 1-6 or the method of irrigation water pressure control based on a deep neural network according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110024413.3A CN112733710A (en) | 2021-01-08 | 2021-01-08 | Method for training a neural network for irrigation water pressure control of an irrigation device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110024413.3A CN112733710A (en) | 2021-01-08 | 2021-01-08 | Method for training a neural network for irrigation water pressure control of an irrigation device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112733710A true CN112733710A (en) | 2021-04-30 |
Family
ID=75589920
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110024413.3A Withdrawn CN112733710A (en) | 2021-01-08 | 2021-01-08 | Method for training a neural network for irrigation water pressure control of an irrigation device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733710A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113141940A (en) * | 2021-06-01 | 2021-07-23 | 中国农业科学院蔬菜花卉研究所 | Intelligent water precise irrigation control system and method for fruit and vegetable cultivation in sunlight greenhouse |
| CN113469249A (en) * | 2021-06-30 | 2021-10-01 | 阿波罗智联(北京)科技有限公司 | Image classification model training method, classification method, road side equipment and cloud control platform |
| CN117882546A (en) * | 2024-03-13 | 2024-04-16 | 山西诚鼎伟业科技有限责任公司 | Intelligent planting method for agricultural operation robot |
-
2021
- 2021-01-08 CN CN202110024413.3A patent/CN112733710A/en not_active Withdrawn
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113141940A (en) * | 2021-06-01 | 2021-07-23 | 中国农业科学院蔬菜花卉研究所 | Intelligent water precise irrigation control system and method for fruit and vegetable cultivation in sunlight greenhouse |
| CN113469249A (en) * | 2021-06-30 | 2021-10-01 | 阿波罗智联(北京)科技有限公司 | Image classification model training method, classification method, road side equipment and cloud control platform |
| CN113469249B (en) * | 2021-06-30 | 2024-04-09 | 阿波罗智联(北京)科技有限公司 | Image classification model training method, classification method, road side equipment and cloud control platform |
| CN117882546A (en) * | 2024-03-13 | 2024-04-16 | 山西诚鼎伟业科技有限责任公司 | Intelligent planting method for agricultural operation robot |
| CN117882546B (en) * | 2024-03-13 | 2024-05-24 | 山西诚鼎伟业科技有限责任公司 | Intelligent planting method for agricultural operation robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Eldesokey et al. | Propagating confidences through cnns for sparse data regression | |
| CN112733710A (en) | Method for training a neural network for irrigation water pressure control of an irrigation device | |
| CN111539370A (en) | Image pedestrian re-identification method and system based on multi-attention joint learning | |
| CN113936256A (en) | Image target detection method, device, equipment and storage medium | |
| CN114049381A (en) | Twin cross target tracking method fusing multilayer semantic information | |
| Zhao et al. | Stacked multilayer self-organizing map for background modeling | |
| CN116343330A (en) | Abnormal behavior identification method for infrared-visible light image fusion | |
| CN112528974B (en) | Distance measuring method and device, electronic equipment and readable storage medium | |
| CN110570443A (en) | Image linear target extraction method based on structural constraint condition generation model | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN111833360B (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN112052771A (en) | Object re-identification method and device | |
| CN111291695B (en) | Training method and recognition method for recognition model of personnel illegal behaviors and computer equipment | |
| CN112396036A (en) | Method for re-identifying blocked pedestrians by combining space transformation network and multi-scale feature extraction | |
| Liu et al. | Two-stream refinement network for RGB-D saliency detection | |
| CN112085680B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN113657200A (en) | Video behavior action identification method and system based on mask R-CNN | |
| EP4287145A1 (en) | Statistical model-based false detection removal algorithm from images | |
| CN113792660B (en) | Pedestrian detection method, system, medium and equipment based on improved YOLOv3 network | |
| CN116797814A (en) | Intelligent building site safety management system | |
| CN112419387B (en) | Unsupervised depth estimation method for solar greenhouse tomato plant image | |
| JP2023063768A (en) | Image descriptor update model and method using graph structure, image matching apparatus, and program | |
| CN114445881A (en) | Face detection method and related equipment | |
| EP3401843A1 (en) | A method, an apparatus and a computer program product for modifying media content | |
| CN113537359A (en) | Training data generation method and device, computer readable medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210430 |