CN111507399A - Cloud recognition and model training method, device, terminal and medium based on deep learning - Google Patents
Cloud recognition and model training method, device, terminal and medium based on deep learning Download PDFInfo
- Publication number
- CN111507399A CN111507399A CN202010300620.2A CN202010300620A CN111507399A CN 111507399 A CN111507399 A CN 111507399A CN 202010300620 A CN202010300620 A CN 202010300620A CN 111507399 A CN111507399 A CN 111507399A
- Authority
- CN
- China
- Prior art keywords
- cloud
- image
- deep learning
- training
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 84
- 238000013135 deep learning Methods 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000013145 classification model Methods 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 25
- 238000010606 normalization Methods 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 6
- 238000005096 rolling process Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 10
- 238000013528 artificial neural network Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000002093 peripheral effect Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 235000019800 disodium phosphate Nutrition 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Abstract
The invention provides a cloud identification and model training method, a cloud identification and model training device, a cloud identification and model training terminal and a medium based on deep learning, wherein the cloud identification and model training method comprises the following steps: acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image; inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training, and accordingly obtaining classification confidence coefficients of the cloud image corresponding to a plurality of cloud categories; if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the cloud image to be recognized; otherwise, determining that the cloud of the cloud category does not appear in the cloud image to be identified. The technical scheme of the invention is an intelligent identification method based on deep learning, and a multi-label classification deep neural network is used, so that clouds of multiple categories can be judged simultaneously, and a more intelligent technical solution with better cloud observation effect is provided.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a cloud identification and model training method, a cloud identification and model training device, a cloud identification and model training terminal and a cloud identification and model training medium based on deep learning.
Background
In the field of aeronautical weather, cloud identification is an extremely important ring, cloud classification plays an important role in weather identification and weather disaster prediction, and part of low cloud families can directly influence the flight taking off and landing; therefore, cloud information in the meteorological observation report is always a part of great concern for airports and airlines.
In the prior art, people are usually used for cloud observation, but the cloud observation needs long-term training and is very time-consuming and labor-consuming; some prior arts also adopt detection and identification software to perform cloud observation, but because the shape change of the cloud itself is complex, multiple types of clouds often appear in the visual field, and the existing detection and identification software has poor cloud effect.
Therefore, a technical solution with more intelligence and better cloud observation effect is needed in the art.
Content of application
In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a cloud recognition and model training method, apparatus, terminal and medium based on deep learning, which are used to solve the problems in the prior art.
To achieve the above and other related objects, a first aspect of the present invention provides a cloud identification method based on deep learning, including: acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image; inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training, and accordingly obtaining classification confidence coefficients of the cloud image corresponding to a plurality of cloud categories; if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the cloud image to be recognized; otherwise, determining that the cloud of the cloud category does not appear in the cloud image to be identified.
In some embodiments of the first aspect of the present invention, the image pre-processing comprises an image normalization process comprising: and respectively subtracting the corresponding preset channel data from the three color channel data of each pixel point in the cloud image, and zooming to the preset pixel size.
In some embodiments of the first aspect of the present invention, the cloud-like classes of the cloud include some or all of high-lying clouds, rolling clouds, raining clouds, rolling clouds, layer clouds, and layering clouds, and the classification network model includes a Resnet classification network model including convolution layers, BatchNorm layers, Scale layers, Re L U active layers, Eltwiese layers, pooling layers, at least one full-connectivity layer, and at least one Sigmoid active layer.
To achieve the above and other related objects, a second aspect of the present invention provides a deep learning-based model training method for training a cloud recognition multi-label classification model in the first aspect of the present invention; the method comprises the following steps: collecting a plurality of cloud images at different times and different categories; marking category labels for each cloud in each image, and converting the marked category labels into corresponding label data; and constructing a classification network model, and training the classification network model by taking the cloud images and the corresponding label data thereof as training data until the classification network model converges.
In some embodiments of the first aspect of the present invention, the training the classification network model with the cloud images and their corresponding label data as training data until convergence comprises: carrying out normalization processing on the cloud-shaped image; inputting the normalized image data and the corresponding label data into a classification network model, and calculating a binary cross entropy loss value; and determining the convergence of the classification network model under the condition that the binary cross entropy loss value is smaller than a preset loss value.
To achieve the above and other related objects, a third aspect of the present invention provides a cloud recognition apparatus based on deep learning, including: the image acquisition module is used for acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image; the image recognition module is used for inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training so as to obtain classification confidence coefficients of the cloud image corresponding to a plurality of cloud categories; if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the image to be recognized; otherwise, determining that the cloud of the cloud category does not appear in the image to be recognized.
To achieve the above and other related objects, a fourth aspect of the present invention provides a deep learning-based model training apparatus, comprising: the training data acquisition module is used for acquiring a plurality of cloud images at different moments and different categories; the label labeling module is used for labeling the category labels for the clouds in each image and converting the labeled category labels into corresponding label data; and the model training module is used for constructing a classification network model and training the classification network model by taking the cloud images and the corresponding label data thereof as training data until the classification network model is converged.
To achieve the above and other related objects, a fifth aspect of the present invention provides a computer-readable storage medium having stored thereon a first computer program and/or a second computer program; the first computer program, when executed by a processor, implements the deep learning based cloud identification method of the first aspect of the invention; the second computer program, when executed by a processor, implements the deep learning based model training method of the second aspect of the invention.
To achieve the above and other related objects, a sixth aspect of the present invention provides a cloud recognition terminal based on deep learning, comprising: a processor and a memory; the memory is used for storing a computer program; the processor is configured to execute the computer program stored in the memory, so as to enable the terminal to execute the cloud identification method based on deep learning of the first aspect of the present invention.
To achieve the above and other related objects, a seventh aspect of the present invention provides a deep learning based model training terminal, which includes: a processor and a memory; the memory is used for storing a computer program; the processor is configured to execute the computer program stored in the memory to cause the terminal to perform the deep learning based model training method of the second aspect of the present invention.
As described above, the cloud recognition and model training method, device, terminal and medium based on deep learning according to the present invention have the following beneficial effects: the technical scheme of the invention is an intelligent identification method based on deep learning, and a multi-label classification deep neural network is used, which can judge clouds of multiple categories at the same time, thereby being a more intelligent technical solution with better cloud observation effect.
Drawings
Fig. 1 is a flowchart illustrating a cloud identification method based on deep learning according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating a deep learning-based model training method according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a cloud identification apparatus based on deep learning according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a deep learning-based model training apparatus according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of a cloud identification terminal based on deep learning according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of a model training terminal based on deep learning according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It is noted that in the following description, reference is made to the accompanying drawings which illustrate several embodiments of the present invention. It is to be understood that other embodiments may be utilized and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description is not to be taken in a limiting sense, and the scope of embodiments of the present invention is defined only by the claims of the issued patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Spatially relative terms, such as "upper," "lower," "left," "right," "lower," "below," "lower," "above," "upper," and the like, may be used herein to facilitate describing one element or feature's relationship to another element or feature as illustrated in the figures.
In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "secured," "retained," and the like are to be construed broadly, e.g., as meaning fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," and/or "comprising," when used in this specification, specify the presence of stated features, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, operations, elements, components, items, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions or operations are inherently mutually exclusive in some way.
The existing cloud distinguishing method is mostly realized by adopting manual observation or a simple algorithm, but the manual observation method is time-consuming, labor-consuming and easy to make mistakes, some existing cloud identification algorithms can only identify one type of cloud in the current image generally, or the cloud identification can be realized only by detecting the position of the cloud in advance, so that the identification method is complicated and cannot reflect the whole condition of the sky.
In view of this, the invention provides a cloud identification and model training method, device, terminal and medium based on deep learning, so as to effectively solve the technical problems of time and labor consumption, poor detection effect and the like in the existing cloud identification method. The technical scheme of the invention is an intelligent identification method based on deep learning, and a multi-label classification deep neural network is used, so that clouds of multiple categories can be judged at the same time.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention are further described in detail by the following embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example one
Fig. 1 is a schematic flow chart illustrating a cloud identification method based on deep learning according to an embodiment of the present invention. The cloud identification method of the embodiment mainly includes steps S11 to S15.
Step S11: and acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image.
Optionally, the cloud image to be recognized is acquired by an image acquisition device such as a video camera, a camera module integrated with an optical system or a CCD chip, a camera module integrated with an optical system and a CMOS chip, and the like. In consideration of the field of aeronautical weather to which the present invention is applicable, an airport observation station or a weather monitoring station, etc. may be selected as a collection position to collect a picture of the surrounding sky, so as to collect a cloud-like image, which is not limited in this embodiment.
For example, the three color channel data of each pixel point in the cloud image can be respectively subtracted by the corresponding preset channel data and scaled to the preset pixel size, for example, the RGB three channel data of each pixel point in the whole image can be respectively subtracted by 104, 117 and 124 and scaled to the pixel size of 224 × 224.
Step S12: inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training, and accordingly obtaining classification confidence degrees of the cloud image corresponding to a plurality of cloud categories.
The model comprises 53 convolutional layers, 50 BatchNorm layers, 50 Scale layers, 49 Re L U active layers, 16 Eltwise layers, 2 pooling layers, 1 full connection layer and 1 sigmoid active layer by taking the ResNet neural network model as an example.
It should be noted that the ResNet neural network adopted in this embodiment is different from a general classification network, and a Sigmoid activation layer is added behind a last full connection layer, and is used to convert a multidimensional float array output of the full connection layer into a multidimensional 0-1 probability array and then output the multidimensional 0-1 probability array (for example, a 10-dimensional float array is converted into a 10-dimensional 0-1 probability array), where an ith value in the array represents a probability (i ═ 0,1,2,3,4,5,6,7,8,9) that an ith class of cloud exists in a picture predicted by the network.
The eltwise layer is pixel-by-pixel operation, and comprises three types of operations of dot product (product), summation (sum) and maximum value (max), and is set to pixel-by-pixel addition in the invention; the flatten layer is a network layer for unifying multidimensional input; the batchnorm layer is a network layer for carrying out normalization operation on network data; the Scale layer is a network layer that scales and shifts network data.
Optionally, the plurality of cloud categories of the cloud include: high cloud, rolling cloud, raincloud, cumulus cloud, rain cloud, layer cloud, or some or all of the types of layering cloud. And after the preprocessed cloud image is input into a cloud recognition multi-label classification model, the classification confidence degrees of the cloud image corresponding to the classes are output.
Step S13: and judging whether the confidence coefficient of the cloud image corresponding to a cloud class is greater than a preset threshold value of the confidence coefficient of the cloud class. Specifically, according to the trained cloud identification multi-label classification model, a threshold value P enabling each cloud type to obtain the best detection effect can be calculated, and the threshold value Pi represents a confidence coefficient threshold value of the ith cloud type.
Step S14: and if the confidence coefficient is greater than the preset threshold value of the cloud type, determining that the cloud of the cloud type appears in the image to be recognized.
Step S15: and if the confidence coefficient is not greater than the preset threshold value of the cloud type, determining that the cloud of the cloud type does not appear in the cloud image to be recognized.
For example, the preprocessed cloud-like image is input into a cloud-like recognition multi-label classification model which is subjected to deep learning training, classification confidences of the cloud-like image corresponding to 10 cloud-like categories of high-lying cloud, high-layer cloud, rolling cloud, raincloud, lying cloud, accumulated raincloud, layered cloud and layered cloud are respectively obtained and recorded as T0、T1、……T9(ii) a Each cloud type is respectively provided with a preset threshold of confidence degree, which is marked as P0、P1、……P9。
Therefore, the classification confidence degrees of the cloud images corresponding to the 10 cloud categories are respectively compared with the preset confidence threshold value of each cloud category, and the cloud of which category appears in the cloud images is judged according to the comparison result information. For example: t is0＞P0Then it can be determined that a 0 th category of clouds is present in the image; t is3＜P3Then it may be determined that no cloud of category 3 is present in the image.
It should be noted that the cloud identification method based on deep learning provided by the embodiment can be applied to various types of hardware devices. Examples of the hardware devices are arm (advanced RISC machines) controllers, fpga (field programmable Gate array) controllers, soc (system on chip) controllers, dsp (digital signal processing) controllers, mcu (micro controller unit) controllers, and the like. The hardware devices may also be, for example, a computer that includes components such as memory, a memory controller, one or more processing units (CPUs), a peripheral interface, RF circuitry, audio circuitry, speakers, a microphone, an input/output (I/O) subsystem, a display screen, other output or control devices, and external ports; the computer includes, but is not limited to, Personal computers such as desktop computers, notebook computers, tablet computers, smart phones, smart televisions, Personal Digital Assistants (PDAs), and the like. In other embodiments, the hardware device may also be, for example, a server, where the server may be arranged on one or more physical servers according to various factors such as functions, loads, and the like, and may also be formed by a distributed or centralized server cluster, which is not limited in this embodiment.
Example II
Fig. 2 is a schematic flow chart illustrating a deep learning-based model training method according to an embodiment of the present invention. The model training method based on deep learning of the embodiment is used for training the cloud identification multi-label classification model mentioned in the embodiment, and the model training method mainly comprises the steps of S21-S23.
Step S21: and collecting a plurality of cloud images at different moments and in different categories.
Optionally, in order to ensure the universality and validity of the training data, in this embodiment, a video camera, a camera module integrated with an optical system or a CCD chip, a camera module integrated with an optical system and a CMOS chip, and other devices are used to collect a plurality of images of different cloud types at different times.
Step S22: class labels are labeled for each cloud in each image and the labeled class labels are converted to corresponding label data. Wherein the image data and the label data constitute a training data set for training the classification network model.
Specifically, a class label is marked for each cloud in each image, and the marked class labels are converted into a one-dimensional array as labels of the input classification network model, for example: if the 0 th and 3 rd clouds are marked in the current image, the label data input to the network is [1,0,0,1,0,0,0,0 ].
Step S23: and constructing a classification network model, and training the classification network model by taking the cloud images and the corresponding label data thereof as training data until the classification network model converges.
Taking the ResNet neural network model as an example, the classification network model in this embodiment may include 53 convolutional layers, 50 BatchNorm layers, 50 Scale layers, 49 Re L U active layers, 16 Eltwise layers, 2 pooling layers, 1 full-link layer, and 1 sigmoid active layer.
It should be noted that the ResNet neural network adopted in this embodiment is different from a general classification network, and a Sigmoid activation layer is added behind a last full connection layer, and is used to convert a multidimensional float array output of the full connection layer into a multidimensional 0-1 probability array and then output the multidimensional 0-1 probability array (for example, a 10-dimensional float array is converted into a 10-dimensional 0-1 probability array), where an ith value in the array represents a probability (i ═ 0,1,2,3,4,5,6,7,8,9) that an ith class of cloud exists in a picture predicted by the network.
The eltwise layer is pixel-by-pixel operation, and comprises three types of operations of dot product (product), summation (sum) and maximum value (max), and is set to pixel-by-pixel addition in the invention; the flatten layer is a network layer for unifying multidimensional input; the batchnorm layer is a network layer for carrying out normalization operation on network data; the Scale layer is a network layer that scales and shifts network data.
Optionally, the training of the classification network model by using the cloud images and the corresponding label data thereof as training data until convergence specifically includes: carrying out normalization processing on the cloud-shaped image; inputting the normalized image data and the corresponding label data into a classification network model, and calculating a binary cross entropy loss value; and determining the convergence of the classification network model under the condition that the binary cross entropy loss value is smaller than a preset loss value.
Specifically, the normalization process is to convert the dimensional data into dimensionless data and map the dimensionless data into (0,1) or (-1,1) range, for example, three color channel data of each pixel point in the cloud image are subtracted by the corresponding preset channel data respectively and scaled to the preset pixel size, for example, RGB channel data of each pixel point in the whole image are subtracted by 104, 117, 124 respectively and scaled to the pixel size of 224 × 224.
Optionally, when the classification network model is trained, the normalized image data and the corresponding array label data are input into the network model, so as to calculate a binary cross entropy loss value. The calculation formula of the binary cross entropy loss value is shown as the following formula 1:
wherein C represents the total number of cloud categories, for example, if 10 cloud categories are set in this embodiment, C is 10; siRepresenting the value of the ith class in the probability array returned after the network passes through the Sigmoid activation layer; t isiIndicating the value of the current tag.
And calculating back propagation according to the loss, iteratively updating network parameters, and determining the convergence of the classification model when the loss tends to be stable and is less than a preset loss value (such as 0.01). Counting a threshold value P which enables the ith cloud to obtain the best detection effect in all the label dataiAnd presetting a threshold value as the confidence of the ith cloud.
It should be noted that the deep learning-based model training method provided by this embodiment can be applied to various types of hardware devices. Examples of the hardware devices are arm (advanced RISC machines) controllers, fpga (field programmable Gate array) controllers, soc (system on chip) controllers, dsp (digital signal processing) controllers, mcu (micro controller unit) controllers, and the like. The hardware devices may also be, for example, a computer that includes components such as memory, a memory controller, one or more processing units (CPUs), a peripheral interface, RF circuitry, audio circuitry, speakers, a microphone, an input/output (I/O) subsystem, a display screen, other output or control devices, and external ports; the computer includes, but is not limited to, Personal computers such as desktop computers, notebook computers, tablet computers, smart phones, smart televisions, Personal Digital Assistants (PDAs), and the like. In other embodiments, the hardware device may also be, for example, a server, where the server may be arranged on one or more physical servers according to various factors such as functions, loads, and the like, and may also be formed by a distributed or centralized server cluster, which is not limited in this embodiment.
EXAMPLE III
Fig. 3 is a schematic structural diagram illustrating a cloud recognition apparatus based on deep learning according to an embodiment of the present invention. The cloud recognition device of the present embodiment includes an image acquisition module 31 and an image recognition module 32.
The image acquisition module 31 is configured to acquire a cloud image to be identified and perform image preprocessing on the cloud image; the image recognition module 32 is configured to input the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training, so as to obtain classification confidence levels of the cloud image corresponding to a plurality of cloud classes; if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the image to be recognized; otherwise, determining that the cloud of the cloud type does not appear in the image to be recognized, thereby providing a more intelligent technical solution with better cloud observation effect.
Since the implementation of the cloud identification apparatus in this embodiment is similar to the implementation of the cloud identification method based on deep learning in the first embodiment, further description is omitted here.
It should be understood that the above division of the cloud identification device based on deep learning is only a division of logical functions, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the image recognition module may be a processing element separately set up, or may be implemented by being integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and the function of the image recognition module may be called and executed by a processing element of the apparatus. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Example four
Fig. 4 is a schematic structural diagram of a deep learning-based model training apparatus according to an embodiment of the present invention. The model training device of the present embodiment includes a training data acquisition module 41, a label labeling module 42, and a model training module 43.
The training data acquisition module 41 is used for acquiring a plurality of cloud images at different times and in different categories; the label labeling module 42 is configured to label a category label for each cloud in each image, and convert the labeled category label into corresponding label data; the model training module 43 is configured to construct a classification network model, and train the classification network model with the cloud images and the corresponding label data thereof as training data until convergence.
Since the implementation of the model training apparatus in this embodiment is similar to the implementation of the model training method based on deep learning in the second embodiment, further description is omitted here.
It should be understood that the division of the deep learning based model training apparatus is only a division of logic functions, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the model training module may be a separately installed processing element, or may be implemented by being integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the functions of the model training module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
EXAMPLE five
Fig. 5 is a schematic structural diagram illustrating a cloud recognition terminal based on deep learning according to an embodiment of the present invention. The cloud identification terminal of the embodiment includes: a processor 51, a memory 52, a communicator 53; the memory 52 is connected with the processor 51 and the communicator 53 through a system bus and completes mutual communication, the memory 52 is used for storing computer programs, the communicator 53 is used for communicating with other devices, and the processor 51 is used for running the computer programs, so that the cloud identification terminal executes the steps of the cloud identification method based on deep learning.
It should be understood that, in the above cloud identification terminal based on deep learning, the system bus thereof may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
EXAMPLE six
Fig. 6 is a schematic structural diagram illustrating a deep learning-based model training terminal according to an embodiment of the present invention. The model training terminal of the embodiment includes: a processor 61, a memory 62, a communicator 63; the memory 62 is connected with the processor 61 and the communicator 63 through a system bus and completes mutual communication, the memory 62 is used for storing computer programs, the communicator 63 is used for communicating with other devices, and the processor 61 is used for running the computer programs, so that the cloud recognition terminal executes the steps of the model training method based on deep learning.
It should be understood that, in the above deep learning based model training terminal, the system bus thereof may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
EXAMPLE seven
The present embodiment provides a computer-readable storage medium, on which a first computer program and/or a second computer program are stored, the first computer program, when being executed by a processor, implementing the deep learning based cloud recognition method in the first embodiment, and the second computer program, when being executed by a processor, implementing the deep learning based model training method in the second embodiment.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
In summary, the invention provides a cloud identification and model training method, device, terminal and medium based on deep learning, and the technical scheme of the invention is an intelligent identification method based on deep learning, and uses a multi-label classification deep neural network, which can judge multiple types of clouds at the same time, thereby being a more intelligent technical solution with better cloud observation effect. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (10)
1. A cloud identification method based on deep learning is characterized by comprising the following steps:
acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image;
inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training, and accordingly obtaining classification confidence coefficients of the cloud image corresponding to a plurality of cloud categories;
if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the cloud image to be recognized; otherwise, determining that the cloud of the cloud category does not appear in the cloud image to be identified.
2. The cloud identification method of claim 1, wherein said image pre-processing comprises an image normalization process comprising:
and respectively subtracting the corresponding preset channel data from the three color channel data of each pixel point in the cloud image, and zooming to the preset pixel size.
3. The cloud identification method according to claim 1, comprising:
the plurality of cloud categories of the cloud include: high cloud, rolling cloud, raincloud, cumulus cloud, rain cloud, layer cloud, or all of the categories; and
the classification network model comprises a Resnet classification network model which comprises a plurality of convolution layers, a plurality of BatchNorm layers, a plurality of Scale layers, a plurality of Re L U active layers, a plurality of Eltwiese layers, a plurality of pooling layers, at least one full connection layer and at least one Sigmoid active layer.
4. A deep learning-based model training method, which is used for training the cloud recognition multi-label classification model of claim 1; the method comprises the following steps:
collecting a plurality of cloud images at different times and different categories;
marking category labels for each cloud in each image, and converting the marked category labels into corresponding label data;
and constructing a classification network model, and training the classification network model by taking the cloud images and the corresponding label data thereof as training data until the classification network model converges.
5. The model training method according to claim 4, wherein the training of the classification network model with cloud images and their corresponding label data as training data until convergence comprises:
carrying out normalization processing on the cloud-shaped image;
inputting the normalized image data and the corresponding label data into a classification network model, and calculating a binary cross entropy loss value; and determining the convergence of the classification network model under the condition that the binary cross entropy loss value is smaller than a preset loss value.
6. A cloud recognition device based on deep learning, comprising:
the image acquisition module is used for acquiring a cloud image to be identified and carrying out image preprocessing on the cloud image;
the image recognition module is used for inputting the preprocessed cloud image into a cloud recognition multi-label classification model which is subjected to deep learning training so as to obtain classification confidence coefficients of the cloud image corresponding to a plurality of cloud categories; if the confidence coefficient of the cloud image corresponding to a cloud type is greater than the preset threshold of the confidence coefficient of the cloud type, determining that the cloud of the cloud type appears in the image to be recognized; otherwise, determining that the cloud of the cloud category does not appear in the image to be recognized.
7. A deep learning-based model training device, comprising:
the training data acquisition module is used for acquiring a plurality of cloud-shaped image data at different moments and different categories;
the label labeling module is used for labeling the category labels for the clouds in each image and converting the labeled category labels into corresponding label data;
and the model training module is used for constructing a classification network model and training the classification network model by taking the cloud images and the corresponding label data thereof as training data until the classification network model is converged.
8. A computer-readable storage medium, on which a first computer program and/or a second computer program is stored, which when executed by a processor implements the deep learning based cloud recognition method of any one of claims 1 to 3; the second computer program, when executed by a processor, implements the deep learning based model training method of claim 4 or 5.
9. A cloud identification terminal based on deep learning is characterized by comprising: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the memory-stored computer program to cause the terminal to perform the deep learning based cloud identification method according to any one of claims 1 to 3.
10. A model training terminal based on deep learning is characterized by comprising: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the memory-stored computer program to cause the terminal to perform the deep learning based model training method according to claim 4 or 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010300620.2A CN111507399A (en) | 2020-04-16 | 2020-04-16 | Cloud recognition and model training method, device, terminal and medium based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010300620.2A CN111507399A (en) | 2020-04-16 | 2020-04-16 | Cloud recognition and model training method, device, terminal and medium based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111507399A true CN111507399A (en) | 2020-08-07 |
Family
ID=71876170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010300620.2A Pending CN111507399A (en) | 2020-04-16 | 2020-04-16 | Cloud recognition and model training method, device, terminal and medium based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111507399A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036488A (en) * | 2020-08-31 | 2020-12-04 | 中国平安财产保险股份有限公司 | Event identification method, device and equipment based on image identification and storage medium |
| CN112330069A (en) * | 2020-11-27 | 2021-02-05 | 上海眼控科技股份有限公司 | Early warning removing method and device, electronic equipment and storage medium |
| CN112434554A (en) * | 2020-10-16 | 2021-03-02 | 中科院成都信息技术股份有限公司 | Heterogeneous reduction-based cloud image identification method and system |
| CN112488340A (en) * | 2020-12-18 | 2021-03-12 | 无锡挚达物联科技有限公司 | Intelligent energy storage control method and device based on deep learning and storage medium |
| CN112541550A (en) * | 2020-12-16 | 2021-03-23 | 南京掌控网络科技有限公司 | Refrigerator integrity judgment method based on image classification and electronic equipment |
| CN112861948A (en) * | 2021-01-29 | 2021-05-28 | 南京中兴力维软件有限公司 | Method, device and equipment for identifying working state of siphon type sludge discharge equipment |
| CN112861934A (en) * | 2021-01-25 | 2021-05-28 | 深圳市优必选科技股份有限公司 | Image classification method and device of embedded terminal and embedded terminal |
| CN114022960A (en) * | 2022-01-05 | 2022-02-08 | 阿里巴巴达摩院(杭州)科技有限公司 | Model training and behavior recognition method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165674A (en) * | 2018-07-19 | 2019-01-08 | 南京富士通南大软件技术有限公司 | A kind of certificate photo classification method based on multi-tag depth convolutional network |
| CN110414564A (en) * | 2019-06-27 | 2019-11-05 | 浙江大学 | A kind of optic nerve disease sorter based on optic nerve eye fundus image |
| CN110929602A (en) * | 2019-11-09 | 2020-03-27 | 北京工业大学 | Foundation cloud picture cloud shape identification method based on convolutional neural network |
-
2020
- 2020-04-16 CN CN202010300620.2A patent/CN111507399A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165674A (en) * | 2018-07-19 | 2019-01-08 | 南京富士通南大软件技术有限公司 | A kind of certificate photo classification method based on multi-tag depth convolutional network |
| CN110414564A (en) * | 2019-06-27 | 2019-11-05 | 浙江大学 | A kind of optic nerve disease sorter based on optic nerve eye fundus image |
| CN110929602A (en) * | 2019-11-09 | 2020-03-27 | 北京工业大学 | Foundation cloud picture cloud shape identification method based on convolutional neural network |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036488A (en) * | 2020-08-31 | 2020-12-04 | 中国平安财产保险股份有限公司 | Event identification method, device and equipment based on image identification and storage medium |
| CN112434554A (en) * | 2020-10-16 | 2021-03-02 | 中科院成都信息技术股份有限公司 | Heterogeneous reduction-based cloud image identification method and system |
| CN112434554B (en) * | 2020-10-16 | 2023-08-04 | 中科院成都信息技术股份有限公司 | Cloud image recognition method and system based on heterogeneous reduction |
| CN112330069A (en) * | 2020-11-27 | 2021-02-05 | 上海眼控科技股份有限公司 | Early warning removing method and device, electronic equipment and storage medium |
| CN112541550A (en) * | 2020-12-16 | 2021-03-23 | 南京掌控网络科技有限公司 | Refrigerator integrity judgment method based on image classification and electronic equipment |
| CN112488340A (en) * | 2020-12-18 | 2021-03-12 | 无锡挚达物联科技有限公司 | Intelligent energy storage control method and device based on deep learning and storage medium |
| CN112488340B (en) * | 2020-12-18 | 2021-07-27 | 无锡挚达物联科技有限公司 | Intelligent energy storage control method and device based on deep learning and storage medium |
| CN112861934A (en) * | 2021-01-25 | 2021-05-28 | 深圳市优必选科技股份有限公司 | Image classification method and device of embedded terminal and embedded terminal |
| CN112861948A (en) * | 2021-01-29 | 2021-05-28 | 南京中兴力维软件有限公司 | Method, device and equipment for identifying working state of siphon type sludge discharge equipment |
| CN112861948B (en) * | 2021-01-29 | 2024-04-09 | 南京中兴力维软件有限公司 | Method, device and equipment for identifying working state of siphon type sludge discharge equipment |
| CN114022960A (en) * | 2022-01-05 | 2022-02-08 | 阿里巴巴达摩院(杭州)科技有限公司 | Model training and behavior recognition method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111507399A (en) | Cloud recognition and model training method, device, terminal and medium based on deep learning | |
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| US20220092351A1 (en) | Image classification method, neural network training method, and apparatus | |
| Zhang et al. | Surface defect detection of steel strips based on classification priority YOLOv3-dense network | |
| US9830704B1 (en) | Predicting performance metrics for algorithms | |
| CN111259957A (en) | Visibility monitoring and model training method, device, terminal and medium based on deep learning | |
| CN112836713A (en) | Image anchor-frame-free detection-based mesoscale convection system identification and tracking method | |
| CN111368636B (en) | Object classification method, device, computer equipment and storage medium | |
| CN110852233A (en) | Hand-off steering wheel detection and training method, terminal, device, medium, and system | |
| US20230326173A1 (en) | Image processing method and apparatus, and computer-readable storage medium | |
| CN111046910A (en) | Image classification, relation network model training and image annotation method and device | |
| CN113807399A (en) | Neural network training method, neural network detection method and neural network detection device | |
| CN115331172A (en) | Workshop dangerous behavior recognition alarm method and system based on monitoring video | |
| CN111429424A (en) | Heating furnace inlet abnormity identification method based on deep learning | |
| CN114676776A (en) | Fine-grained image classification method based on Transformer | |
| CN113780423A (en) | Single-stage target detection neural network based on multi-scale fusion and industrial product surface defect detection model | |
| CN115393690A (en) | Light neural network air-to-ground observation multi-target identification method | |
| Venkatesvara Rao et al. | Real-time video object detection and classification using hybrid texture feature extraction | |
| Zheng et al. | Stripe segmentation of oceanic internal waves in SAR images based on SegNet | |
| CN106960188B (en) | Weather image classification method and device | |
| Osuna-Coutiño et al. | Structure extraction in urbanized aerial images from a single view using a CNN-based approach | |
| Huang et al. | Deep residual network training for reinforced concrete defects intelligent classifier | |
| CN111967287A (en) | Pedestrian detection method based on deep learning | |
| WO2022127333A1 (en) | Training method and apparatus for image segmentation model, image segmentation method and apparatus, and device | |
| CN115063831A (en) | High-performance pedestrian retrieval and re-identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200807 |
|
| WD01 | Invention patent application deemed withdrawn after publication |