CN110490831B - Image generation method and system - Google Patents

Image generation method and system Download PDF

Info

Publication number
CN110490831B
CN110490831B CN201910783477.4A CN201910783477A CN110490831B CN 110490831 B CN110490831 B CN 110490831B CN 201910783477 A CN201910783477 A CN 201910783477A CN 110490831 B CN110490831 B CN 110490831B
Authority
CN
China
Prior art keywords
infrared image
current
neural network
convolutional neural
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910783477.4A
Other languages
Chinese (zh)
Other versions
CN110490831A (en
Inventor
张骏
袁烨
李培华
孙瑞
吉涛
周萌
章盛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AVIC Huadong Photoelectric Co Ltd
Original Assignee
AVIC Huadong Photoelectric Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AVIC Huadong Photoelectric Co Ltd filed Critical AVIC Huadong Photoelectric Co Ltd
Priority to CN201910783477.4A priority Critical patent/CN110490831B/en
Publication of CN110490831A publication Critical patent/CN110490831A/en
Application granted granted Critical
Publication of CN110490831B publication Critical patent/CN110490831B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image generation method and system, relating to the technical field of computers, wherein the image generation method comprises the following steps: acquiring a current infrared image; acquiring a current entity outline corresponding to the current infrared image based on the trained convolutional neural network model; and generating a fused image based on the current entity outline and the current infrared image. The image generation method and the image generation system enable the entity to be visualized even if the entity is occluded.

Description

Image generation method and system
Technical Field
The invention relates to the technical field of computers, in particular to an image generation method.
Background
In the existing fire rescue, the position of an entity can be monitored by adopting an infrared image, although the infrared image can be convenient for a rescuer to observe in a fire scene, the infrared image has the characteristics of poor resolution, low contrast, low signal-to-noise ratio and the like, and the characteristics easily cause the blurring of the visual effect of the rescuer. In addition, entities often hide from rescue, however, when the entities are partially obstructed by obstacles, the outline of the entities in the acquired infrared images is often incomplete, which can easily make the entities undiscoverable to rescue personnel. Therefore, the existing infrared image cannot meet the use requirement of rescuers.
Disclosure of Invention
The invention aims to provide an image generation method and an image generation system, which can realize visualization even if an entity is blocked.
In order to achieve the above object, the present invention provides an image generation method including: acquiring a current infrared image; acquiring a current entity outline corresponding to the current infrared image based on the trained convolutional neural network model; and generating a fused image based on the current entity outline and the current infrared image.
Preferably, the obtaining of the current entity profile corresponding to the current infrared image based on the trained convolutional neural network model includes: establishing a convolutional neural network model, wherein the convolutional neural network model takes an infrared image as input and takes an entity contour as output; counting parameters related to each infrared image and parameters related to entity outlines in each infrared image, and constructing a training knowledge base based on the counted parameters; training the established convolutional neural network model by using the parameters in the training knowledge base; and inputting the current infrared image into the trained convolutional neural network model to obtain a current entity profile output by the convolutional neural network model.
Preferably, before said counting the parameters associated with each infrared image, the method further comprises: performing data enhancement on each infrared image of the parameters to be statistically correlated by at least one of: and performing geometric transformation on each infrared image and performing cutting transformation on each infrared image.
Preferably, the establishing a convolutional neural network model comprises: performing the following operations on the initial convolutional neural network: replacing the first layer of Conv convolutional layer with a first Dense-block layer, replacing the second layer of Conv convolutional layer with a second Dense-block layer, replacing the first layer of maximum pooling layer with a first transition layer, and replacing the second layer of maximum pooling layer with a second transition layer; and establishing a convolutional neural network model based on the operated initial convolutional neural network, wherein the convolutional neural network model is classified based on a SoftMax function, and a cross entropy function is used as a loss function.
Preferably, the first transition layer comprises, in sequence: the first roll-up sublayer, the first Dropout sublayer and the first pooling sublayer; and/or the second transition layer comprises the following components arranged in sequence: a second convolution sublayer, a second Dropout sublayer, and a second pooling sublayer.
Preferably, the generating a fused image based on the current entity profile and the current infrared image comprises: generating a fused image of the current entity outline data and the current infrared image data through an Opencv module; the weight of the current entity profile data input in the Opencv module is a first set value, the weight of the current infrared image data is a second set value, the G channel inputs the current entity profile data, and the R channel and the B channel input the current infrared image data.
In another aspect, the present invention also provides an image generation system, including: the image acquisition module is used for acquiring a current infrared image; the contour acquisition module is used for acquiring a current entity contour corresponding to the current infrared image based on the trained convolutional neural network model; and the image generation module is used for generating a fused image based on the current entity outline and the current infrared image.
In another aspect, the present invention also provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps according to the above-described image generation method when executing the program.
In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps according to the above-described image generation method.
In another aspect, the present invention further provides a processor for executing a program, where the program is executed to perform: the method of claim.
According to the technical scheme, the entity outline corresponding to the current infrared image can be obtained through the established trained convolutional neural network model, wherein the current infrared image does not have the entity identification, the outline can be obtained when the entity is partially shielded by utilizing the established convolutional neural network model, and the outline and the infrared image are fused to obtain the infrared image containing the outline, so that rescuers can obtain the entity outline image even if the entity is partially shielded during rescue, and the requirements of the rescuers are met.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart illustrating an image generation method of the present invention;
FIG. 2a is a diagram illustrating the effect of an infrared image of the present invention;
FIG. 2b is an effect diagram of the solid outline of FIG. 2 a;
FIG. 2c is an effect diagram of the fused image of FIGS. 2a and 2 b;
FIG. 2d is an effect diagram of the fused image in the occlusion case;
FIG. 3a is a topology diagram of an initial convolutional neural network;
FIG. 3b is a topological diagram of a convolutional neural network of an embodiment of the present invention;
FIG. 4a is a block diagram of modules of the first transition layer;
FIG. 4b is a block diagram of the second transition layer; and
FIG. 5 is a system block diagram of an image generation system of the present invention.
Description of the reference numerals
1、Dense-block；2、Transition Layer；3、Max Pooling；
4、Conv；5、UpSampling；6、SoftMax；7、Merge；
11. An image acquisition module; 12. a contour acquisition module; 13. an image generation module;
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
Before setting forth the invention in detail, the characteristics of infrared images are briefly introduced, which are often used to search for entities in a fire scene in the field of fire fighting. However, the infrared image has the problems of poor resolution, low contrast and low signal-to-noise ratio, and the entity reflected by the infrared image is often blocked, which easily causes the problem that the search and rescue personnel neglect the entity in the infrared image. The invention aims at the problems and further processes the infrared image.
Example 1
Fig. 1 is a flowchart of an image generation method of embodiment 1.
As shown in fig. 1, the image generation method includes:
and S101, acquiring a current infrared image.
The current infrared image is an infrared image acquired by search and rescue personnel when the infrared image is used, and the infrared image can be acquired through camera equipment in a fire scene. The particular displayed infrared image may be as shown in fig. 2a, where in reality the dominant hue of fig. 2a is red.
And S102, acquiring a current entity outline corresponding to the current infrared image based on the trained convolutional neural network model.
In this embodiment, the entity contour is a contour of a person to be searched and rescued, and in an actual display process, the contour may be a displayed wire frame, as shown in fig. 2 b.
In this embodiment, the following manner may be adopted to obtain the current entity profile corresponding to the current infrared image.
A) And establishing a convolutional neural network model, wherein the convolutional neural network model takes the infrared image as input and takes the entity outline as output. In which, the initial convolutional neural network in the prior art is shown in figure 3a,
Figure BDA0002177275600000051
is conv3x3, reLU; />
Figure BDA0002177275600000052
Copy and crop; />
Figure BDA0002177275600000053
Max pool2x2; />
Figure BDA0002177275600000054
Is up-conv 2x2. In order to achieve the purpose of the present invention, the present application adopts an improved neural network, specifically, the neural network adopted is shown in fig. 3b, which is based on the numbers inside the arrow, including sense-block 1; transition Layer2; max Pooling3; conv4; upSamplling 5; softMax6; merge7. Compared with the initial convolutional neural network, the first layer of Conv convolutional layer is replaced by a first Dense-block layer, the second layer of Conv convolutional layer is replaced by a second Dense-block layer, the first layer of maximum pooling layer is replaced by a first transition layer, and the second layer of maximum pooling layer is replaced by a second transition layer. First, the first layer Conv convolutional layer is replaced by a first Dense-block layer, and the second layer Conv convolutional layer is replaced by a second Dense-block layer, so as to extract shallow layer feature information, wherein the shallow layer feature information may be information such as boundaries and colors, and for infrared images, because of more convolution operations in deep layers, the extracted image features are difficult to describe clearly. The Dense-block has the advantages of relieving gradient disappearance, saving parameters and calculation, and playing a role in resisting overfitting by reusing features. And these advantagesIt is exactly what is needed for the idea of "adding shallow feature extraction". In the convolutional neural network adopted by the embodiment, the first two layers of the U-Net network are used as shallow layer feature extraction layers, and the Dense-block is adopted for convolution so as to ensure the maximization of shallow layer information utilization. Secondly, as the Dense-block replaces the Conv operation, a large amount of redundancy occurs to the data, so that the parameter number and the dimensionality crisis are increased, it is necessary to add a transition layer to achieve dimensionality reduction and prevent overfitting, based on which, the first layer of the maximum pooling layer is replaced by a first transition layer and the second layer of the maximum pooling layer is replaced by a second transition layer, wherein, as shown in fig. 4a, the first transition layer comprises the following components which are arranged in sequence: the first roll-up sublayer, the first Dropout sublayer and the first pooling sublayer; and/or as shown in fig. 4b, the second transition layer may include, in order: a second convolution sublayer, a second Dropout sublayer, and a second pooling sublayer. The first convolution sub-layer or the second convolution sub-layer is 1 × 1Conv, the number of channels is reduced to achieve the purpose of dimension reduction, the first Dropout sub-layer or the second Dropout sub-layer is used for reducing the probability of overfitting, and the first pooling sub-layer or the second pooling sub-layer (MaxPoling) is used for achieving the down-sampling function in the U-Net network.
And establishing a convolutional neural network model based on the operated initial convolutional neural network, wherein the convolutional neural network model is classified based on a SoftMax function, and a cross entropy function is used as a loss function.
Specifically, the SoftMax function includes:
Figure BDA0002177275600000061
in the above formula, C represents the number of classes, i represents the number of pels in each sample map, i belongs to { 1.
Figure BDA0002177275600000062
Represents the output feature vector of the c-th class, the feature vector is ≥ via equation 1>
Figure BDA0002177275600000063
The linear predicted values of all the categories in the image are converted into probability values, namely the prediction probability that the ith pixel belongs to the c-th category.
The Cross Entropy function (Cross Entropy) comprises the following formula and is taken as a loss function:
Figure BDA0002177275600000064
wherein, in the above formula
Figure BDA0002177275600000065
Value of a marker, p, expressed as a true class c (x i ) Are the results of formula 1.
B) And counting parameters related to the infrared images and parameters related to the entity outline in each infrared image, and constructing a training knowledge base based on the counted parameters. The parameters related to the infrared images can be parameters such as resolution, pixel color values and adjacent color value differences of the infrared images, the parameters related to the entity outline can be coordinates and identification types of the entity outline, and identification modes can be guaranteed based on the two data.
Before step B), it is further preferred that each infrared image of the statistically relevant parameters can be subjected to data enhancement by at least one of: and performing geometric transformation on each infrared image and performing cutting transformation on each infrared image. For example, a randomly inverted geometric transformation is used.
C) And training the established convolutional neural network model by using the parameters in the training knowledge base.
D) And inputting the current infrared image into the trained convolutional neural network model to obtain a current entity profile output by the convolutional neural network model.
And S103, generating a fused image based on the current complete entity outline and the current infrared image.
In the embodiment, the following method is adopted:
and generating a fused image of the current entity outline data and the current infrared image data through an Opencv module.
The weight of the current entity profile data input into the Opencv module is a first set value, the weight of the current infrared image data is a second set value, the G channel inputs the current entity profile data, and the R channel and the B channel input the current infrared image data.
The displayed style, as shown in fig. 2c, that is, the entity outline is generated outside the current infrared image, and the color of the entity outline can be set to green, so that the person to be searched and rescued becomes more obvious, and the person to be searched and rescued can be more easily observed by the search and rescue person, so as to better perform search and rescue. In the effect graph, fig. 2d is a fused picture when the leg of the person to be searched and rescued is shielded, wherein the leg of the outline frame in fig. 2d is shielded, and no identification indication is performed in the picture.
Through the above embodiment, a contour frame identifier can be generated on the external contour of the current infrared image, so that search and rescue personnel can find the personnel to be searched and rescued more easily in a fire scene, the probability of finding the personnel to be searched and rescued is improved, and the problem that the body of the personnel to be searched and rescued is easily overlooked when the body is partially shielded is avoided.
Example 2
Fig. 5 is a system block diagram of an image generation system of embodiment 2.
As shown in fig. 5, the image generation system may include: and the image acquisition module 11 is used for acquiring a current infrared image. And the contour acquisition module 12 is configured to acquire a current entity contour corresponding to the current infrared image based on the trained convolutional neural network model. And the image generation module 13 is configured to generate a fused image based on the current entity outline and the current infrared image.
Preferably, the contour acquisition module 12 comprises: and the model establishing submodule is used for establishing a convolutional neural network model, wherein the convolutional neural network model takes the infrared image as input and takes the entity outline as output. And the knowledge base training submodule is used for counting parameters related to the infrared images and parameters related to the entity outline in each infrared image, and constructing a training knowledge base based on the counted parameters. The training sub-module is used for training the established convolutional neural network model by using the parameters in the training knowledge base; and the contour acquisition submodule is used for inputting the current infrared image into the trained convolutional neural network model and acquiring the current entity contour output by the convolutional neural network model.
Preferably, the image generation system may further include: a data enhancement module, configured to, before counting the parameters related to the infrared images, perform data enhancement on the infrared images of the parameters related to the statistics through at least one of: and performing geometric transformation on each infrared image and performing cutting transformation on each infrared image.
Preferably, the model building module may include: an alternative submodule, configured to perform the following operations on the initial convolutional neural network: replacing the first layer of Conv convolutional layer with a first Dense-block layer, replacing the second layer of Conv convolutional layer with a second Dense-block layer, replacing the first layer of maximum pooling layer with a first transition layer, and replacing the second layer of maximum pooling layer with a second transition layer; and the model establishing submodule is used for establishing a convolutional neural network model based on the operated initial convolutional neural network, wherein the convolutional neural network model is classified based on a SoftMax function, and a cross entropy function is used as a loss function.
Preferably, the first transition layer may include, in order: the first roll-up sublayer, the first Dropout sublayer and the first pooling sublayer; and/or the second transition layer may comprise, arranged in sequence: a second convolution sublayer, a second Dropout sublayer, and a second pooling sublayer.
Preferably, the image generation module 13 may include: the Opencv module is used for generating a fused image of the current entity outline data and the current infrared image data; the weight of the current entity profile data input in the Opencv module is a first set value, the weight of the current infrared image data is a second set value, the G channel inputs the current entity profile data, and the R channel and the B channel input the current infrared image data.
The image generation system in embodiment 2 can achieve the same technical effects as the image generation method in embodiment 1 with respect to the related art.
The image generation system comprises a processor and a memory, wherein the image acquisition module 11, the contour acquisition module 12, the image generation module 13 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more, and the image generation is realized by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
An embodiment of the present invention provides a storage medium having a program stored thereon, the program implementing the image generation method when executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the image generation method is executed when the program runs.
An embodiment of the present invention provides an apparatus, where the apparatus includes a processor, a memory, and a program stored in the memory and capable of running on the processor, and the processor implements the method in embodiment 1 when executing the program. The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: the method of example 1.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (8)

1. An image generation method, characterized by comprising:
acquiring a current infrared image;
acquiring a current entity outline corresponding to the current infrared image based on the trained convolutional neural network model; and
generating a fused image based on the current entity outline and the current infrared image; wherein the content of the first and second substances,
the acquiring of the current entity profile corresponding to the current infrared image based on the trained convolutional neural network model comprises:
establishing a convolutional neural network model, wherein the convolutional neural network model takes an infrared image as input and takes an entity contour as output;
counting parameters related to each infrared image and parameters related to entity outlines in each infrared image, and constructing a training knowledge base based on the counted parameters;
training the established convolutional neural network model by using the parameters in the training knowledge base; and
inputting the current infrared image into the trained convolutional neural network model to obtain a current entity profile output by the convolutional neural network model;
the generating a fused image based on the current entity profile and the current infrared image comprises:
generating a fused image of the current entity outline data and the current infrared image data through an Opencv module;
the weight of the current entity outline data input in the Opencv module is a first set value, the weight of the current infrared image data is a second set value, the current entity outline data is input through a G channel, and the current infrared image data is input through an R channel and a B channel.
2. The image generation method of claim 1, wherein prior to said counting parameters associated with each infrared image, the method further comprises:
performing data enhancement on each infrared image of the parameter to be statistically correlated by at least one of:
and performing geometric transformation on each infrared image and performing cutting transformation on each infrared image.
3. The image generation method of claim 1, wherein the building a convolutional neural network model comprises:
performing the following operations on the initial convolutional neural network: replacing the first layer of Conv convolutional layer with a first Dense-block layer, replacing the second layer of Conv convolutional layer with a second Dense-block layer, replacing the first layer of maximum pooling layer with a first transition layer, and replacing the second layer of maximum pooling layer with a second transition layer; and
and establishing a convolutional neural network model based on the operated initial convolutional neural network, wherein the convolutional neural network model is classified based on a SoftMax function, and a cross entropy function is used as a loss function.
4. The image generation method according to claim 3,
the first transition layer comprises the following components arranged in sequence: the first roll-up sublayer, the first Dropout sublayer and the first pooling sublayer; and/or
The second transition layer comprises the following components in sequence: a second convolution sublayer, a second Dropout sublayer, and a second pooling sublayer.
5. An image generation system, comprising:
the image acquisition module is used for acquiring a current infrared image;
the contour acquisition module is used for acquiring a current entity contour corresponding to the current infrared image based on the trained convolutional neural network model; and
the image generation module is used for generating a fused image based on the current entity outline and the current infrared image;
wherein the content of the first and second substances,
the acquiring of the current entity profile corresponding to the current infrared image based on the trained convolutional neural network model comprises:
establishing a convolutional neural network model, wherein the convolutional neural network model takes an infrared image as input and takes an entity contour as output;
counting parameters related to each infrared image and parameters related to entity outlines in each infrared image, and constructing a training knowledge base based on the counted parameters;
training the established convolutional neural network model by using the parameters in the training knowledge base; and
inputting the current infrared image into the trained convolutional neural network model to obtain a current entity profile output by the convolutional neural network model;
the generating a fused image based on the current entity profile and the current infrared image comprises:
generating a fused image of the current entity outline data and the current infrared image data through an Opencv module;
the weight value of the current entity outline data input in the Opencv module is a first set value, the weight value of the current infrared image data is a second set value, the current entity outline data is input in a G channel, and the current infrared image data is input in an R channel and a B channel.
6. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the steps of the image generation method according to any of claims 1 to 4 are implemented when the program is executed by the processor.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of an image generation method according to any one of claims 1 to 4.
8. A processor configured to execute a program, wherein the program when executed is configured to perform: the steps of the image generation method according to any of claims 1 to 4.
CN201910783477.4A 2019-08-23 2019-08-23 Image generation method and system Active CN110490831B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910783477.4A CN110490831B (en) 2019-08-23 2019-08-23 Image generation method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910783477.4A CN110490831B (en) 2019-08-23 2019-08-23 Image generation method and system

Publications (2)

Publication Number Publication Date
CN110490831A CN110490831A (en) 2019-11-22
CN110490831B true CN110490831B (en) 2023-03-24

Family

ID=68553211

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910783477.4A Active CN110490831B (en) 2019-08-23 2019-08-23 Image generation method and system

Country Status (1)

Country Link
CN (1) CN110490831B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961180A (en) * 2018-06-22 2018-12-07 理光软件研究所（北京）有限公司 infrared image enhancing method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10303977B2 (en) * 2016-06-28 2019-05-28 Conduent Business Services, Llc System and method for expanding and training convolutional neural networks for large size input images
EP3631751A4 (en) * 2017-05-24 2021-02-24 HRL Laboratories, LLC Transfer learning of convolutional neural networks from visible color (rbg)to infrared (ir) domain

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961180A (en) * 2018-06-22 2018-12-07 理光软件研究所（北京）有限公司 infrared image enhancing method and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于卷积神经网络的多光谱图像多标签场景分类;李一松;《电子设计工程》;20181205(第23期);全文 *
基于密集卷积神经网络的遥感飞机识别;于丽等;《计算机工程与应用》;20181001(第19期);全文 *

Also Published As

Publication number Publication date
CN110490831A (en) 2019-11-22

Similar Documents

Publication Publication Date Title
CN109325954B (en) Image segmentation method and device and electronic equipment
Xu et al. Deep image matting
Crabb et al. Real-time foreground segmentation via range and color imaging
JP6798183B2 (en) Image analyzer, image analysis method and program
CN109993824B (en) Image processing method, intelligent terminal and device with storage function
CN110176024B (en) Method, device, equipment and storage medium for detecting target in video
KR102166841B1 (en) System and method for improving image quality
CN115131229A (en) Image noise reduction and filtering data processing method and device and computer equipment
WO2024041108A1 (en) Image correction model training method and apparatus, image correction method and apparatus, and computer device
CN110490831B (en) Image generation method and system
CN111160240B (en) Image object recognition processing method and device, intelligent device and storage medium
CN116309494B (en) Method, device, equipment and medium for determining interest point information in electronic map
JP7163703B2 (en) Detection device, computer program and detection method
Zhao et al. Saliency map-aided generative adversarial network for raw to rgb mapping
CN113313635A (en) Image processing method, model training method, device and equipment
CN110889352A (en) Image blurring processing method, computer device, and computer-readable storage medium
CN116883770A (en) Training method and device of depth estimation model, electronic equipment and storage medium
CN112561815B (en) Image processing method and device, storage medium and electronic device
CN114360015A (en) Living body detection method, living body detection device, living body detection equipment and storage medium
CN112085025B (en) Object segmentation method, device and equipment
CN114299105A (en) Image processing method, image processing device, computer equipment and storage medium
CN109712094B (en) Image processing method and device
KR20180064064A (en) Method for Detecting Edges on Color Image Based on Fuzzy Theory
CN113628122A (en) Image processing method, model training method, device and equipment
CN113837948B (en) Image generation method, apparatus and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant