CN111340188A - Image processing method and device, electronic equipment and readable storage medium - Google Patents
Image processing method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN111340188A CN111340188A CN202010106684.9A CN202010106684A CN111340188A CN 111340188 A CN111340188 A CN 111340188A CN 202010106684 A CN202010106684 A CN 202010106684A CN 111340188 A CN111340188 A CN 111340188A
- Authority
- CN
- China
- Prior art keywords
- image
- shading
- neural network
- network model
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000003062 neural network model Methods 0.000 claims abstract description 112
- 238000012549 training Methods 0.000 claims abstract description 44
- 238000001914 filtration Methods 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims description 76
- 238000013527 convolutional neural network Methods 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 23
- 230000001537 neural effect Effects 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 10
- 230000003042 antagnostic effect Effects 0.000 claims description 7
- 230000009467 reduction Effects 0.000 claims description 7
- 238000007906 compression Methods 0.000 claims description 6
- 230000000306 recurrent effect Effects 0.000 claims description 3
- 238000011946 reduction process Methods 0.000 claims description 3
- 241001522296 Erithacus rubecula Species 0.000 claims 1
- 238000000034 method Methods 0.000 abstract description 28
- 239000000284 extract Substances 0.000 abstract description 4
- 238000011176 pooling Methods 0.000 description 29
- 238000010586 diagram Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 10
- 230000004913 activation Effects 0.000 description 6
- 125000004122 cyclic group Chemical group 0.000 description 6
- 210000005036 nerve Anatomy 0.000 description 5
- 210000002569 neuron Anatomy 0.000 description 5
- 230000006835 compression Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008485 antagonism Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image processing method and device, electronic equipment and a readable storage medium, wherein the image processing method comprises the following steps: acquiring a collected image; inputting the collected image into a pre-trained neural network model to obtain a shading image in the collected image; obtaining an image after filtering the shading based on the collected image and the shading image, wherein the neural network model is trained in the following way: inputting each background image sample in the background image sample set into a neural network model, outputting the judged background image by the neural network model, and comparing the judged background image with the known background image of the background image sample, thereby training the characteristic coefficients among all layers of hidden layer nodes of the neural network model and obtaining the trained neural network model. The method and the device can better extract the shading information in the collected image under the current environment, further obtain effective information with better quality, and enhance the accuracy of the image recognition result.
Description
Technical Field
The invention relates to the technical field of information processing, in particular to an image processing method and device, electronic equipment and a readable storage medium.
Background
Biometric identification techniques such as fingerprint identification and face identification are currently widely used in both civilian and military applications. For example, for an access control system, many enterprises and organizations have adopted fingerprint identification technology to identify users, so as to perform security access, statistics and attendance checking, and the like.
In fingerprint identification, generally, an image acquisition device is used to acquire an image sample with a shading, and then effective information and shading information are separated from the image sample with the shading, as shown in fig. 1, fig. 1 shows a structural block diagram of an existing image processing device, the existing image processing device includes a difference module 100, when image processing such as shading removal is performed, a shading image is pre-stored first, then difference operation is performed on the acquired image and the pre-stored shading image, an image with the shading filtered, that is, an effective image is obtained, and then effective information in the effective image is obtained.
However, since the image capturing device may be affected by various environmental factors, the shading information included in the captured image may change to some extent, and the matching degree with the information included in the pre-stored shading image is reduced, so that ideal effective information cannot be obtained.
Therefore, there is a need to provide an improved technical solution to overcome the above technical problems in the prior art.
Disclosure of Invention
In order to solve the technical problem, the invention provides an image processing method and device, an electronic device and a readable storage medium, which can better extract shading information in an acquired image in the current environment, further obtain effective information with better quality and enhance the accuracy of an image identification result.
The image processing method provided by the invention comprises the following steps: acquiring a collected image; inputting the collected image into a pre-trained neural network model to obtain a shading image in the collected image; obtaining an image after filtering the shading based on the collected image and the shading image, wherein the neural network model is trained in the following way: inputting each background image sample in the background image sample set into a neural network model, outputting the judged background image by the neural network model, and comparing the judged background image with the known background image of the background image sample, thereby training the characteristic coefficients among all layers of hidden layer nodes of the neural network model and obtaining the trained neural network model.
Preferably, the pre-trained neural network model is a self-encoder neural network model.
Preferably, the pre-trained self-encoder neural network model comprises: pre-training a self-encoder neural network model with an artificial neural network structure for images acquired at fixed positions; for images acquired at different positions, a self-encoder neural network model with a convolutional neural network structure is trained in advance.
Preferably, the self-encoder neural network model comprises a symmetric encoder neural network and a decoder neural network, wherein the encoder neural network is used for receiving the image sample with the shading, compressing and extracting the characteristic information of the image sample with the shading layer by layer; the decoder neural network is used for receiving the characteristic information extracted by the encoder neural network and restoring the image sample with the shading layer by layer according to the characteristic information.
Preferably, the pre-training of the self-encoder neural network model with the artificial neural network structure comprises: inputting pixel data of a multi-dimensional image sample with the shading, and performing dimension reduction output on the pixel data; compressing the pixel data output in a dimensionality reduction mode layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the compression process; reducing the compressed pixel data layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the reduction process; and performing upscaling output on the restored pixel data, and changing the output shape of the pixel data to be consistent with the shape of the input multi-dimensional pixel data, wherein the characteristic coefficient of the artificial neural network is a weighting coefficient of each neural core in the artificial neural network aiming at the gray level of each image pixel.
Preferably, the pre-training of the self-encoder neural network model having the convolutional neural network structure comprises: inputting pixel data of the image sample with the shading, convolving the pixel data according to the characteristic coefficient of the convolutional neural network, and extracting the image characteristics of the image sample with the shading layer by layer; performing convolution inverse operation on the pixel data, and reducing the image information of the input image sample with the shading layer by layer; after the image features of the image sample with the shading are input every time, compressing the image features; and decompressing the image information after the image information of the image sample with the shading is input every time of restoration, restoring the image size layer by layer, wherein the characteristic coefficient of the convolutional neural network is the convolutional kernel of each neural kernel in the convolutional neural network.
Preferably, the pre-trained neural network model is a recurrent consensus antagonism network model, which includes a first self-encoder, a second self-encoder and a classifier.
Preferably, the pre-trained self-encoder neural network model comprises: training a classifier to enable the classifier to judge whether the received shading image has effective shading information and is similar to a shading image sample enough; receiving the image sample with the shading by a first self-encoder, and performing feature extraction on the image sample with the shading to generate a first shading image; judging the first shading image by the classifier and outputting one of a first classification result and a second classification result; and when the classifier outputs a first classification result, the second self-encoder receives the first shading image, performs learning training on the first shading image and outputs a second shading image, wherein the first classification result shows that the first shading image has effective shading information and the similarity of the first shading image and the shading image sample is greater than or equal to a threshold value.
Preferably, the training of the feature coefficients between hidden nodes of each layer of the neural network model further comprises: and adjusting the characteristic coefficient of the nerve core of the first self-encoder when the classifier outputs a second classification result, wherein the second classification result indicates that the first shading image does not have effective shading information and/or the similarity of the first shading image and the shading image sample is less than a threshold value.
Preferably, before each of the background-image-samples in the background-image-sample set is input into the neural network model, normalization processing is further performed on the background images in a large number of different environments.
Preferably, each of the set of background image samples is input to the neural network model in batches.
An image processing apparatus according to the present invention includes: the system comprises an acquisition module, a shading generation module and a difference module, wherein the acquisition module is used for acquiring an acquired image; the shading generation module is connected with the acquisition module, receives the acquired image and is used for acquiring a shading image in the acquired image according to the neural network model; the difference module is respectively connected with the acquisition module and the shading generation module and used for receiving the acquired image and the shading image and obtaining the image after shading filtering through difference processing.
Preferably, the image processing device further comprises a storage module, connected to the shading generation module, and configured to store the set of shading-image samples, so as to train the neural network model in the shading generation module.
According to the present invention, there is provided an electronic apparatus comprising: a memory for storing computer readable instructions; and a processor for executing the computer readable instructions, such that the processor when executing implements the image processing method as described above.
According to a computer-readable storage medium provided by the present invention, computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a computer, the computer is caused to execute the image processing method as described above.
The invention has the beneficial effects that: the invention discloses an image processing method and device, electronic equipment and a readable storage medium. The neural network has good learning ability and self-adaptive ability, can improve the image information identification efficiency, and saves a large amount of manpower and material resources. Meanwhile, shading images in various different environments are collected and used as sample sets to train the neural network model, shading information extraction can be well carried out on the images collected in different environments, therefore, effective information which is clearer and better in quality is separated, and the neural network model has good environmental adaptability.
The self-encoder neural network model is adopted, so that the complexity of building the network model is simplified while the established function is realized.
Different neural network structures are adopted for the collected images at different positions, so that the operation resources are optimized, and the accuracy and the operation efficiency of processing the information of the different collected images can be improved.
The method has the advantages that the cyclic consistent countermeasure network is used as the neural network model to extract the shading images, so that the reality of the shading images corresponding to the collected images is improved when the neural network model is trained, and the problem that shading image samples used for training the neural network model are different from the shading images in the actually collected images can be solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings.
Fig. 1 is a block diagram showing a configuration of an image processing apparatus;
fig. 2 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method of image processing provided by an embodiment of the present invention;
FIG. 4(a) shows a connection block diagram of the shading generation module and the difference module during the training phase;
FIG. 4(b) shows a connection block diagram of the shading generation module and the difference module in the shading removal phase;
FIG. 5 is a schematic structural diagram of a neural network model provided in the first embodiment of the present invention;
FIG. 6 is a schematic diagram of the neural network model of FIG. 5 being an artificial neural network;
FIG. 7 is a schematic diagram of the neural network model of FIG. 5 as a convolutional neural network;
FIG. 8 is a schematic diagram of a neural network model provided in a second embodiment of the present invention;
fig. 9 is a schematic diagram illustrating a process of image processing in a captured image according to an embodiment of the present invention.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the drawings. The invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
The present invention will be described in detail below with reference to the accompanying drawings.
Fig. 2 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
As shown in fig. 2, in the present embodiment, the image processing apparatus includes an acquisition module 1, a shading generation module 2, and a difference module 3.
The acquisition module 1 is used for acquiring an acquired image.
The shading generation module 2 is provided with a corresponding neural network model, the shading generation module 2 is connected with the acquisition module 1 and used for receiving the acquired image, and the shading generation module 2 is used for acquiring the shading image in the acquired image according to the neural network model.
The difference module 3 is respectively connected with the acquisition module 1 and the shading generation module 2 and is used for receiving the acquired image and the shading image and obtaining the image after shading filtering after difference processing.
In a preferred embodiment, the image processing apparatus further comprises a storage module 4, and the storage module 4 is connected to the shading generation module 2 and is used for storing the set of shading image samples to train the neural network model in the shading generation module 2. When the neural network model in the shading generation module 2 is trained, the shading generation module 2 is further configured to receive the comparison result output by the difference module 3, and adjust the characteristic coefficient between hidden layer nodes of each layer in the neural network model according to the comparison result.
Further, the background image sample set includes a plurality of background image samples under different environments.
Fig. 3 shows a method flowchart of an image processing method according to an embodiment of the present invention, fig. 4(a) shows a connection block diagram of a shading generation module and a difference module in a training phase, fig. 4(b) shows a connection block diagram of a shading generation module and a difference module in a shading removal phase, and fig. 9 shows a schematic diagram of a process of image processing in a captured image according to an embodiment of the present invention.
With reference to fig. 2, fig. 3 and fig. 9, the captured image that needs to be subjected to information extraction generally includes valid information (such as fingerprint information), shading information and environment information, where the environment information may affect the acquisition of the shading information in the captured image. Therefore, when the effective information in the acquired image is extracted, the neural network model is required to extract and obtain the shading image affected by the environment, and then the difference module 3 is used for carrying out difference operation on the acquired image and the shading image affected by the environment so as to obtain the effective image in the acquired image and further obtain the effective information in the effective image. Among them, how to obtain or remove the shading image affected by the environment is a difficulty. In this embodiment, the image processing method includes performing steps S1 to S3, specifically as follows:
in step S1, a captured image is acquired.
In step S2, the captured image is input to a neural network model trained in advance, and a shading image in the captured image is obtained.
To acquire a valid image in the captured image, a shading image in the captured image is acquired first. In this embodiment, due to the influence of environmental factors, in order to improve the accuracy of image recognition and obtain shading images corresponding to different current environments, a neural network model needs to be trained in advance, an acquired image is input into the neural network model trained in advance, shading features in the acquired image are learned through the neural network model, and then the shading images are automatically extracted and generated. Because the neural network has good learning ability and self-adaptive ability, when image recognition is carried out, the neural network can slowly learn to recognize similar images through the self-learning ability only by inputting different image templates and recognition results into the neural network. The training process of the neural network is simple and convenient, the image recognition efficiency can be improved, and manpower and material resources are saved.
Further, in conjunction with fig. 4(a), the training process for the neural network model includes: collecting a large number of background image samples under different environments to form a background image sample set, inputting each background image sample in the background image sample set into a neural network model in a background generation module 2, extracting feature information of each background image sample in the background image sample set one by the neural network model, and correspondingly outputting a determined background image according to the feature information of each background image sample.
Then, the difference module 3 compares the judged shading image output by the neural network model with the known shading image of the sample with the shading image, judges whether the difference value is less than or equal to a preset value, and inputs the comparison result into the neural network model in the shading generation module 2. Further, when the difference value is smaller than or equal to a preset value, the neural network model learns and records the characteristic information of the shading image sample; and when the difference is larger than a preset value, adjusting the characteristic coefficients among hidden layer nodes of each layer of the neural network model, and then learning and recording the characteristic information of the shading image sample, so as to train the characteristic coefficients among the hidden layer nodes of each layer of the neural network model and obtain the trained neural network model.
After the neural network model finishes learning all the shading image sample sets in different environments, in the trained neural network model, various shading information can be recorded, and shading information extraction can be well carried out on images collected in different environments, so that more clear effective information with better quality can be separated.
Further, in order to minimize the influence of the shading differences on the neural network model training process, in this embodiment, a large number of shading images in different environments are normalized respectively, so as to ensure that the sizes of the shading images are consistent. Such as normalizing for size, coordinate centering, x-sharpening, scaling, and rotation.
Optionally, in this embodiment, in order to further improve the processing efficiency of the training process and reduce the number of times of the adjustment, a batch training mode may be adopted. Specifically, a large number of shading image samples under different environments are collected in a shading image sample set and are batched randomly, for example, each batch of 100 shading images, and each shading image sample in a batch of shading image samples is sequentially and randomly input into a neural network model to be trained in the shading generation module 2, so as to obtain a corresponding training result.
In step S3, an image after shading filtering is obtained based on the captured image and the shading image.
As shown in fig. 4(b), in the present embodiment, in the shading removal stage, the acquired collected image in the current environment is input into the trained neural network model in the shading generation module 2, the trained neural network model extracts shading information in the collected image, and the shading image is obtained according to the extracted shading information.
Then, the difference module 3 performs difference operation based on the collected image and the shading image to obtain an image (i.e. an effective image) with the shading filtered, so as to conveniently obtain effective information (such as fingerprint information) therefrom.
In the shading generation module 2 of this embodiment, the type of the pre-trained neural network model is not limited, and may be, for example, an Auto-Encoder (Auto-Encoder) neural network model or a cyclic consensus antagonistic (CycleGAN) neural network model.
Further, in a possible embodiment of the present invention, the pre-trained neural network model is a self-encoder neural network model, and the specific training process is as follows:
fig. 5 is a schematic structural diagram of a neural network model provided in a first embodiment of the present invention, fig. 6 is a schematic structural diagram of the neural network model in fig. 5 being an artificial neural network, and fig. 7 is a schematic structural diagram of the neural network model in fig. 5 being a convolutional neural network.
As shown in fig. 5, in the present embodiment, the self-encoder neural network model 21 includes two symmetric neural networks: an Encoder (Encoder) neural network 211 and a Decoder neural network (Decoder) 212.
The encoder neural network 211 is configured to receive the image sample with the shading, compress and extract feature information of the image sample with the shading layer by layer (i.e., encode pixel data of the image sample with the shading layer by layer); the decoder neural network 212 is configured to receive the feature information extracted by the encoder neural network 211, and recover the image information layer by layer according to the feature information (i.e., decode the feature information layer by layer), so as to obtain a recovered image with an outline. The similarity between the finally obtained restored image with the ground tint and the input image sample with the ground tint is larger than a preset threshold, namely the finally obtained restored image with the ground tint and the input image sample with the ground tint are similar as much as possible.
Further, the self-encoder neural network model 21 receives the image sample with the ground tint in the training stage and outputs the restored image with the ground tint; and in the shading removal stage, the collected image is received from the encoder neural network model 21, and the shading image in the collected image is output.
Further, when the self-encoder neural network model 21 is used to extract the feature information of the shading image sample, the number of nodes or information of the encoder neural network 211 is gradually decreased, and the number of nodes or information of the corresponding decoder neural network 212 is gradually increased. The symmetrical encoder neural network 211 and the decoder neural network 212 are adopted to compress and restore the image, and the processing process is simple.
Further, the encoder neural network 211 and the decoder neural network 212 in the self-encoder neural network model 21 may be any one of an Artificial Neural Network (ANN)211 (i.e., each neural core is a weighting coefficient for each image pixel gray scale) and a Convolutional Neural Network (CNN)212 (i.e., each neural network is a convolutional core, which may be considered as a filter). However, in an actual application process, in order to achieve the best acquisition of the shading image of the collected image in different environments, different neural networks are usually selected as the neural network models in the shading generation module 2 for different situations to extract the shading information, so as to improve the extraction efficiency and accuracy of the shading information in the collected image.
The method comprises the following specific steps:
as shown in fig. 6, for an image collected at a fixed position (in the same area of the sensor), a neural network model with an artificial neural network 211 structure should be selected for pre-training since the shading information of the image changes linearly with respect to pixels. At this time, the feature coefficients between hidden layer nodes of each layer of the neural network model in the shading generation module 2 are expressed as weighting coefficients of each neural core in the artificial neural network 211 for each image pixel gray scale.
In the artificial neural network 211, a flattening layer (Flatten) and a plurality of all-coupled layers (Dense) are included.
The flattening layer is used for receiving the multi-dimensional normalized shading image sample and outputting the multi-dimensional normalized shading image sample in a dimensionality reduction mode.
Since the full-link layer requires that the input data must be one-dimensional, the flattening layer must "flatten" the multi-dimensional, background-image sample input data into one dimension before it can enter the full-link layer. The flattening layer functions purely and therefore does not require any input parameters to be reflected on the code.
Preferably, the pixel data of the background-containing image sample is an array arrangement of m rows and n columns, and after the pixel data is input into the flattening layer, the flattening layer converts the pixel data of the m rows and n columns, and finally outputs a one-dimensional pixel data arrangement of m · n rows and 1 column. Where, denotes multiplication, and m and n are both positive integers.
And the plurality of full-link layers are positioned behind the flattening layer and used for receiving one-dimensional pixel data, compressing and restoring the one-dimensional pixel data according to the number of the nerve cores of each full-link layer, realizing weighted output in the compression and restoration processes, and finally obtaining a restored shading image with the same dimension and shape as the sample with the shading image.
Further, in the encoder neural network 2111, the number of the nerve cores of each full-link layer is sequentially decreased, in the decoder neural network 2112, the number of the nerve cores of each full-link layer is sequentially increased, and the number of the nerve cores of the last full-link layer in the decoder neural network 2112 is equal to the number of pixel data in the image sample with the shading, so that the weighted output of each pixel data in the image sample with the shading is realized through the cooperation of the encoder neural network 2111 and the decoder neural network 2112.
For example, the encoder neural network 2111 includes a flattening layer 21111, a first concatenated layer 21112, and a second concatenated layer 21113 which are arranged in this order, and the decoder neural network 2112 includes a third concatenated layer 21121, a fourth concatenated layer 21122, and a fifth concatenated layer 21123 which are arranged in this order. If the normalized shading sample image size input to the artificial neural network 211 is 144x144, the flattening layer 21111 may be set to change the input image into a one-dimensional vector (144x144) output, and the first full cascade layer 21112 has 64 neurons, the second full cascade layer 21113 has 16 neurons, the third full cascade layer 21121 has 64 neurons, the fourth full cascade layer 21122 has 16 neurons, the fifth full cascade layer 21123 has 144x144 neurons, and after the normalized image is weighted by the multiple full cascade layers, the original size of the image is finally restored.
The first full link layer 21112, the second full link layer 21113, the third full link layer 21121 and the fourth full link layer 21122 are hidden layers, and the activation function used is ReLu. The fifth inline layer 21123 is the output layer and the activation function used is tanh.
Further, the value ranges of the input and output images of the artificial neural network 211 are-1, and the images are subjected to gray scale compression and stretching transformation before and after entering the artificial neural network 211.
In summary, pre-training a self-encoder neural network model with an artificial neural network structure comprises: inputting pixel data of a multi-dimensional image sample with the shading, and performing dimension reduction output on the pixel data; compressing the pixel data output in a dimensionality reduction mode layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the compression process; reducing the compressed pixel data layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the reduction process; and performing up-dimensional output on the restored pixel data, and changing the output shape of the pixel data to be consistent with the shape of the input multi-dimensional pixel data. The collected image of the shading information which changes linearly relative to the pixels is processed by adopting the artificial neural network, and the speed and the accuracy of extracting the shading information of the image are improved by weighting the multi-level pixel data.
In the artificial neural network 21, the number of the flattening layers and the full link layers is not limited, and the above functions may be implemented as long as the above features are satisfied.
As shown in fig. 7, for the images collected at different positions (different areas of the sensor), a neural network model with a convolutional neural network 212 structure should be selected for pre-training, since the neural network must learn the shading characteristics. At this time, the feature coefficients between hidden layer nodes of each layer in the neural network model in the shading generation module 2 appear as convolution kernels of each neural kernel in the convolution neural network 212.
In the convolutional neural network 212, it includes: a plurality of convolutional layers (e.g., a first convolutional layer 21211, a second convolutional layer 21213, and a third convolutional layer 21225), the convolution kernel of each convolutional layer being convolved with the image for extracting image features layer by layer; a plurality of pooling layers (e.g., first pooling layer 21212 and second pooling layer 21214) for compressing image features layer-by-layer, simplifying the computational complexity of the neural network, and highlighting main features; a plurality of deconvolution layers (e.g., a first deconvolution layer 21221 and a second deconvolution layer 21223) for performing an inverse operation of image convolution to restore image information layer by layer; a plurality of anti-pooling layers (e.g., first anti-pooling layer 21222 and second anti-pooling layer 21224) are used to decompress the image information and restore the image size layer-by-layer.
Wherein, the encoder neural network 2121 comprises a plurality of convolutional layers and a plurality of pooling layers, and the convolutional layers and the pooling layers are alternately arranged; the decoder neural network 2122 includes a plurality of deconvolution layers and a plurality of inverse pooling layers, and the plurality of deconvolution layers and the plurality of inverse pooling layers are alternately arranged; and the last convolutional layer in the convolutional layers is positioned behind the anti-pooling layer of the last layer and is used for restoring the number of image channels.
Furthermore, the number of the output ends of each convolution layer is greater than that of the output ends of the subsequent pooling layer; the number of the output ends of each pooling layer is equal to that of the output ends of the convolution layer or the deconvolution layer of the next stage; the number of the output ends of each deconvolution layer is less than that of the output ends of the next-stage anti-pooling layer; the number of output ends of each anti-pooling layer is equal to the number of output ends of the anti-convolution layer or convolution layer at the next stage.
For example, the normalized shading sample image input to the convolutional neural network 212 is 144x144 in size, a first convolutional layer 21211 with 64 neural kernels may be set, a first pooling layer 21212 with a maximum of 2x2 pooling, a second convolutional layer 21213 with 32 neural kernels, a second pooling layer 21214 with a maximum of 3x3 pooling, a first deconvolution layer 21221 with 32 neural kernels, a second deconvolution layer 21222 with a deconvolution of 3x3, a second deconvolution layer 21223 with 128 neural kernels, a second deconvolution layer 21224 with a deconvolution of 3x3, and a third convolutional layer 21225 with 1 neural kernel, reducing the number of image input channels to 1 channel, and finally restoring the original size of the image.
The first convolution layer 21211, the second convolution layer 21213, the first deconvolution layer 21221, and the second deconvolution layer 21223 are hidden layers, and the activation function used is ReLu. The third convolutional layer 21225 is an output layer, and the activation function used is tanh.
Further, the padding values (i.e., the boundaries of a certain size filled before and after each dimension of the vector) of all convolutional or deconvolution layers in convolutional neural network 212 are the same, i.e., the input and output sizes of each convolutional or deconvolution layer are the same.
Further, the value ranges of the input and output images of the convolutional neural network 212 are-1, and the images are subjected to gray level compression and stretching transformation before and after entering the convolutional neural network 212.
In the convolutional neural network 212, the number of convolutional layers, pooling layers, anti-convolutional layers, and anti-pooling layers is not limited, and the above functions may be implemented as long as the above features are satisfied.
In summary, pre-training a self-encoder neural network model with a convolutional neural network structure comprises: inputting pixel data of the image sample with the shading, convolving the pixel data according to the characteristic coefficient of the convolutional neural network, and extracting the image characteristics of the image sample with the shading layer by layer; performing convolution inverse operation on the pixel data, and reducing the image information of the input image sample with the shading layer by layer; after the image characteristics of the image sample with the shading are input every time, compressing the image characteristics; and decompressing the image information after the image information of the image sample with the shading is input every time of restoration, and restoring the image size layer by layer. The convolution neural network is adopted to extract shading information from the relatively complex collected images at different positions, so that the accuracy of shading information extraction results is improved, and the operation is simple.
In this embodiment, the subsequent stage of each convolution layer is provided with one pooling layer, and the subsequent stage of each deconvolution layer is provided with one anti-pooling layer.
It should be noted that fig. 6 and fig. 7 are only exemplary drawings of the present invention, and the illustration of the number of layers of each neural network is also only exemplary, and these should not be construed as limiting the technical solution of the present invention. The invention aims at the prior art, the number of layers of the neural network is not limited, and the number of layers should be determined according to the actual design requirement.
In another possible embodiment of the present invention, the pre-trained neural network model is a cyclic consensus antagonistic network model, and the specific training process is as follows:
fig. 8 is a schematic structural diagram of a neural network model provided in the second embodiment of the present invention.
The ideal neural network model is to input the sample of the image with the shading and output the corresponding shading image. However, in practical application, when the neural network model is trained, it is difficult to obtain the corresponding real shading when the image is acquired, that is, there is a certain difference between the shading image sample used for training the neural network model and the shading image in the actually acquired image, and when the trained neural network model is used for shading extraction of the acquired image, deviation in accuracy is caused due to the difference, so that accuracy of effective information in the finally extracted acquired image is affected. Therefore, the present embodiment uses the cyclic consensus network model to obtain the shading image in the captured image, so as to enhance the capability of acquiring the shading image in different environments.
As shown in fig. 8, in the present embodiment, the loop consensus antagonistic network model 22 includes: a first self-encoder 221, a second self-encoder 222, and a classifier 223. The first self-encoder 221 is configured to receive the shading image sample and output a first shading image, the second self-encoder 222 is configured to receive the first shading image and output a second shading image, and the classifier 223 is configured to receive the first shading image, determine whether the first shading image has valid shading information and is sufficiently similar to the shading image sample, and output one of a first classification result and a second classification result. Further, when the first self-encoder 221 receives the first classification result, the characteristic coefficient of the own neural nucleus is adjusted.
Further, the pre-training of the cyclic consensus antagonistic network model comprises: training a classifier to enable the classifier to judge whether the received shading image has effective shading information and is similar to a shading image sample enough; receiving the image sample with the shading by a first self-encoder, and performing feature extraction on the image sample with the shading to generate a first shading image; judging the first shading image by a classifier and outputting one of a first classification result and a second classification result; and when the classifier outputs a first classification result, the second self-encoder receives the first shading image, performs learning training on the first shading image and outputs a second shading image. The second self-encoder is a neural network used for shading extraction of the collected image. By the method, the collected shading image sample set can be mapped to other training sample sets which are not collected but are similar to the shading image sample set, so that the number of training samples of the neural network corresponding to the second self-encoder is increased, the difference between the shading image sample used for training the neural network model and the shading image in the actually collected image is reduced, and the accuracy of obtaining the shading image of the collected image is improved.
Furthermore, the first shading image is expressed as a shading image similar to the input shading image sample and is equivalent to a mapping image of the input shading image sample; the second shading image is represented as a reconstructed shading image, and the similarity between the second shading image and the input shading image sample is greater than a preset threshold value.
Further, in the present embodiment, the auto-encoder in the cyclic consensus antagonistic network model 22 can use the corresponding neural network structure (ANN or CNN) according to different conditions. For example, the classifier 223 in the recurrent consensus antagonistic network model 22 may use a convolutional neural network classifier, which has the following structure (where the image size input to the classifier 223 is assumed to be 144x 144):
a classifier convolutional layer having 16 neural nuclei; a first classifier pooling layer, performing a maximum pooling of 2x 2; a first classifier fully-connected layer with 32 nuclei; a second classifier pooling layer, performing a maximum pooling of 2x 2; a second classifier fully-connected layer with 64 nuclei; a third classifier pooling layer, performing maximum pooling of 2x 2; a classifier flattening layer that changes the image to a 1-dimensional vector (144x 144); a third classifier fully connected layer with 64 nuclei; dropping (dropout) layers, the output of the upper layer (third cascade layer) being randomly discarded with a 50% probability, so that training of one nucleus is not dependent on another; the fourth classifier is fully connected, has 1 nucleus, and finally restores the original size of the image.
The classifier convolutional layer, the first classifier full-link layer, the second classifier full-link layer and the third classifier full-link layer are hidden layers, and the used activation function is ReLu. The fourth classifier full-link layer is an output layer, and the used activation function is sigmoid.
It should be noted that, in this document, the transformation parameters from the neural network of each layer to the neural network of the next layer are trained parameters, and all the trained parameters are corresponding neural network models for training.
In one possible embodiment, the classification result output by the classifier may be 1 to 4, where 1 represents the first classification result and 2 to 4 represent the second classification result. Specifically, 1 indicates that the first shading image has effective information and the similarity of the first shading image and the input shading image sample is greater than or equal to a threshold value; 2, the first shading image does not carry effective information but has similarity with the input shading image sample which is greater than or equal to a threshold value; 3, the first shading image is provided with effective information and has similarity with the input shading image sample less than a threshold value; 4 means that the first shading image does not carry valid information and the similarity with the input sample of the shading image is less than the threshold value.
In another possible embodiment, the classifier outputs classification results of 0 and 1. Wherein, 1 represents that the first shading image has effective information and the similarity with the input shading image sample is larger than or equal to the threshold value, and 0 represents other conditions.
In one possible embodiment, training the feature coefficients between hidden layer nodes of each layer of the neural network model further includes: the feature coefficients of the neural nuclei of the first autoencoder are adjusted when the classifier output represents the second classification result. Different degrees of mapping of the input background-textured image samples are achieved by adjusting the characteristic coefficients of the neural nuclei of the first self-encoder.
The method has the advantages that the anti-network model with the consistent circulation is adopted to extract the shading images, so that the authenticity of the shading images in the collected images is improved when the neural network model is trained, meanwhile, more extensive shading images in different environments can be obtained, and the acquisition capacity and the acquisition quality of the shading images in the collected images in different environments are further improved.
The embodiment of the invention also discloses electronic equipment which comprises a memory and a processor. Wherein the memory is to store computer readable instructions; the processor is configured to execute the computer readable instructions, such that the processor when executing implements the image processing method as described above.
The embodiment of the invention also discloses a computer readable storage medium, wherein computer readable instructions are stored on the computer readable storage medium, and when the computer readable instructions are executed by a computer, the computer is enabled to execute the image processing method.
In conclusion, the method and the device can be applied to the process of separating effective information and shading in the image acquisition process. The shading under the current environment can be better extracted according to the input collected image. And further, more clear and better-quality effective information is separated from the collected image, so that preparation is made for subsequent work (such as fingerprint identification), and the subsequent work result such as a fingerprint identification result is more accurate.
It should be noted that the technical solution disclosed in the present invention is not only suitable for removing shading information in an acquired image, but also suitable for extracting and removing other information in the image, and meanwhile, a technical solution obtained by a person skilled in the art on the basis of this or through simple deformation should also be within the scope of the present invention.
It should be noted that, in this document, the contained terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that: it should be understood that the above examples are only for clearly illustrating the present invention and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.
Claims (15)
1. An image processing method, comprising:
acquiring a collected image;
inputting the collected image into a pre-trained neural network model to obtain a shading image in the collected image;
obtaining an image after filtering the shading based on the collected image and the shading image,
wherein the neural network model is trained by: inputting each ground tint image sample in the ground tint image sample set into the neural network model, outputting the judged ground tint image by the neural network model, and comparing the ground tint image with the known ground tint image of the ground tint image sample, thereby training the characteristic coefficients among hidden layer nodes of each layer of the neural network model and obtaining the trained neural network model.
2. The image processing method of claim 1, wherein the pre-trained neural network model is a self-encoder neural network model.
3. The image processing method of claim 2, wherein pre-training the self-encoder neural network model comprises:
pre-training a self-encoder neural network model with an artificial neural network structure for images acquired at fixed positions;
for images acquired at different positions, a self-encoder neural network model with a convolutional neural network structure is trained in advance.
4. The image processing method of claim 3, wherein the self-encoder neural network model comprises a symmetric encoder neural network and a decoder neural network,
the encoder neural network is used for receiving the image sample with the shading, compressing and extracting the characteristic information of the image sample with the shading layer by layer;
the decoder neural network is used for receiving the characteristic information extracted by the encoder neural network and restoring the image sample with the shading layer by layer according to the characteristic information.
5. The image processing method of claim 4, wherein pre-training the self-encoder neural network model with the artificial neural network structure comprises:
inputting pixel data of the multi-dimensional shading image sample, and performing dimension reduction output on the pixel data;
compressing the pixel data output in a dimensionality reduction mode layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the compression process;
reducing the compressed pixel data layer by layer, and performing weighted output on each pixel data in the pixel data according to the characteristic coefficient of the artificial neural network in the reduction process;
performing upscaling output on the restored pixel data, changing the output shape of the pixel data to be consistent with the shape of the input multi-dimensional pixel data,
wherein the characteristic coefficient of the artificial neural network is a weighting coefficient of each neural core in the artificial neural network for each image pixel gray scale.
6. The image processing method of claim 4, wherein pre-training the self-encoder neural network model having the convolutional neural network structure comprises:
inputting pixel data of the image sample with the shading, convolving the pixel data according to the characteristic coefficient of the convolutional neural network, and extracting the image characteristics of the input image sample with the shading layer by layer;
performing convolution inverse operation on the pixel data, and reducing the image information of the input image sample with the ground pattern layer by layer;
wherein after the image features of the input image sample with the ground tint are extracted once, the image features are compressed,
decompressing the image information after restoring the image information of the input image sample with the shading once, restoring the image size layer by layer,
the characteristic coefficient of the convolutional neural network is a convolutional kernel of each neural kernel in the convolutional neural network.
7. The image processing method according to claim 1, wherein the pre-trained neural network model is a recurrent consensus antagonistic network model,
the round robin reconciliation countermeasure network model includes a first autoencoder, a second autoencoder, and a classifier.
8. The image processing method of claim 7, wherein pre-training a round-robin reconciliation countermeasure network model comprises:
training the classifier to enable the classifier to judge whether the received shading image has effective shading information and is similar to the shading image sample enough;
receiving the shading image sample by the first self-encoder, performing feature extraction on the shading image sample and generating a first shading image;
judging the first shading image by the classifier and outputting one of a first classification result and a second classification result;
when the classifier outputs the first classification result, the second self-encoder receives the first shading image, performs learning training on the first shading image and outputs a second shading image,
wherein the first classification result indicates that the first shading image has effective shading information and the similarity of the first shading image and the shading image sample is greater than or equal to a threshold value.
9. The image processing method of claim 8, wherein training the feature coefficients between hidden layer nodes of each layer of the neural network model further comprises:
adjusting feature coefficients of a neural kernel of the first self-encoder when the classifier outputs the second classification result,
wherein the second classification result indicates that the first shading image does not have effective shading information and/or that the similarity of the first shading image and the shading image sample is less than a threshold value.
10. The image processing method according to claim 1, wherein before inputting each of the set of the background-image samples into the neural network model, further comprising normalizing the background images in a plurality of different environments.
11. The image processing method of claim 10, wherein each of the set of background-image samples is input to the neural network model in batches.
12. An image processing apparatus, comprising: an acquisition module, a shading generation module and a difference module,
the acquisition module is used for acquiring an acquired image;
the shading generation module is connected with the acquisition module, receives the acquired image and is used for acquiring a shading image in the acquired image according to a neural network model;
the difference module is respectively connected with the acquisition module and the shading generation module and used for receiving the acquired image and the shading image and obtaining the image after shading filtering through difference processing.
13. The image processing apparatus according to claim 12, wherein the image processing apparatus further comprises a storage module connected to the shading generation module for storing a set of shading image samples for training the neural network model in the shading generation module.
14. An electronic device, comprising:
a memory for storing computer readable instructions; and
a processor for executing the computer readable instructions such that the processor when executed implements the image processing method of any of claims 1-11.
15. A computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a computer, cause the computer to perform the image processing method of any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010106684.9A CN111340188B (en) | 2020-02-21 | Image processing method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010106684.9A CN111340188B (en) | 2020-02-21 | Image processing method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111340188A true CN111340188A (en) | 2020-06-26 |
| CN111340188B CN111340188B (en) | 2024-06-28 |
Family
ID=
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112686289A (en) * | 2020-12-24 | 2021-04-20 | 微梦创科网络科技（中国）有限公司 | Picture classification method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657969A (en) * | 2013-11-25 | 2015-05-27 | 方正国际软件(北京)有限公司 | Method and system for removing image background |
| CN107977686A (en) * | 2017-12-22 | 2018-05-01 | 浙江大学 | A kind of sorting technique of industrial X-ray image |
| CN109831597A (en) * | 2019-02-28 | 2019-05-31 | 江苏实达迪美数据处理有限公司 | A kind of shading halftoning method based on deep learning |
| US20190220746A1 (en) * | 2017-08-29 | 2019-07-18 | Boe Technology Group Co., Ltd. | Image processing method, image processing device, and training method of neural network |
| CN110232349A (en) * | 2019-06-10 | 2019-09-13 | 北京迈格威科技有限公司 | Shield lower fingerprint and removes shading method, apparatus, computer equipment and storage medium |
| CN110263754A (en) * | 2019-06-28 | 2019-09-20 | 北京迈格威科技有限公司 | Shield lower fingerprint and removes shading method, apparatus, computer equipment and storage medium |
| CN110348425A (en) * | 2019-07-19 | 2019-10-18 | 北京迈格威科技有限公司 | Go the method, apparatus, equipment and computer readable storage medium of shading |
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657969A (en) * | 2013-11-25 | 2015-05-27 | 方正国际软件(北京)有限公司 | Method and system for removing image background |
| US20190220746A1 (en) * | 2017-08-29 | 2019-07-18 | Boe Technology Group Co., Ltd. | Image processing method, image processing device, and training method of neural network |
| CN107977686A (en) * | 2017-12-22 | 2018-05-01 | 浙江大学 | A kind of sorting technique of industrial X-ray image |
| CN109831597A (en) * | 2019-02-28 | 2019-05-31 | 江苏实达迪美数据处理有限公司 | A kind of shading halftoning method based on deep learning |
| CN110232349A (en) * | 2019-06-10 | 2019-09-13 | 北京迈格威科技有限公司 | Shield lower fingerprint and removes shading method, apparatus, computer equipment and storage medium |
| CN110263754A (en) * | 2019-06-28 | 2019-09-20 | 北京迈格威科技有限公司 | Shield lower fingerprint and removes shading method, apparatus, computer equipment and storage medium |
| CN110348425A (en) * | 2019-07-19 | 2019-10-18 | 北京迈格威科技有限公司 | Go the method, apparatus, equipment and computer readable storage medium of shading |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112686289A (en) * | 2020-12-24 | 2021-04-20 | 微梦创科网络科技（中国）有限公司 | Picture classification method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Divakar et al. | Image denoising via CNNs: An adversarial approach | |
| Bell et al. | The “independent components” of natural scenes are edge filters | |
| CN111444881A (en) | Fake face video detection method and device | |
| EP1262907A1 (en) | Pattern recognition with hierarchical networks | |
| Nhat et al. | Feature fusion by using LBP, HOG, GIST descriptors and Canonical Correlation Analysis for face recognition | |
| CN112818764B (en) | Low-resolution image facial expression recognition method based on feature reconstruction model | |
| US20120063689A1 (en) | Object recognition in an image | |
| AU2019226224B2 (en) | A method of extracting features from a fingerprint represented by an input image | |
| CN110245621B (en) | Face recognition device, image processing method, feature extraction model, and storage medium | |
| CN113095156B (en) | Double-current network signature identification method and device based on inverse gray scale mode | |
| Lakshmi et al. | Off-line signature verification using Neural Networks | |
| CN110222568B (en) | Cross-visual-angle gait recognition method based on space-time diagram | |
| CN111382601A (en) | Illumination face image recognition preprocessing system and method for generating confrontation network model | |
| CN116229528A (en) | Living body palm vein detection method, device, equipment and storage medium | |
| Hotta et al. | Face matching through information theoretical attention points and its applications to face detection and classification | |
| Hussain et al. | Image denoising to enhance character recognition using deep learning | |
| Hasan et al. | Arabian date classification using CNN algorithm with various pre-trained models | |
| CN110503157B (en) | Image steganalysis method of multitask convolution neural network based on fine-grained image | |
| Chartier et al. | FEBAM: A feature-extracting bidirectional associative memory | |
| Gokul Rajan et al. | Modified CNN based feature extraction for sclera recognition of off | |
| KR20180092453A (en) | Face recognition method Using convolutional neural network and stereo image | |
| CN111340188B (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN111340188A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| Pal et al. | Face detection using artificial neural network and wavelet neural network | |
| Wu et al. | Sparse graph based deep learning networks for face recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |