CN111797842A - Image analysis method and device and electronic equipment - Google Patents
Image analysis method and device and electronic equipment Download PDFInfo
- Publication number
- CN111797842A CN111797842A CN202010641901.4A CN202010641901A CN111797842A CN 111797842 A CN111797842 A CN 111797842A CN 202010641901 A CN202010641901 A CN 202010641901A CN 111797842 A CN111797842 A CN 111797842A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- biological
- model
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image analysis method and device and electronic equipment. Wherein, the method comprises the following steps: receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed; extracting a target symmetric feature in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the integral symmetric feature of the original biological image into a regional symmetric feature of the target biological image so as to enable the target network model to extract the target symmetric feature; and displaying the target symmetric characteristics of the biological image. The invention solves the technical problem that the symmetric characteristics of the biological image are difficult to identify in the related art.
Description
Technical Field
The invention relates to the technical field of image analysis, in particular to an image analysis method and device and electronic equipment.
Background
In the related art, in terms of image recognition and image classification, a common way is to recognize image-related features by using image library similarity comparison or using various network models, current image recognition, but current image recognition cannot be applied to biological images in which symmetric features of biological images (structural features of biological molecules are indispensable parts in biological research, wherein symmetry is a feature common in biological molecules and cell structures (especially, biological genetic materials)) exist in the whole image, depending on all the features in the image, and current image feature extraction is to extract local features of images (for example, CNN extracts local features of images by convolution kernel calculation, which weakens correlation information between image features) which weakens correlation information between image features, there is often a need to identify image symmetry features, and the traditional method for identifying the symmetry of biological images is to use geometric mathematical computation or a convolutional neural network, but there are still some problems and disadvantages in using the geometric mathematical computation method: the first point is as follows: the position of the rotation axis or the symmetry axis needs to be assumed in advance, resulting in complicated calculation; second, processing low-precision or noisy data is not performed well; third, a threshold is required to be set for determining the local non-strict symmetry, but the threshold selection and the threshold calculation algorithm are difficult to determine. The convolutional neural network is difficult to identify the features depending on the whole image, and as the number of layers of convolution increases, the symmetry of the input image discarded by the model becomes stronger and stronger, so that the accuracy of identifying the symmetric features of the biological image is lower.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides an image analysis method, an image analysis device and electronic equipment, which are used for at least solving the technical problem that the symmetric characteristics of a biological image are difficult to identify in the related technology.
According to an aspect of an embodiment of the present invention, there is provided an image analysis method including: receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed; extracting a target symmetric feature in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the integral symmetric feature of the original biological image into a region symmetric feature of the target biological image so as to enable the target network model to extract the target symmetric feature; and displaying the target symmetric characteristics of the biological image.
Optionally, the target network model is a model constructed based on a convolutional neural network CNN.
Optionally, the preprocessing core includes: the conversion component and the sampling component adopt a target network model to extract target symmetric characteristics in the biological image, and the steps comprise: receiving an original biological image to be analyzed by adopting a conversion component in the preprocessing kernel; rotating each original biological image according to a preset interval angle by adopting a conversion component in the preprocessing kernel; calculating the image difference between the rotated target biological image and the original biological image by adopting a conversion component in the preprocessing kernel to obtain a plurality of images with multi-dimensional pixels with rotation characteristics, and outputting the images with the multi-dimensional pixels; dividing the image of the multi-dimensional pixels into a plurality of pixel regions by adopting the sampling assembly; calculating an average value of all pixels in the plurality of pixel regions; based on the average value, performing pixel down-sampling processing on the image of the multi-dimensional pixels to obtain a target biological image of one-dimensional pixels; and extracting the target symmetric characteristics in the target biological image of the one-dimensional pixels.
Optionally, the target network model is constructed by an interactive visual analysis system, wherein the interactive visual analysis system comprises: the system comprises a network generation and training module, a deconvolution network generation module and a visual analysis module, wherein the network generation and training module is used for setting the structure of a target network model and the parameters of each layer in the target network model and training the target network model; the deconvolution network generation module is used for developing a deconvolution network and generating a deconvolution image of the input biological image; the visual analysis module is used for comparing and analyzing a plurality of constructed network models and analyzing the hierarchical structure parameters of each network model, wherein the hierarchical structure parameters at least comprise: the number of convolution input channels, the number of convolution output channels and the sampling data of the preprocessing kernel.
Optionally, the analysis method further comprises: obtaining the accuracy of each network model in the plurality of network models when analyzing the symmetric characteristics of the biological image; determining rotation parameters of a preprocessing kernel in each network model when the preprocessing kernel analyzes the symmetric features of the biological image; determining model identification data based on the accuracy and a rotation parameter of the preprocessing kernel; and indicating the model identification accuracy by a vertical axis, indicating the rotation parameters of the preprocessing kernel by a horizontal axis, and displaying the model identification data of each network model on a preset overview view interface.
Optionally, the analysis method further comprises: acquiring model construction information; determining a model structure of each network model, a kernel parameter of a preprocessing kernel and a layer propagation direction of each model layer based on the model construction information; and displaying the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer on a visual interface of a preset structure.
Optionally, the analysis method further comprises: receiving case selection information, wherein the case selection information is used for indicating a selected case biological image in an image library; importing the case biological image into a target network model, wherein after the target network model analyzes the case biological image, case image information is output, and the case image information at least comprises: whether the case biological images are symmetrical or not, the image symmetry characteristics, the case output images of the deconvolution layer based on the deconvolution network and the image enhancement function; and displaying the case image information on a preset case image visual interface.
Optionally, the step of displaying the case image information on the case image visualization interface includes: after receiving a case output image, performing decoding processing on the case output image to obtain a first decoded image, wherein the decoding processing mode is to divide the case output image into one-dimensional vectors according to rows and shape each vector into an output image of a multi-dimensional pixel to determine the first decoded image; analyzing the case output image by adopting a preprocessing kernel to obtain a second decoding image; analyzing information entropies of the first decoded image and the second decoded image in an image entropy mode, wherein the information entropies are used for indicating average characteristic information quantity in the images; drawing an output line graph of the case output image based on the information entropy; and displaying the output line graph on the case image visualization interface.
According to another aspect of the embodiments of the present invention, there is also provided an image analysis apparatus including: the receiving unit is used for receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed; the extraction unit is used for extracting target symmetric features in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric features of an original biological image into regional symmetric features of a target biological image so as to enable the target network model to extract the target symmetric features; and the display unit is used for displaying the target symmetric characteristics of the biological image.
Optionally, the target network model is a model constructed based on a convolutional neural network CNN.
Optionally, the preprocessing core includes: a conversion component and a sampling component, the extraction unit comprising: the first receiving module is used for receiving an original biological image to be analyzed by adopting a conversion component in the preprocessing kernel; the first rotating module is used for rotating each original biological image according to a preset interval angle by adopting a conversion component in the preprocessing kernel; the first calculation module is used for calculating the image difference between the rotated target biological image and the original biological image by adopting a conversion component in the preprocessing kernel to obtain a plurality of images with multi-dimensional pixels with rotation characteristics and outputting the images with the multi-dimensional pixels; a first dividing module for dividing the image of the multi-dimensional pixels into a plurality of pixel regions using the sampling component; the second calculation module is used for calculating the average value of all pixels in the pixel areas; the down-sampling processing module is used for carrying out pixel down-sampling processing on the image of the multidimensional pixel based on the average value to obtain a target biological image of the one-dimensional pixel; the first extraction module is used for extracting target symmetric features in the target biological image of the one-dimensional pixels.
Optionally, the target network model is constructed by an interactive visual analysis system, wherein the interactive visual analysis system comprises: the system comprises a network generation and training module, a deconvolution network generation module and a visual analysis module, wherein the network generation and training module is used for setting the structure of a target network model and the parameters of each layer in the target network model and training the target network model; the deconvolution network generation module is used for developing a deconvolution network and generating a deconvolution image of the input biological image; the visual analysis module is used for comparing and analyzing a plurality of constructed network models and analyzing the hierarchical structure parameters of each network model, wherein the hierarchical structure parameters at least comprise: the number of convolution input channels, the number of convolution output channels and the sampling data of the preprocessing kernel.
Optionally, the image analysis apparatus further comprises: the first acquisition module is used for acquiring the accuracy of each network model in the plurality of network models when the symmetrical features of the biological image are analyzed; the first determination module is used for determining rotation parameters of a preprocessing kernel in each network model when the preprocessing kernel analyzes the symmetric features of the biological image; a second determination module for determining model identification data based on the accuracy and a rotation parameter of the preprocessing kernel; the first display module is used for indicating the model identification accuracy by a vertical axis, indicating the rotation parameters of the preprocessing kernel by a horizontal axis and displaying the model identification data of each network model on a preset overview view interface.
Optionally, the image analysis apparatus further comprises: the second acquisition module is used for acquiring model construction information; the third determining module is used for determining the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer based on the model construction information; and the second display module is used for displaying the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer on a visual interface of a preset structure.
Optionally, the image analysis apparatus further comprises: the second receiving module is used for receiving case selection information, wherein the case selection information is used for indicating the selected case biological image in the image library; an importing module, configured to import the case biological image into a target network model, where after the target network model analyzes the case biological image, case image information is output, where the case image information at least includes: whether the case biological images are symmetrical or not, the image symmetry characteristics, the case output images of the deconvolution layer based on the deconvolution network and the image enhancement function; and the third display module is used for displaying the case image information on a preset case image visual interface.
Optionally, the third display module comprises: the decoding submodule is used for performing decoding processing on the case output image after receiving the case output image to obtain a first decoded image, wherein the decoding processing mode is that the case output image is divided into one-dimensional vectors according to rows, and each vector is shaped into an output image of multi-dimensional pixels to determine the first decoded image; the first analysis submodule is used for analyzing the case output image by adopting a preprocessing kernel to obtain a second decoding image; the second analysis submodule is used for analyzing the information entropy of the first decoded image and the second decoded image in an image entropy mode, wherein the information entropy is used for indicating the average characteristic information quantity in the images; the display submodule is used for drawing an output line graph of the case output image based on the information entropy; and displaying the output line graph on the case image visualization interface.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the image analysis methods described above via execution of the executable instructions.
In the embodiment of the invention, when the symmetric features of a biological image are analyzed, an image set to be processed is received, wherein the image set comprises a plurality of biological images to be analyzed, a target network model is adopted to extract the target symmetric features in the biological images, the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric features of an original biological image into the regional symmetric features of the target biological image, so that the target network model extracts the target symmetric features and displays the target symmetric features of the biological images. In the embodiment, the integral symmetric feature of the biological image can be converted into the regional symmetric feature of the target biological image by using the preprocessing kernel carried by the model, so that the model can accurately identify the symmetric feature of the biological image, and the technical problem that the symmetric feature of the biological image is difficult to identify in the related technology is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of an alternative image analysis method according to an embodiment of the invention;
fig. 2 is a schematic diagram of an alternative image analysis apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
To facilitate an understanding of the present application, some terms or nouns referred to in the embodiments of the present application are explained below:
the Convolutional Neural network, called CNN for short, is a kind of feedforward Neural network that includes convolution calculation and has a depth structure, and includes: convolutional layers, pooling layers, etc., by which features are extracted and by which the number of parameters is reduced.
The CNN model is the most commonly used method in image classification and image recognition at present, particularly, as data accumulation increases, the efficiency and robustness of the CNN in an image pattern recognition task are more obvious, different from the traditional image recognition problem, the symmetric features of the biological image exist in the whole image, and depend on all the features in the image.
The embodiment of the invention can be applied to identification terminals and identification systems of various biological images and biomolecular images, provides a network model (which can be understood as a CNN model) with a preprocessing kernel to solve the problem of symmetric identification of biological images, and also provides an interactive visual analysis system which can provide a network model with highest efficiency and highest identification accuracy for analysts. The invention is illustrated below with reference to various examples.
Example one
In accordance with an embodiment of the present invention, there is provided an image analysis method embodiment, it being noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a flow chart of an alternative image analysis method according to an embodiment of the present invention, as shown in fig. 1, the method comprising the steps of:
step S102, receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed;
step S104, extracting target symmetric features in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric features of the original biological image into regional symmetric features of the target biological image so as to enable the target network model to extract the target symmetric features;
and step S106, displaying the target symmetric characteristics of the biological image.
Through the steps, when the symmetric features of the biological image are analyzed, an image set to be processed is received, wherein the image set comprises a plurality of biological images to be analyzed, the target symmetric features in the biological images are extracted by adopting a target network model, the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric features of the original biological images into the regional symmetric features of the target biological images, so that the target network model extracts the target symmetric features and displays the target symmetric features of the biological images. In the embodiment, the integral symmetric feature of the biological image can be converted into the regional symmetric feature of the target biological image by using the preprocessing kernel carried by the model, so that the model can accurately identify the symmetric feature of the biological image, and the technical problem that the symmetric feature of the biological image is difficult to identify in the related technology is solved.
The present invention will be described in detail with reference to the above steps.
Step S102, receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed.
And step S104, extracting the target symmetric characteristics in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric characteristics of the original biological image into the regional symmetric characteristics of the target biological image so as to enable the target network model to extract the target symmetric characteristics.
Optionally, the target network model is a model constructed based on the convolutional neural network CNN.
In an embodiment of the present invention, the pre-processing core includes: the conversion component and the sampling component adopt a target network model to extract target symmetric characteristics in the biological image, and the method comprises the following steps: receiving an original biological image to be analyzed by adopting a conversion component in a preprocessing kernel; rotating each original biological image according to a preset interval angle by adopting a conversion component in the preprocessing kernel; calculating the image difference between the rotated target biological image and the original biological image by adopting a conversion component in the preprocessing kernel to obtain a plurality of images with multi-dimensional pixels with rotation characteristics, and outputting the images with the multi-dimensional pixels; dividing an image of the multidimensional pixel into a plurality of pixel areas by adopting a sampling component; calculating an average value of all pixels in the plurality of pixel regions; based on the average value, performing pixel down-sampling processing on the image of the multi-dimensional pixels to obtain a target biological image of one-dimensional pixels; and extracting the target symmetric characteristics in the target biological image of the one-dimensional pixels.
The preprocessing kernel comprises a conversion component and a sampling component, and the working process of the preprocessing kernel is described by taking a two-degree rotation kernel, a 64 x 64 pixel input image and a quarter sampling as an exemplary illustration. First, image data is imported into the conversion component, 180 different images of 64 × 64 pixels (indicating the above-mentioned multi-dimensional pixels) with rotation characteristics can be obtained by rotating each image by 2 degrees and calculating the difference between the rotated target biological image and the original biological image, and then each block of the 64 × 64 image output by the conversion component is divided into regions (indicating the above-mentioned multiple pixel regions) based on every 8 × 8 pixels in the sampling component. The average value of pixels in each area is taken, pixel values in all the areas are obtained, each image is down-sampled to be an 8 x 8 pixel image, all obtained data are reconstructed into one dimension from two dimensions, 180 images of 64 x 1 pixels can be obtained, and finally, the one-dimension data are spliced into two-dimension data, so that the symmetric characteristics in the biological image can be slowly analyzed.
The target network model is constructed by an interactive visual analysis system to help the user analyze and compare the convolutional network model from multiple angles and construct an effective network model, an image pre-processing kernel is added inside the network model to process and transform the input image, and by transforming the original image, the overall symmetric features of the image are transformed into the regional features of the new image so that the features of the entire image do not determine it anymore.
In an embodiment of the present invention, the interactive visual analysis system includes a plurality of views, such as a comparison view, a summary view, a structure visualization view, a case image visualization view, a model hierarchy view, and the like.
When designing an interactive visual analysis system, the following requirements need to be considered:
r1-providing a model overview view to compare different network models comprehensively, wherein the feature recognition accuracy of the model is the most important evaluation index, and preprocessing kernel rotation parameters of the model and the convolution layer of the model need to be displayed on the overview view.
R2-provides a structural analysis view of the model. The system is required to provide a hierarchical structure diagram that visualizes the selected model and includes specific details of any layer, such as the number of input channels and the number of output channels for the convolutional layer, as well as the sampling parameters for the processing core.
R3-provides a model analysis function for a particular input image. As the most common means of visualization of convolutional neural networks, a deconvolution network can effectively display which parts or characteristic information of an input image the convolutional neural network model is interested in. The visual analysis system of the present application may support visual analysis of each layer of the model, including convolutional layer processing kernels. The output of each layer of the deconvolution network can be viewed for a particular input image, and if it is a network with processing cores, they also want to be able to view the output image of the deconvolution network through the inverse processing cores.
R4-provides the functions of building models and training models. By setting the structure of the model and the parameters of each layer, the system can construct a relevant network model, train it, and provide the results for model analysis.
Wherein the interactive visual analysis system comprises: a network generation and training module (corresponding to the requirement of R4), a deconvolution network generation module (corresponding to the requirement of R3) and a visualization analysis module (corresponding to the requirements of R1 and R2), wherein the network generation and training module is used for setting the structure of the target network model and the parameters of each layer in the target network model and training the target network model; the deconvolution network generation module is used for developing a deconvolution network and generating a deconvolution image of the input biological image; the visual analysis module is used for comparing and analyzing a plurality of network models which are constructed, and analyzing the hierarchical structure parameters of each network model, wherein the hierarchical structure parameters at least comprise: the number of convolution input channels, the number of convolution output channels and the sampling data of the preprocessing kernel.
Optionally, the interactive visual analysis system further comprises: and the visual analysis module is used for importing the input case biological image into the network model and a pre-generated deconvolution network so as to test the accuracy of the model. Besides constructing a new model, the visual analysis system can also show the performance comparison result of each model and the structural hierarchy of each model.
The visual analysis system can receive at least one designed network model and case image, then the model is sent to the network generation and training module to train the model, the trained model can be used for user interaction, a model analysis result and an image analysis result are obtained through the visual analysis module, and the deconvolution network generation module generates a corresponding deconvolution network according to a preselected network model.
And step S106, displaying the target symmetric characteristics of the biological image.
The visual analysis interface of the interactive visual analysis system is described in detail below.
First model overview visualization
In an embodiment of the present invention, the analysis method further includes: obtaining the accuracy of each network model in the plurality of network models when analyzing the symmetric characteristics of the biological image; determining rotation parameters of a preprocessing kernel in each network model when the preprocessing kernel analyzes the symmetric features of the biological image; determining model identification data based on the accuracy and the rotation parameters of the preprocessing kernel; and indicating the model identification accuracy by a vertical axis, indicating the rotation parameters of the preprocessing kernel by a horizontal axis, and displaying the model identification data of each network model on a preset overview view interface.
The embodiment of the invention provides an overview visualization method, which can visually display the performance of different network models in the image symmetry recognition problem, the model performance and other indexes on a preset overview view interface. In the overview view, each circle may represent a model, and the model may be analyzed from multiple aspects, such as the recognition accuracy of the model. In order to better emphasize the importance of different factors, the model is roughly analyzed in the form of a scatter diagram, wherein the vertical axis of the scatter diagram represents the recognition accuracy of the model, and the horizontal axis of the scatter diagram displays the rotation parameters of the preprocessing kernel of the model. If the analyzed model does not process the preprocessing kernel, its abscissa value is 0 in the graph. Optionally, in the embodiment of the present invention, the color of the circle may represent an operation floating point number required for symmetric recognition of an image, the size of the circle indicates the number of convolution layers included in the model, and a user may search specific structure information and parameter information of the model by selecting the model to be checked.
Second model Structure visualization
Optionally, the analysis method further comprises: acquiring model construction information; determining a model structure of each network model, a kernel parameter of a preprocessing kernel and a layer propagation direction of each model layer based on model construction information; and displaying the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer on a visual interface of a preset structure.
In the embodiment of the invention, the model building module is used for designing a new model by self, and when a user needs to build a new network model, the visual analysis system can acquire detailed information during model building, such as processing kernel parameters of the model, convolution layers and parameters of the model, and the like, according to the interaction with the user. In order to perform in-depth analysis of the model, our system supports visual analysis of the model structure and case image analysis, the visual analysis system visually displays the model selected by the user according to the forward propagation process of the neural network, and at the same time, the present application presents the deconvolution network generated by the system to the user in the same way. Based on the visualization of the model structure, the selected layer can be further analyzed and the system will display the parametric information for that layer in detail.
Third case image visualization
Optionally, the analysis method further comprises: receiving case selection information, wherein the case selection information is used for indicating a selected case biological image in an image library; importing the case biological image into a target network model, wherein after the target network model analyzes the case biological image, case image information is output, and the case image information at least comprises: whether the case biological images are symmetrical or not, the image symmetry characteristics, the case output images of the deconvolution layer based on the deconvolution network and the image enhancement function; and displaying the case image information on a preset case image visual interface.
In another aspect, the step of presenting the case image information in a case image visualization interface includes: after receiving the case output image, performing decoding processing on the case output image to obtain a first decoded image, wherein the decoding processing mode is to divide the case output image into one-dimensional vectors according to rows and shape each vector into an output image of a multi-dimensional pixel to determine the first decoded image; analyzing the case output image by adopting a preprocessing inner core to obtain a second decoding image; analyzing the information entropy of the first decoded image and the second decoded image in an image entropy mode, wherein the information entropy is used for indicating the average characteristic information quantity in the images; drawing an output line graph of the case output image based on the information entropy; and displaying the output line graph on a case image visualization interface.
After determining the case biological image, importing the case biological image into a network model, and giving a conclusion by the vision analysis system based on the result of the model test: whether the current image is symmetric. The application may also select a corresponding convolutional layer based on the deconvolution network to view an output image of the corresponding deconvolution layer. On the other hand, the system compares the results of the deconvolution network with the input image corresponding to the last convolutional layer of the CNN (for CNNs with processing kernel, it is the image after the processing kernel). By pixel comparison calculations, the analytical model enhances which functions relative to the input image and displays them in the form of a heat map.
For the CNN model with the preprocessing kernel, in order to visualize more clearly, it is also necessary to decode the image passing through the deconvolution network, divide the deconvolution image into one-dimensional vectors by rows, and shape each vector into an image of 16 × 16 pixels, so as to obtain a set of image sets including 180 images, which is called set1, and after the image is processed by the decoding operation, another set of images, which is called set2, is obtained. The two sets of images are then further analyzed using image entropy, which is a statistical form of image features that reflects the amount of information averaged in the image, and the one-dimensional entropy of the image, which represents the amount of information contained in the aggregated features of the image gray scale distribution, can be calculated as follows:
From this formula, the information entropy of each image in set1 and set2 is computed in turn and plotted in the same bar graph according to their respective degrees of rotation, and the system displays the line graph and set1 to the user to assist the user in analyzing the model.
The embodiment of the invention provides an improved network model (such as a CNN model) with a preprocessing kernel, solves the problem of low symmetry recognition accuracy of a biomolecule image, and simultaneously provides an interactive visual model design and analysis system.
Example two
Fig. 2 is a schematic diagram of an alternative image analysis apparatus according to an embodiment of the present invention, as shown in fig. 2, the image analysis apparatus may include: a receiving unit 21, an extracting unit 23, a presenting unit 25, wherein,
a receiving unit 21, configured to receive an image set to be processed, where the image set includes a plurality of biological images to be analyzed;
the extraction unit 23 is configured to extract a target symmetric feature in the biological image by using a target network model, where the target network model carries a preprocessing kernel, and the preprocessing kernel is configured to convert an overall symmetric feature of an original biological image into a region symmetric feature of the target biological image, so that the target network model extracts the target symmetric feature;
and the display unit 25 is used for displaying the target symmetric characteristics of the biological image.
The image analysis device may, when analyzing the symmetric features of the biological image, first receive an image set to be processed through the receiving unit 21, where the image set includes a plurality of biological images to be analyzed, and extract a target symmetric feature in the biological image through the extracting unit 23 by using a target network model, where the target network model carries a preprocessing kernel, and the preprocessing kernel is configured to convert an overall symmetric feature of an original biological image into a regional symmetric feature of the target biological image, so that the target network model extracts the target symmetric feature, and display the target symmetric feature of the biological image through the displaying unit 25. In the embodiment, the integral symmetric feature of the biological image can be converted into the regional symmetric feature of the target biological image by using the preprocessing kernel carried by the model, so that the model can accurately identify the symmetric feature of the biological image, and the technical problem that the symmetric feature of the biological image is difficult to identify in the related technology is solved.
Optionally, the target network model is a model constructed based on the convolutional neural network CNN.
Alternatively, the preprocessing core includes: conversion subassembly and sampling subassembly, the extraction element includes: the first receiving module is used for receiving an original biological image to be analyzed by adopting a conversion component in the preprocessing kernel; the first rotation module is used for rotating each original biological image according to a preset interval angle by adopting a conversion component in the preprocessing kernel; the first calculation module is used for calculating the image difference between the rotated target biological image and the original biological image by adopting a conversion component in the preprocessing kernel to obtain a plurality of images with multi-dimensional pixels with rotation characteristics and outputting the images with the multi-dimensional pixels; the device comprises a first dividing module, a second dividing module and a third dividing module, wherein the first dividing module is used for dividing an image of multi-dimensional pixels into a plurality of pixel areas by adopting a sampling assembly; the second calculation module is used for calculating the average value of all pixels in the pixel areas; the down-sampling processing module is used for carrying out pixel down-sampling processing on the image of the multi-dimensional pixels based on the average value to obtain a target biological image of one-dimensional pixels; the first extraction module is used for extracting target symmetric features in the target biological image of the one-dimensional pixels.
Optionally, the target network model is constructed by an interactive visual analysis system, wherein the interactive visual analysis system includes: the system comprises a network generation and training module, a deconvolution network generation module and a visual analysis module, wherein the network generation and training module is used for setting the structure of a target network model and the parameters of each layer in the target network model and training the target network model; the deconvolution network generation module is used for developing a deconvolution network and generating a deconvolution image of the input biological image; the visual analysis module is used for comparing and analyzing a plurality of network models which are constructed, and analyzing the hierarchical structure parameters of each network model, wherein the hierarchical structure parameters at least comprise: the number of convolution input channels, the number of convolution output channels and the sampling data of the preprocessing kernel.
In an embodiment of the present invention, the image analysis apparatus further includes: the first acquisition module is used for acquiring the accuracy of each network model in the plurality of network models when the symmetrical features of the biological image are analyzed; the first determination module is used for determining rotation parameters of a preprocessing kernel in each network model when the preprocessing kernel analyzes the symmetric features of the biological image; a second determination module for determining model identification data based on the accuracy and the rotation parameters of the preprocessing kernel; the first display module is used for indicating the model identification accuracy by a vertical axis, indicating the rotation parameters of the preprocessing kernel by a horizontal axis and displaying the model identification data of each network model on a preset overview view interface.
Optionally, the image analysis apparatus further includes: the second acquisition module is used for acquiring model construction information; the third determining module is used for determining the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer based on the model construction information; and the second display module is used for displaying the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer on a visual interface of a preset structure.
Another optional, the image analysis apparatus further comprises: the second receiving module is used for receiving case selection information, wherein the case selection information is used for indicating the selected case biological image in the image library; the importing module is used for importing the case biological image into a target network model, wherein after the target network model analyzes the case biological image, case image information is output, and the case image information at least comprises: whether the case biological images are symmetrical or not, the image symmetry characteristics, the case output images of the deconvolution layer based on the deconvolution network and the image enhancement function; and the third display module is used for displaying the case image information on a preset case image visual interface.
Optionally, the third display module includes: the decoding submodule is used for performing decoding processing on the case output image after receiving the case output image to obtain a first decoded image, wherein the decoding processing mode is that the case output image is divided into one-dimensional vectors according to rows, and each vector is shaped into an output image of a multi-dimensional pixel to determine the first decoded image; the first analysis submodule is used for analyzing the case output image by adopting the preprocessing inner core to obtain a second decoding image; the second analysis submodule is used for analyzing the information entropy of the first decoded image and the information entropy of the second decoded image in an image entropy mode, wherein the information entropy is used for indicating the average characteristic information quantity in the images; the display submodule is used for drawing an output line graph of the case output image based on the information entropy; and displaying the output line graph on a case image visualization interface.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions for the processor; wherein the processor is configured to perform the image analysis method of any of the above via execution of executable instructions.
The image analysis device may further include a processor and a memory, the receiving unit 21, the extracting unit 23, the presenting unit 25, and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
The processor comprises a kernel, and the kernel calls a corresponding program unit from the memory. The kernel can be set to be one or more, and target symmetric characteristics of the biological image are shown by adjusting kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed; extracting a target symmetric feature in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the integral symmetric feature of the original biological image into a regional symmetric feature of the target biological image so as to enable the target network model to extract the target symmetric feature; and displaying the target symmetric characteristics of the biological image.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. An image analysis method, comprising:
receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed;
extracting a target symmetric feature in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the integral symmetric feature of the original biological image into a region symmetric feature of the target biological image so as to enable the target network model to extract the target symmetric feature;
and displaying the target symmetric characteristics of the biological image.
2. The analysis method according to claim 1, characterized in that the target network model is a model constructed based on a Convolutional Neural Network (CNN).
3. The analysis method of claim 1, wherein the pre-processing core comprises: the conversion component and the sampling component adopt a target network model to extract target symmetric characteristics in the biological image, and the steps comprise:
receiving an original biological image to be analyzed by adopting a conversion component in the preprocessing kernel; rotating each original biological image according to a preset interval angle by adopting a conversion component in the preprocessing kernel; calculating the image difference between the rotated target biological image and the original biological image by adopting a conversion component in the preprocessing kernel to obtain a plurality of images with multi-dimensional pixels with rotation characteristics, and outputting the images with the multi-dimensional pixels;
dividing the image of the multi-dimensional pixels into a plurality of pixel regions by adopting the sampling assembly; calculating an average value of all pixels in the plurality of pixel regions; based on the average value, performing pixel down-sampling processing on the image of the multi-dimensional pixels to obtain a target biological image of one-dimensional pixels;
and extracting the target symmetric characteristics in the target biological image of the one-dimensional pixels.
4. The analytical method of claim 1, wherein the target network model is constructed by an interactive visual analytics system, wherein the interactive visual analytics system comprises: the system comprises a network generation and training module, a deconvolution network generation module and a visual analysis module, wherein the network generation and training module is used for setting the structure of a target network model and the parameters of each layer in the target network model and training the target network model; the deconvolution network generation module is used for developing a deconvolution network and generating a deconvolution image of the input biological image; the visual analysis module is used for comparing and analyzing a plurality of constructed network models and analyzing the hierarchical structure parameters of each network model, wherein the hierarchical structure parameters at least comprise: the number of convolution input channels, the number of convolution output channels and the sampling data of the preprocessing kernel.
5. The analytical method of claim 4, further comprising:
obtaining the accuracy of each network model in the plurality of network models when analyzing the symmetric characteristics of the biological image;
determining rotation parameters of a preprocessing kernel in each network model when the preprocessing kernel analyzes the symmetric features of the biological image;
determining model identification data based on the accuracy and a rotation parameter of the preprocessing kernel;
and indicating the model identification accuracy by a vertical axis, indicating the rotation parameters of the preprocessing kernel by a horizontal axis, and displaying the model identification data of each network model on a preset overview view interface.
6. The analytical method of claim 4, further comprising:
acquiring model construction information;
determining a model structure of each network model, a kernel parameter of a preprocessing kernel and a layer propagation direction of each model layer based on the model construction information;
and displaying the model structure of each network model, the kernel parameters of the preprocessing kernel and the layer propagation direction of each model layer on a visual interface of a preset structure.
7. The analytical method of claim 4, further comprising:
receiving case selection information, wherein the case selection information is used for indicating a selected case biological image in an image library;
importing the case biological image into a target network model, wherein after the target network model analyzes the case biological image, case image information is output, and the case image information at least comprises: whether the case biological images are symmetrical or not, the image symmetry characteristics, the case output images of the deconvolution layer based on the deconvolution network and the image enhancement function;
and displaying the case image information on a preset case image visual interface.
8. The method of claim 7, wherein the step of presenting case image information in the case image visualization interface comprises:
after receiving a case output image, performing decoding processing on the case output image to obtain a first decoded image, wherein the decoding processing mode is to divide the case output image into one-dimensional vectors according to rows and shape each vector into an output image of a multi-dimensional pixel to determine the first decoded image;
analyzing the case output image by adopting a preprocessing kernel to obtain a second decoding image;
analyzing information entropies of the first decoded image and the second decoded image in an image entropy mode, wherein the information entropies are used for indicating average characteristic information quantity in the images;
drawing an output line graph of the case output image based on the information entropy;
and displaying the output line graph on the case image visualization interface.
9. An image analysis apparatus, comprising:
the receiving unit is used for receiving an image set to be processed, wherein the image set comprises a plurality of biological images to be analyzed;
the extraction unit is used for extracting target symmetric features in the biological image by adopting a target network model, wherein the target network model carries a preprocessing kernel, and the preprocessing kernel is used for converting the overall symmetric features of an original biological image into regional symmetric features of a target biological image so as to enable the target network model to extract the target symmetric features;
and the display unit is used for displaying the target symmetric characteristics of the biological image.
10. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the image analysis method of any of claims 1 to 8 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010641901.4A CN111797842B (en) | 2020-07-06 | 2020-07-06 | Image analysis method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010641901.4A CN111797842B (en) | 2020-07-06 | 2020-07-06 | Image analysis method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797842A true CN111797842A (en) | 2020-10-20 |
| CN111797842B CN111797842B (en) | 2023-04-07 |
Family
ID=72811289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010641901.4A Active CN111797842B (en) | 2020-07-06 | 2020-07-06 | Image analysis method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797842B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6038334A (en) * | 1997-02-21 | 2000-03-14 | Dew Engineering And Development Limited | Method of gathering biometric information |
| CN105488463A (en) * | 2015-11-25 | 2016-04-13 | 康佳集团股份有限公司 | Lineal relationship recognizing method and system based on face biological features |
| US20160232425A1 (en) * | 2013-11-06 | 2016-08-11 | Lehigh University | Diagnostic system and method for biological tissue analysis |
| CN107545248A (en) * | 2017-08-24 | 2018-01-05 | 北京小米移动软件有限公司 | Biological characteristic biopsy method, device, equipment and storage medium |
| CN107862267A (en) * | 2017-10-31 | 2018-03-30 | 天津科技大学 | Face recognition features' extraction algorithm based on full symmetric local weber description |
| CN110866893A (en) * | 2019-09-30 | 2020-03-06 | 中国科学院计算技术研究所 | Pathological image-based TMB classification method and system and TMB analysis device |
-
2020
- 2020-07-06 CN CN202010641901.4A patent/CN111797842B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6038334A (en) * | 1997-02-21 | 2000-03-14 | Dew Engineering And Development Limited | Method of gathering biometric information |
| US20160232425A1 (en) * | 2013-11-06 | 2016-08-11 | Lehigh University | Diagnostic system and method for biological tissue analysis |
| CN105488463A (en) * | 2015-11-25 | 2016-04-13 | 康佳集团股份有限公司 | Lineal relationship recognizing method and system based on face biological features |
| CN107545248A (en) * | 2017-08-24 | 2018-01-05 | 北京小米移动软件有限公司 | Biological characteristic biopsy method, device, equipment and storage medium |
| CN107862267A (en) * | 2017-10-31 | 2018-03-30 | 天津科技大学 | Face recognition features' extraction algorithm based on full symmetric local weber description |
| CN110866893A (en) * | 2019-09-30 | 2020-03-06 | 中国科学院计算技术研究所 | Pathological image-based TMB classification method and system and TMB analysis device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797842B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jia et al. | Image transformation based on learning dictionaries across image spaces | |
| Bernard et al. | A survey and task-based quality assessment of static 2D colormaps | |
| CA3066029A1 (en) | Image feature acquisition | |
| JP2014029732A (en) | Method for generating representation of image contents using image search and retrieval criteria | |
| US9171226B2 (en) | Image matching using subspace-based discrete transform encoded local binary patterns | |
| CN108961180B (en) | Infrared image enhancement method and system | |
| Jiang et al. | Learning sparse representation for objective image retargeting quality assessment | |
| CN107590505B (en) | Learning method combining low-rank representation and sparse regression | |
| US20130301910A1 (en) | Extracting object edges from images | |
| Abouelaziz et al. | Blind 3D mesh visual quality assessment using support vector regression | |
| CN114743009B (en) | Hyperspectral image band selection method and system and electronic equipment | |
| CN117333409A (en) | Big data analysis method based on image | |
| Venkatachalam et al. | An efficient Gabor Walsh-Hadamard transform based approach for retrieving brain tumor images from MRI | |
| Mancas | Relative influence of bottom-up and top-down attention | |
| Belabbas et al. | On landmark selection and sampling in high-dimensional data analysis | |
| Fang et al. | Learning explicit smoothing kernels for joint image filtering | |
| CN111797842B (en) | Image analysis method and device and electronic equipment | |
| Dang et al. | Single image super resolution via manifold linear approximation using sparse subspace clustering | |
| Versteegen et al. | Texture modelling with nested high-order Markov–Gibbs random fields | |
| Malo et al. | Geometrical and statistical properties of vision models obtained via maximum differentiation | |
| Denitto et al. | Multiple structure recovery via probabilistic biclustering | |
| CN110929731A (en) | Medical image processing method and device based on pathfinder intelligent search algorithm | |
| JP4434868B2 (en) | Image segmentation system | |
| CN113208641B (en) | Auxiliary diagnosis method for lung nodule based on three-dimensional multi-resolution attention capsule network | |
| Chehdi et al. | Stable and unsupervised fuzzy C-means method and its validation in the context of multicomponent images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |