CN111178445A - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN111178445A CN111178445A CN201911416999.7A CN201911416999A CN111178445A CN 111178445 A CN111178445 A CN 111178445A CN 201911416999 A CN201911416999 A CN 201911416999A CN 111178445 A CN111178445 A CN 111178445A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- images
- neural network
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000003062 neural network model Methods 0.000 claims abstract description 101
- 238000012549 training Methods 0.000 claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000012216 screening Methods 0.000 claims abstract description 12
- 238000011156 evaluation Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 19
- 238000001914 filtration Methods 0.000 claims description 7
- 238000003709 image segmentation Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 18
- 238000002372 labelling Methods 0.000 abstract description 5
- 230000007547 defect Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Abstract
The invention provides an image processing method and device, which are used for segmenting an original image to obtain a larger number of subimages, screening target subimages which are more matched with a neural network model application scene based on the image types of the subimages, and training a neural network model by using the obtained target subimages, so that the defects of less training samples and poor pertinence of the training samples can be overcome, and the detection precision of the trained neural network model can be effectively improved. In addition, when the image type of each sub-image is determined, the method does not adopt manual labeling, but adopts an initially trained neural network model to determine, so that the human resources can be effectively saved, and the efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of image processing, and in particular, to an image processing method and apparatus.
Background
The neural network model is more and more widely applied, and the efficiency is higher and unnecessary human resources are saved by classifying the images by using the neural network model.
Before using the neural network model, the neural network model needs to be trained first, and the model training is generally performed by using the image sample set and the real image type of each image in the image sample set. However, when the neural network model is trained, the number of images in the image sample set is small, the image pertinence is poor, and the like, so that the trained neural network model has low detection accuracy.
Disclosure of Invention
In view of the above, the present disclosure provides at least an image processing method and an image processing apparatus.
In a first aspect, the present disclosure provides an image processing method, including:
respectively segmenting each original image in a plurality of original images to obtain a plurality of sub-images;
inputting each subimage into an initially trained neural network model to obtain the image type of each subimage;
and selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of the adjacent sub-images of the sub-image.
According to the method, the original image is segmented to obtain a larger number of sub-images, the target sub-images which are more matched with the application scene of the neural network model are screened based on the image types of the sub-images, and the neural network model is trained based on the target sub-images. In addition, when the image type of each sub-image is determined, the method does not adopt manual labeling, but adopts an initially trained neural network model to determine, so that the human resources can be effectively saved, and the efficiency is improved.
In one possible embodiment, selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of the adjacent sub-images of the sub-image comprises:
for each sub-image, determining the number of the adjacent sub-images of which the corresponding image types are the same as the image types of the sub-images in the adjacent sub-images of the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined number of adjacent sub-images.
According to the embodiment, the sub-image with the more accurate image type can be used as the target sub-image based on the number of the adjacent sub-images with the same image type as the current sub-image, and the rest redundant sub-images are deleted, so that the number of samples is increased, the pertinence of the samples is improved, and the detection precision of the trained neural network model can be effectively improved.
In a possible embodiment, the selecting a target sub-image from the plurality of sub-images based on the determined number of the neighboring sub-images includes:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
In the above embodiment, different confidence evaluation thresholds are set for different image types. By using the confidence evaluation threshold matched with the image type of the current sub-image, the confidence with higher accuracy can be obtained, and the sub-image with more accurate image type can never be used as the target sub-image.
In a possible implementation, the image processing method further includes:
determining an expansion sub-image corresponding to each target sub-image;
and retraining the neural network model by using each expansion subimage.
In a possible embodiment, determining the expansion sub-image corresponding to each target sub-image includes:
and for each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
In the above embodiment, after the target sub-images are obtained by screening, the size of the target sub-images is expanded by taking the target sub-images as a center, so that a plurality of expanded sub-images with different sizes are obtained, the number of samples for training the neural network model is increased, and training samples with larger sizes and more diversified sizes, namely expanded sub-images, can be obtained.
In one possible embodiment, the retraining the neural network model with the respective expansion subimages includes:
retraining the neural network model based on the extended subimages and the image types of the target subimages obtained through the initially trained neural network model.
In the embodiment, the neural network model is retrained by utilizing a plurality of extended subimages with different sizes, so that the number of samples is increased, and training samples with larger sizes and more diversified sizes, namely the extended subimages, can be obtained.
In one possible embodiment, the image processing method further includes a step of training the initially trained neural network model, including:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image;
and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
In the above embodiment, the initially trained neural network model is trained using an image set including a plurality of sample images, and the model is subsequently used to determine the image type of each sub-image, so that human resources can be effectively saved, and efficiency can be improved.
In a possible implementation manner, the segmenting each of the original images to obtain a plurality of sub-images includes:
sequentially intercepting image areas on each original image by using an image intercepting frame with a first preset size to obtain a plurality of sub-images;
after a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
In the above embodiment, a plurality of sub-images can be captured from the original image based on the first preset size and the second preset size, so that the number of samples for training the neural network model can be increased, and the detection accuracy of the trained neural network model can be improved.
In a second aspect, the present disclosure provides an image processing apparatus comprising:
the image segmentation module is used for segmenting each original image in a plurality of original images to obtain a plurality of sub-images;
the image type determining module is used for inputting each subimage into an initially trained neural network model to obtain the image type of each subimage;
and the image screening module is used for selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image type of the adjacent sub-image of the sub-image.
In a possible embodiment, the image filtering module, when selecting the target sub-image from the plurality of sub-images, is configured to:
for each sub-image, determining the number of the adjacent sub-images of which the corresponding image types are the same as the image types of the sub-images in the adjacent sub-images of the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined number of adjacent sub-images.
In a possible embodiment, the image filtering module, when selecting the target sub-image from the plurality of sub-images based on the determined number of the neighboring sub-images, is configured to:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
In a possible implementation, the foregoing further includes:
and the model training module is used for determining an extended sub-image corresponding to each target sub-image and retraining the neural network model by using each extended sub-image.
In a possible embodiment, the model training module, when determining the extended sub-image corresponding to each target sub-image, is configured to:
and for each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
In a possible embodiment, the model training module, when retraining the neural network model with each expansion subimage, is configured to:
retraining the neural network model based on the extended subimages and the image types of the target subimages obtained through the initially trained neural network model.
In one possible embodiment, the model training module is further configured to train the initially trained neural network model by:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image;
and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
In a possible embodiment, the image segmentation module, when segmenting each of a plurality of original images to obtain a plurality of sub-images, is configured to:
sequentially intercepting image areas on each original image by using an image intercepting frame with a first preset size to obtain a plurality of sub-images;
after a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
In a third aspect, the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the image processing method as described above.
In a fourth aspect, the present disclosure also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the image processing method as described above.
The above-mentioned apparatus, electronic device, and computer-readable storage medium of the present disclosure at least include technical features substantially the same as or similar to technical features of any aspect or any implementation manner of any aspect of the above-mentioned method of the present disclosure, and therefore, for the description of the effects of the above-mentioned apparatus, electronic device, and computer-readable storage medium, reference may be made to the description of the effects of the above-mentioned method contents, which is not repeated herein.
Drawings
To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present disclosure and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings may be obtained from the drawings without inventive effort.
Fig. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure;
fig. 2 is a flowchart illustrating selecting a target sub-image from the plurality of sub-images in another image processing method provided by the embodiment of the disclosure;
FIG. 3 is a flow chart illustrating a retraining of the neural network model based on a selected target sub-image in yet another image processing method provided by an embodiment of the present disclosure;
FIG. 4 illustrates a sub-image expansion diagram in an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an image processing apparatus provided in an embodiment of the present disclosure;
fig. 6 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it should be understood that the drawings in the present disclosure are for illustrative and descriptive purposes only and are not used to limit the scope of the present disclosure. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this disclosure illustrate operations implemented according to some embodiments of the present disclosure. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. In addition, one skilled in the art, under the direction of the present disclosure, may add one or more other operations to the flowchart, and may remove one or more operations from the flowchart.
In addition, the described embodiments are only a few embodiments of the present disclosure, not all embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
To enable one of ordinary skill in the art to use the present disclosure, the following embodiments are given in conjunction with the application-specific scenario "detecting fracture images using neural network models". It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications requiring image classification without departing from the spirit and scope of the present disclosure. Although the present disclosure is primarily described in the context of detecting a fracture image using a neural network model, it should be understood that this is but one exemplary embodiment.
It is to be noted that the term "comprising" will be used in the disclosed embodiments to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
The disclosure provides an image processing method and device for improving the detection accuracy of a neural network model when detecting images of types such as cracks, wherein an original image is firstly segmented to obtain a larger number of sub-images, and then a target sub-image which is more matched with the application scene of the neural network model is screened based on the image type of the sub-images. The obtained target subimages are used for training the neural network model, so that the defects of few training samples and poor pertinence of the training samples can be overcome, and the detection precision of the neural network model obtained through training can be effectively improved. In addition, when the image type of each sub-image is determined, the method does not adopt manual labeling, but adopts an initially trained neural network model to determine, so that the human resources can be effectively saved, and the efficiency is improved.
As shown in fig. 1, an embodiment of the present disclosure provides an image processing method, which is applied to a terminal device capable of performing image processing and model training, and specifically, the method may include the following steps:
s110, segmenting each original image in the plurality of original images respectively to obtain a plurality of sub-images.
Here, the original image is an image with a relatively large size, and a plurality of sub-images can be obtained by dividing the original image, and in a specific implementation, the division may be performed according to a preset size, for example, the size of the original image is 1024 pixels × 1024 pixels, and the size of the sub-image obtained by the division is 16 pixels × 16 pixels.
The original image is divided into a plurality of sub-images, so that the number of samples for training the neural network model can be increased, and the detection precision of the neural network model can be improved.
In a specific application scenario of performing the crack image detection, the neural network model is a neural network model for determining whether an image is a crack image. In order to improve the quality of the sample and the detection accuracy of the trained neural network model, the original image may be an image containing a crack. Before this step is performed, a plurality of original images containing the crack need to be acquired first.
Some of the segmented sub-images include crack content, some do not include crack content, the sub-image including crack content may be referred to as a crack bin, and the image not including crack content may be referred to as a background bin.
And S120, inputting each sub-image into the initially trained neural network model to obtain the image type of each sub-image.
Before this step is performed, it is first necessary to train the initially trained neural network model or obtain the initially trained neural network model.
Here, the initially trained neural network model can be used to perform image processing on each input sub-image, resulting in an image type for each sub-image. In a specific application scenario for performing crack image detection, the image types may include a crack type and a background type, and then the sub-image includes a sub-image of the crack type (i.e., the crack bin described above) and a sub-image of the background type (i.e., the background bin described above).
The image type of each sub-image is determined by using the initially trained neural network model, so that the link of manual labeling is replaced, the human resources are saved, and the efficiency is improved.
S130, selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image type of the adjacent sub-image of the sub-image.
Before this step is performed, it is preferred to determine the neighboring sub-images of each sub-image. Here, the adjacent sub-images of each sub-image may include all adjacent sub-images of the sub-image, or only some adjacent sub-images of the sub-image may be selected, for example, only the upper adjacent and the right adjacent sub-images of the sub-image are selected, which is not limited by the disclosure.
Here, based on the image type of the sub-image and the image type of the adjacent sub-image of the sub-image, the accuracy or confidence of the image type determined by the initially trained neural network model can be determined, the sub-image with the more accurate image type, namely the target sub-image, can be obtained by screening based on the accuracy or confidence, the neural network model is further trained by using the sub-image with the more accurate image type, and the detection precision of the neural network model can be improved.
After the plurality of target sub-images are obtained by screening, the neural network model may be further trained based on the obtained target sub-images and the image type of each target sub-image determined by the initially trained neural network model.
In the specific training process, firstly, the extended sub-image corresponding to each target sub-image needs to be determined, and then, the neural network model is retrained by using each extended sub-image and the image type of each target sub-image obtained by the initially trained neural network model.
In addition, in the specific training process, iterative training is carried out based on the detection precision of the neural network model until the detection precision of the neural network model meets the precision requirement. For example, in a specific application scenario of performing crack image detection, it cannot be determined that the neural network model is trained until the accuracy of the detected crack image of the neural network model can meet a preset accuracy requirement.
According to the embodiment, the original image is segmented to obtain a larger number of sub-images, the target sub-images which are more matched with the application scene of the neural network model are screened based on the image types of the sub-images, and the neural network model is trained based on the target sub-images. In addition, when the image type of each sub-image is determined, the method does not adopt manual labeling, but adopts an initially trained neural network model to determine, so that the human resources can be effectively saved, and the efficiency is improved.
In some embodiments, as shown in fig. 2, the selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image type of the adjacent sub-image of the sub-image may specifically be implemented by:
s210, aiming at each sub-image, determining the number of the adjacent sub-images of which the corresponding image types are the same as the image types of the sub-image.
In practical applications, more or less of the neighboring sub-images of the sub-image should have the same image type as the sub-image, and therefore, the accuracy of determining the image type of the sub-image may be based on the number of neighboring sub-images of the sub-image, which have the same image type as the sub-image.
For example, for a sub-image of a crack type, the accuracy or confidence of the image type of the sub-image may be determined based on the number of sub-images of the crack type in neighboring sub-images of the sub-image. For a sub-image of a background type, the accuracy or confidence of the image type of the sub-image may be determined based on the number of sub-images of the background type in neighboring sub-images of the sub-image.
S220, selecting a target sub-image from the plurality of sub-images based on the determined number of the adjacent sub-images.
Specifically, a sub-image with a more accurate image type is selected from the plurality of sub-images as a target sub-image based on the determined number of the adjacent sub-images.
According to the embodiment, the sub-image with the more accurate image type can be used as the target sub-image based on the number of the adjacent sub-images with the same image type as the current sub-image, and the rest redundant sub-images are deleted, so that the number of samples is increased, the pertinence of the samples is improved, and the detection precision of the trained neural network model can be effectively improved.
In a specific implementation, the step 220 may be implemented by the following steps:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image; selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
The proportion of the sub-images of each image type in all the sub-images obtained by segmentation is different, and the distribution rules of the sub-images of different image types have different characteristics, so that the confidence evaluation thresholds corresponding to the sub-images of different image types are different. For example, of the neighboring sub-images of the crack type, the number of sub-images that also belong to the crack type is smaller; in the adjacent sub-images of the background type, the number of sub-images also belonging to the background type is large, so that when each sub-image has 8 adjacent sub-images, the confidence evaluation threshold value matched with the crack type may be 2, and the confidence evaluation threshold value matched with the background type may be 8.
Before the above steps are performed, it is first required to obtain a confidence evaluation threshold corresponding to each image type.
After the confidence evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image are obtained, the number is calculated and divided by the confidence evaluation threshold value, and the obtained quotient is the confidence.
And after the confidence coefficient is obtained, judging whether the confidence coefficient is larger than or equal to an image screening threshold value, and taking the sub-image larger than or equal to the image screening threshold value as a target sub-image. For example, when each sub-image has 8 adjacent sub-images, the confidence evaluation threshold value matched with the crack type is set to 2, and the confidence evaluation threshold value matched with the background type is set to 8, the image screening threshold value may be set to 1, which means that for a sub-image of a crack type, more than two sub-images of crack types need to be present in the adjacent sub-image to use the sub-image as a target sub-image, and for a sub-image of a background type, eight sub-images of background types need to be present in the adjacent sub-image to use the sub-image as a target sub-image.
The confidence evaluation threshold may be flexibly set based on the actual application scenario, for example, when each sub-image has 8 adjacent sub-images, the confidence evaluation threshold matching with the crack type may be set to 4, the confidence evaluation threshold matching with the background type may be set to 16, and the image screening threshold may be set to 0.5. The setting values also indicate that for a crack type sub-image, more than two crack type sub-images are required in the adjacent sub-image to be used as a target sub-image, and for a background type sub-image, eight background type sub-images are required in the adjacent sub-image to be used as a target sub-image.
It should be noted that, after the number of adjacent sub-images of the same image type as the sub-image is determined in step 210, the confidence level may not be determined, and the target sub-image is selected from the plurality of sub-images based on the determined number. Specifically, the same-type neighboring sub-image quantity threshold is set for each image type in advance, and then the sub-images of which the quantity is greater than or equal to the corresponding same-type neighboring sub-image quantity threshold are used as target sub-images.
For example, when each sub-image has 8 adjacent sub-images, the threshold for the number of adjacent sub-images of the same type matching the crack type is set to 2, and the threshold for the number of adjacent sub-images of the same type matching the background type is set to 8. At this time, for a sub-image of a crack type, two or more sub-images of the crack type are required in adjacent sub-images to be able to use the sub-image as a target sub-image, and for a sub-image of a background type, eight sub-images of the background type are required in adjacent sub-images to be able to use the sub-image as a target sub-image.
In some embodiments, as shown in fig. 3, the above-mentioned retraining of the neural network model may be implemented by the following steps:
s310, aiming at each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
Before this step is performed, the sizes of a plurality of images as training samples need to be preset. After the target sub-images are obtained through screening, the target sub-images are expanded according to a plurality of preset sizes, and each target can obtain a plurality of expanded sub-images. The number of images used for training is increased, the size of the images used for training is increased, and the defect of low detection precision of the neural network model caused by small size of the images used for training can be overcome. For example, a neural network model trained with only small-sized images cannot detect images with large cracks.
In specific implementation, the preset size may be flexibly set based on an actual application scene, for example, the preset size may be set to 48 pixels × 48 pixels, 36 pixels × 36 pixels, or the like.
And S320, retraining the neural network model based on the intercepted extended subimages.
Fig. 4 shows a schematic diagram of the propagation of a crack bin, i.e. a sub-image of the crack type, and a background bin, i.e. a sub-image of the background type.
In the above embodiment, after the target sub-images are obtained by screening, size expansion is performed with the target sub-images as the center, so that a plurality of expansion sub-images with different sizes are obtained, the number of samples for training the neural network model is increased, and training samples with larger sizes and more diversified sizes, that is, expansion sub-images, can be obtained.
In some embodiments, the above image processing method may further include the step of training the initially trained neural network model:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image; and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
The image type label may be manually labeled. The initially trained neural network model is trained by using an image set comprising a plurality of sample images, and the model is subsequently used for determining the image type of each sub-image, so that the human resources can be effectively saved, and the efficiency is improved.
In some embodiments, the segmenting of each of the plurality of original images to obtain the plurality of sub-images may be implemented by using the following steps:
and sequentially intercepting the image areas on each original image by using an image intercepting frame with a first preset size aiming at each original image to obtain a plurality of sub-images.
After a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
The image capturing mode is a sliding capturing mode, and each time the image capturing frame moves by a second preset size, a sub-image is captured from the original image.
The preset direction may include left to right, top to bottom. The second predetermined size may be set to be equal to the first predetermined size or slightly smaller than the first predetermined size, which is not preferable to be set to be small, because if the second predetermined size is small, the adjacent sub-images obtained by capturing have many overlapped pixels, which causes image redundancy.
In the above embodiment, a plurality of sub-images can be captured from the original image based on the first preset size and the second preset size, so that the number of samples for training the neural network model can be increased, and the detection accuracy of the trained neural network model can be improved.
In practical application, the sub-image may also be intercepted without adopting a sliding manner, and the original image may be directly divided into a plurality of sub-images according to the first preset size.
In the above embodiment, the obtained sub-image has a first preset size, and when determining the first preset size, the size of the image used for training the initially trained neural network model may be first obtained, and the first preset size may be set as the size. This is because the initially trained neural network model has higher detection accuracy for an image having the same size as the image used for training the model, and in order to improve the accuracy of the image type of the determined sub-image, the original image is divided into a plurality of sub-images having a first preset size.
Corresponding to the image processing method, the embodiment of the present disclosure further provides an image processing apparatus, which is applied to a terminal device capable of performing image processing and neural network model training, and the apparatus and each module thereof can perform the same method steps as the image processing method, and can achieve the same or similar beneficial effects, so that repeated parts are not described again.
As shown in fig. 5, the present disclosure provides an image processing apparatus including:
the image segmentation module 510 is configured to segment each of the plurality of original images to obtain a plurality of sub-images.
And an image type determining module 520, configured to input each sub-image into the initially trained neural network model, so as to obtain an image type of each sub-image.
An image filtering module 530, configured to select a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of adjacent sub-images of the sub-image.
In some embodiments, the image filtering module 530, when selecting the target sub-image from the plurality of sub-images, is configured to:
and determining the number of adjacent sub-images of which the corresponding image types are the same as the image types of the sub-images in the adjacent sub-images of each sub-image.
Selecting a target sub-image from the plurality of sub-images based on the determined number of adjacent sub-images.
In some embodiments, the image filtering module 530, when selecting a target sub-image from the plurality of sub-images based on the determined number of neighboring sub-images, is configured to:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
In some embodiments, the above apparatus further comprises:
and the model training module 540 is configured to determine an extended sub-image corresponding to each target sub-image, and retrain the neural network model using each extended sub-image.
In some embodiments, the model training module 540, when determining the extended sub-image corresponding to each of the target sub-images, is configured to:
and for each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
In some embodiments, the model training module 540, when retraining the neural network model with each of the expansion subimages, is configured to:
retraining the neural network model based on the extended subimages and the image types of the target subimages obtained through the initially trained neural network model.
In some embodiments, the model training module 540 is further configured to train the initially trained neural network model to:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image;
and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
In some embodiments, the image segmentation module 510, when segmenting each of the original images into a plurality of sub-images, is configured to:
sequentially intercepting image areas on each original image by using an image intercepting frame with a first preset size to obtain a plurality of sub-images;
after a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
An embodiment of the present disclosure discloses an electronic device, as shown in fig. 6, including: a processor 601, a memory 602, and a bus 603, wherein the memory 602 stores machine-readable instructions executable by the processor 601, and when the electronic device is operated, the processor 601 and the memory 602 communicate via the bus 603.
The machine readable instructions, when executed by the processor 601, perform the steps of the image processing method of:
respectively segmenting each original image in a plurality of original images to obtain a plurality of sub-images;
inputting each subimage into an initially trained neural network model to obtain the image type of each subimage;
and selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of the adjacent sub-images of the sub-image.
In addition, when the processor 601 executes the machine readable instructions, the method contents in any embodiment described in the above method part can be executed, which is not described herein again.
A computer program product corresponding to the method and the apparatus provided in the embodiments of the present disclosure includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method in the foregoing method embodiments, and specific implementation may refer to the method embodiments, which is not described herein again.
The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to one another, which are not repeated herein for brevity.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this disclosure. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above are only specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present disclosure, and shall be covered by the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (18)
1. An image processing method, comprising:
respectively segmenting each original image in a plurality of original images to obtain a plurality of sub-images;
inputting each subimage into an initially trained neural network model to obtain the image type of each subimage;
and selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of the adjacent sub-images of the sub-image.
2. The image processing method of claim 1, wherein selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image types of adjacent sub-images of the sub-image comprises:
for each sub-image, determining the number of the adjacent sub-images of which the corresponding image types are the same as the image types of the sub-images in the adjacent sub-images of the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined number of adjacent sub-images.
3. The method of claim 2, wherein selecting a target sub-image from the plurality of sub-images based on the determined number of neighboring sub-images comprises:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
4. The image processing method according to claim 1, further comprising:
determining an expansion sub-image corresponding to each target sub-image;
and retraining the neural network model by using each expansion subimage.
5. The image processing method according to claim 4, wherein the determining the expansion sub-image corresponding to each target sub-image comprises:
and for each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
6. The image processing method of claim 4, wherein said retraining said neural network model with each said expansion subimage comprises:
retraining the neural network model based on the extended subimages and the image types of the target subimages obtained through the initially trained neural network model.
7. The method of image processing according to claim 1, further comprising the step of training the initially trained neural network model, comprising:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image;
and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
8. The method according to claim 1, wherein the segmenting each of the plurality of original images into a plurality of sub-images comprises:
sequentially intercepting image areas on each original image by using an image intercepting frame with a first preset size to obtain a plurality of sub-images;
after a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
9. An image processing apparatus characterized by comprising:
the image segmentation module is used for segmenting each original image in a plurality of original images to obtain a plurality of sub-images;
the image type determining module is used for inputting each subimage into an initially trained neural network model to obtain the image type of each subimage;
and the image screening module is used for selecting a target sub-image from the plurality of sub-images based on the image type of each sub-image and the image type of the adjacent sub-image of the sub-image.
10. The image processing apparatus of claim 9, wherein the image filtering module, when selecting the target sub-image from the plurality of sub-images, is configured to:
for each sub-image, determining the number of the adjacent sub-images of which the corresponding image types are the same as the image types of the sub-images in the adjacent sub-images of the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined number of adjacent sub-images.
11. The image processing apparatus of claim 10, wherein the image filtering module, when selecting the target sub-image from the plurality of sub-images based on the determined number of neighboring sub-images, is configured to:
for each sub-image, determining the confidence coefficient of the image type of the sub-image based on the confidence coefficient evaluation threshold value matched with the image type of the sub-image and the number corresponding to the sub-image;
selecting a target sub-image from the plurality of sub-images based on the determined confidence level.
12. The image processing apparatus according to claim 9, further comprising:
and the model training module is used for determining an extended sub-image corresponding to each target sub-image and retraining the neural network model by using each extended sub-image.
13. The image processing apparatus according to claim 12, wherein the model training module, when determining the extended sub-image corresponding to each target sub-image, is configured to:
and for each target sub-image, in the original image corresponding to the target sub-image, taking the target sub-image as the center, and intercepting a plurality of expansion sub-images containing the image content of the target sub-image according to a plurality of preset sizes.
14. The image processing apparatus of claim 12, wherein the model training module, when retraining the neural network model with each of the expanded sub-images, is configured to:
retraining the neural network model based on the extended subimages and the image types of the target subimages obtained through the initially trained neural network model.
15. The image processing apparatus of claim 12, wherein the model training module is further configured to train the initially trained neural network model to:
acquiring an image set comprising a plurality of sample images and an image type label of each sample image;
and training to obtain the initially trained neural network model for identifying the image type by using the image set and the image type label of each sample image.
16. The image processing apparatus according to claim 9, wherein the image segmentation module, when segmenting each of the plurality of original images into a plurality of sub-images, is configured to:
sequentially intercepting image areas on each original image by using an image intercepting frame with a first preset size to obtain a plurality of sub-images;
after a sub-image is intercepted, moving the image intercepting frame on the original image according to a preset direction and a second preset size, and then intercepting an adjacent sub-image of the sub-image; the second preset size is smaller than or equal to the first preset size.
17. An electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to execute the image processing method according to any one of claims 1 to 8.
18. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the image processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911416999.7A CN111178445A (en) | 2019-12-31 | 2019-12-31 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911416999.7A CN111178445A (en) | 2019-12-31 | 2019-12-31 | Image processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111178445A true CN111178445A (en) | 2020-05-19 |
Family
ID=70656007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911416999.7A Withdrawn CN111178445A (en) | 2019-12-31 | 2019-12-31 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111178445A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681224A (en) * | 2020-06-09 | 2020-09-18 | 上海联影医疗科技有限公司 | Method and device for acquiring blood vessel parameters |
| CN113724214A (en) * | 2021-08-23 | 2021-11-30 | 唯智医疗科技(佛山)有限公司 | Image processing method and device based on neural network |
| CN113807132A (en) * | 2020-06-12 | 2021-12-17 | 广州极飞科技股份有限公司 | Method and device for identifying irrigation state of plant growing area and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101526994A (en) * | 2009-04-03 | 2009-09-09 | 山东大学 | Fingerprint image segmentation method irrelevant to collecting device |
| CN101795374A (en) * | 2009-11-23 | 2010-08-04 | 清投视讯(北京)科技有限公司 | Method and system for image display of true seamless projection type large screen |
| CN104517116A (en) * | 2013-09-30 | 2015-04-15 | 北京三星通信技术研究有限公司 | Device and method for confirming object region in image |
| CN105069480A (en) * | 2015-08-19 | 2015-11-18 | 西安电子科技大学 | Polarized SAR terrain classification method based on Gauss filtering and PSO |
| CN105957063A (en) * | 2016-04-22 | 2016-09-21 | 北京理工大学 | CT image liver segmentation method and system based on multi-scale weighting similarity measure |
| CN107358262A (en) * | 2017-07-13 | 2017-11-17 | 京东方科技集团股份有限公司 | The sorting technique and sorter of a kind of high-definition picture |
| CN108229514A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, device and electronic equipment |
| CN108830221A (en) * | 2018-06-15 | 2018-11-16 | 北京市商汤科技开发有限公司 | The target object segmentation of image and training method and device, equipment, medium, product |
| CN109427063A (en) * | 2017-08-30 | 2019-03-05 | 广州极飞科技有限公司 | The storage method and device of image data |
| CN109583489A (en) * | 2018-11-22 | 2019-04-05 | 中国科学院自动化研究所 | Defect classifying identification method, device, computer equipment and storage medium |
| CN109671058A (en) * | 2018-12-05 | 2019-04-23 | 武汉精立电子技术有限公司 | A kind of defect inspection method and system of big image in different resolution |
| CN110047078A (en) * | 2019-04-18 | 2019-07-23 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| US20190354811A1 (en) * | 2017-12-07 | 2019-11-21 | Shanghai Cambricon Information Technology Co., Ltd | Image compression method and related device |
| CN110569854A (en) * | 2019-09-12 | 2019-12-13 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
-
2019
- 2019-12-31 CN CN201911416999.7A patent/CN111178445A/en not_active Withdrawn
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101526994A (en) * | 2009-04-03 | 2009-09-09 | 山东大学 | Fingerprint image segmentation method irrelevant to collecting device |
| CN101795374A (en) * | 2009-11-23 | 2010-08-04 | 清投视讯(北京)科技有限公司 | Method and system for image display of true seamless projection type large screen |
| CN104517116A (en) * | 2013-09-30 | 2015-04-15 | 北京三星通信技术研究有限公司 | Device and method for confirming object region in image |
| CN105069480A (en) * | 2015-08-19 | 2015-11-18 | 西安电子科技大学 | Polarized SAR terrain classification method based on Gauss filtering and PSO |
| CN105957063A (en) * | 2016-04-22 | 2016-09-21 | 北京理工大学 | CT image liver segmentation method and system based on multi-scale weighting similarity measure |
| CN108229514A (en) * | 2016-12-29 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, device and electronic equipment |
| CN107358262A (en) * | 2017-07-13 | 2017-11-17 | 京东方科技集团股份有限公司 | The sorting technique and sorter of a kind of high-definition picture |
| CN109427063A (en) * | 2017-08-30 | 2019-03-05 | 广州极飞科技有限公司 | The storage method and device of image data |
| US20190354811A1 (en) * | 2017-12-07 | 2019-11-21 | Shanghai Cambricon Information Technology Co., Ltd | Image compression method and related device |
| CN108830221A (en) * | 2018-06-15 | 2018-11-16 | 北京市商汤科技开发有限公司 | The target object segmentation of image and training method and device, equipment, medium, product |
| CN109583489A (en) * | 2018-11-22 | 2019-04-05 | 中国科学院自动化研究所 | Defect classifying identification method, device, computer equipment and storage medium |
| CN109671058A (en) * | 2018-12-05 | 2019-04-23 | 武汉精立电子技术有限公司 | A kind of defect inspection method and system of big image in different resolution |
| CN110047078A (en) * | 2019-04-18 | 2019-07-23 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110569854A (en) * | 2019-09-12 | 2019-12-13 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| WENYI YU等: "Superpixel-Based CFAR Target Detection for High-Resolution SAR Images" * |
| YANTAOWEI等: "Multiscale patch-basedcontrastmeasureforsmallinfrared targetdetection" * |
| 刘培: "基于深度学习的图像目标检测与分割算法的研究与应用" * |
| 赵江华;: "基于深度卷积神经网络的露脊鲸识别" * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111681224A (en) * | 2020-06-09 | 2020-09-18 | 上海联影医疗科技有限公司 | Method and device for acquiring blood vessel parameters |
| CN113807132A (en) * | 2020-06-12 | 2021-12-17 | 广州极飞科技股份有限公司 | Method and device for identifying irrigation state of plant growing area and storage medium |
| CN113724214A (en) * | 2021-08-23 | 2021-11-30 | 唯智医疗科技(佛山)有限公司 | Image processing method and device based on neural network |
| CN113724214B (en) * | 2021-08-23 | 2024-02-23 | 唯智医疗科技(佛山)有限公司 | Image processing method and device based on neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110060237B (en) | Fault detection method, device, equipment and system | |
| CN111507958B (en) | Target detection method, training method of detection model and electronic equipment | |
| CN110705405B (en) | Target labeling method and device | |
| CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| US20130279758A1 (en) | Method and system for robust tilt adjustment and cropping of license plate images | |
| CN111178445A (en) | Image processing method and device | |
| JP2017211259A (en) | Inspection device, inspection method and program | |
| CN111242899B (en) | Image-based flaw detection method and computer-readable storage medium | |
| CN112446363A (en) | Image splicing and de-duplication method and device based on video frame extraction | |
| CN110059635B (en) | Skin defect detection method and device | |
| CN111898581A (en) | Animal detection method, device, electronic equipment and readable storage medium | |
| CN114419029B (en) | Training method of surface defect detection model, surface defect detection method and device | |
| CN111652208A (en) | User interface component identification method and device, electronic equipment and storage medium | |
| CN112906794A (en) | Target detection method, device, storage medium and terminal | |
| CN116168351B (en) | Inspection method and device for power equipment | |
| CN113111903A (en) | Intelligent production line monitoring system and monitoring method | |
| CN112819796A (en) | Tobacco shred foreign matter identification method and equipment | |
| CN115471476A (en) | Method, device, equipment and medium for detecting component defects | |
| CN111611944A (en) | Identity recognition method and device, electronic equipment and storage medium | |
| CN113191235A (en) | Sundry detection method, device, equipment and storage medium | |
| CN116228659A (en) | Visual detection method for oil leakage of EMS trolley | |
| CN110634124A (en) | Method and equipment for area detection | |
| CN114550062A (en) | Method and device for determining moving object in image, electronic equipment and storage medium | |
| CN114399657A (en) | Vehicle detection model training method and device, vehicle detection method and electronic equipment | |
| CN113486856A (en) | Driver irregular behavior detection method based on semantic segmentation and convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200519 |