CN112734778A - Vehicle matting method, system, equipment and storage medium based on neural network - Google Patents
Vehicle matting method, system, equipment and storage medium based on neural network Download PDFInfo
- Publication number
- CN112734778A CN112734778A CN202110116600.4A CN202110116600A CN112734778A CN 112734778 A CN112734778 A CN 112734778A CN 202110116600 A CN202110116600 A CN 202110116600A CN 112734778 A CN112734778 A CN 112734778A
- Authority
- CN
- China
- Prior art keywords
- neural network
- vehicle
- matting
- image
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 44
- 230000000877 morphologic effect Effects 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 11
- 238000003062 neural network model Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 25
- 238000010586 diagram Methods 0.000 description 11
- 230000011218 segmentation Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000004069 differentiation Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 230000003628 erosive effect Effects 0.000 description 2
- 238000005111 flow chemistry technique Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 241000760358 Enodes Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Probability & Statistics with Applications (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a vehicle matting method, a system, equipment and a storage medium based on a neural network, wherein the method comprises the following steps: inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point; when the object identification information is a pixel point of the vehicle, performing morphological operation on an area of which the object identification information is a pixel set of the vehicle, and corroding and expanding the edge of the area to obtain an area to be identified; inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network to output at least one track to be scratched of a vehicle; and carrying out matting on a closed graph area surrounded by a track to be matting in the image to be identified. The invention can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.
Description
Technical Field
The invention relates to the field of picture identification, in particular to a vehicle matting method, a system, equipment and a storage medium based on a neural network.
Background
The automatic image matting technology is a branch of digital image processing and is an important image editing technology. The image cutout is used for extracting the outline of an interested target in an image, so that the interference of a disordered background is avoided, the effective display of the information of an article to be displayed can be ensured, the attractiveness of the image display is improved, and the image cutout has wide application in the internet image display.
The automatic cutout of vehicle model picture is an application field of the automatic cutout technology of the image. The difficulty of the cutout of the car-type figure is higher compared with the cutout of other objects such as a commodity picture, because the car-type figure is usually mixed with non-solid background information irrelevant to a car-type main body, such as a road, a sky, a background building and the like, besides displaying main car-type contents. In addition, some pictures may have other information interfering with the vehicle type, such as non-subject vehicle type, unrelated prominent persons, etc. In addition, the vehicle model contour curve is irregular, and the requirement for accurate contour extraction is high. Because the cutout processing complexity of the car-shaped drawing is higher, the manual PS processing of the car-shaped drawing is usually time-consuming and labor-consuming.
In view of this, the invention provides a vehicle matting method, a system, a device and a storage medium based on a neural network.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a vehicle matting method, a system, equipment and a storage medium based on a neural network, overcomes the difficulties in the prior art, can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.
The embodiment of the invention provides a vehicle matting method based on a neural network, which comprises the following steps:
s110, inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
s120, judging whether the object identification information is a pixel point of the vehicle, if so, executing the step S130, and if not, ending;
s130, performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region obtained after corroding and expanding the edge of the region;
s140, inputting the multiple characteristic information of the pixel points in the area to be identified into a trained second convolutional neural network to output at least one track to be scratched of the vehicle;
s150, matting the closed graph area surrounded by the track to be matte from the image to be identified.
Preferably, the step S110 includes the following steps:
s111, inputting an image to be recognized into a trained first convolution neural network for image segmentation based on the image;
s112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network; and
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
Preferably, before the step S110, the method further includes the following steps:
acquiring a first preset number of learning images at least comprising vehicles and recognition results of all graphic areas in the learning images;
and inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of an original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
Preferably, the step S110 further comprises: and training the second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, outputting the category of the vehicle and the track to be scratched of the vehicle by the second convolutional neural network, wherein the track to be scratched is a closed pattern.
Preferably, in the step S130, a background area and a foreground area, of which the areas are not changed after the morphological operation, are respectively used as the area to be identified and the background area.
Preferably, in the step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
Preferably, in the step S140, only each region to be recognized is input into the trained second convolutional neural network, so as to obtain a contour track of the vehicle in each region to be recognized, and a contour track with the maximum total area of pixels enclosed by the contour in the image is output as a track to be scratched.
The embodiment of the present invention further provides a vehicle matting system based on a neural network, which is used for implementing the above vehicle matting method based on the neural network, and the vehicle matting system based on the neural network includes:
the region extraction module is used for inputting an image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
the region judgment module is used for judging whether the object identification information is a pixel point of the vehicle, and if so, the form operation module is executed;
the morphological operation module is used for performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region after the edge of the region is corroded and expanded;
the matting track module is used for inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network and outputting at least one matting track of the vehicle;
and the area matting module is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
Embodiments of the present invention also provide a neural network-based vehicle matting apparatus, including:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the neural network-based vehicle matting method described above via execution of the executable instructions.
Embodiments of the present invention also provide a computer-readable storage medium storing a program that, when executed, implements the steps of the neural network-based vehicle matting method described above.
The invention aims to provide a vehicle matting method, a system, equipment and a storage medium based on a neural network, which can greatly improve the processing efficiency on large-scale image processing, ensure the matting quality and have higher application value.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, with reference to the accompanying drawings.
FIG. 1 is a flow chart diagram of a neural network-based vehicle matting method of the present invention.
Fig. 2 to 5 are process diagrams of a flow implementing the neural network-based vehicle matting method of the present invention.
Fig. 6 is a schematic structural diagram of a neural network-based vehicle matting system of the present invention.
Fig. 7 is a schematic structural diagram of a neural network-based vehicle matting device of the present invention.
Fig. 8 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus their repetitive description will be omitted.
FIG. 1 is a flow chart diagram of a neural network-based vehicle matting method of the present invention. As shown in fig. 1, an embodiment of the present invention provides a neural network-based vehicle matting method, including the following steps:
s110, inputting the image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point.
And S120, judging whether the object identification information is a pixel point of the vehicle, if so, executing a step S130, and if not, executing a step S160.
S130, performing morphological operation on the region with the object identification information as the pixel set of the vehicle, and obtaining the region to be identified from the region with the edge of the region subjected to corrosion and expansion.
S140, inputting the multiple characteristic information of the pixel points in the area to be identified into the trained second convolutional neural network to output at least one track to be scratched of the vehicle.
S150, matting the closed graph area surrounded by the track to be matting in the image to be identified.
And S160, ending.
In a preferred embodiment, step S110 includes the following steps:
and S111, inputting the image to be recognized into the trained first convolution neural network for image segmentation based on the image.
And S112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network. And
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
In a preferred embodiment, before step S110, the following steps are further included:
a first predetermined number of learning images including at least a vehicle and recognition results of respective pattern regions in the learning images are acquired.
And inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of the original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
In a preferred embodiment, a large number of pictures with subareas and object identifications are adopted to learn and train the first convolutional neural network, so that the first convolutional neural network can automatically perform area differentiation on the input pictures, perform image recognition on each area separately, and output the object identification of each area.
In a preferred embodiment, a second convolutional neural network is trained by using a large amount of point cloud information with multiple characteristic information, so that the second convolutional neural network can accurately output the category of the 3D target. In the present invention, the difference between training the first convolutional neural network and the second convolutional neural network is: the first convolutional neural network has the functions of regional differentiation and primary image identification, and the second convolutional neural network has no function of regional differentiation and only has the function of accurately identifying regions compared with the first convolutional neural network.
In a preferred embodiment, step S110 further includes: and training a second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, wherein the second convolutional neural network outputs the category of the vehicle and the track to be scratched of the vehicle, and the track to be scratched is a closed pattern.
In a preferred embodiment, in step S130, a background area and a foreground area, which are unchanged after morphological operations, are respectively used as the area to be identified and the background area.
In a preferred embodiment, in step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
In a preferred embodiment, in step S140, only the trained second convolutional neural network is input into each region to be recognized, so as to obtain the contour trajectory of the vehicle in each region to be recognized, and the contour trajectory with the largest total area of pixels enclosed by the contour in the image is output as the trajectory to be scratched.
Fig. 2 to 5 are process diagrams of a flow implementing the neural network-based vehicle matting method of the present invention. One embodiment of the present invention is described below with reference to fig. 2 to 5:
first, a first predetermined number of learning images including at least a vehicle and recognition results of respective pattern regions in the learning images are acquired. And inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of the original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
And training a second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, wherein the second convolutional neural network outputs the category of the vehicle and the track to be scratched of the vehicle, and the track to be scratched is a closed pattern. For example: and training a second convolutional neural network by adopting a large amount of point cloud information with multiple characteristic information, so that the second convolutional neural network can accurately output the category and the range of the 3D target in the image, and the multiple characteristic information at least comprises color information RGB (red, green and blue) of each pixel point and object identification information D.
Referring to fig. 2, an image 10 to be recognized including a vehicle 1 and a vehicle 2 parked on the ground is input into a trained first convolutional neural network to extract multi-feature information of each pixel in image information, where the multi-feature information at least includes color information of each pixel and object identification information. The identification is performed through the first convolutional neural network to add the object identification information to each pixel, for example, the multiple feature information of each pixel in the pattern where the vehicle 2 is located is (R, G, B, D), the object identification information D of this part of the pixels is the vehicle, similarly, the multiple feature information of each pixel in the pattern where the vehicle 1 is located is (R, G, B, D), the object identification information D of this part of the pixels is the vehicle, the multiple feature information of the rest of the pixels is (R, G, B, D), and the object identification information D of this part of the pixels is the ground. In this embodiment, the first convolutional neural network plays a role of quickly locating the position of the vehicle in the picture, which is equivalent to the operation of rough matting.
Referring to fig. 3, since there is a pixel point of the vehicle corresponding to the object identification information, morphological operation is performed on the region where the object identification information is the pixel set of the vehicle, and the region to be identified is obtained from the region where the edge of the region is corroded and expanded. In order to avoid the operation of rough matting of the first convolution neural network from damaging the edge outline of the vehicle, certain region expansion is carried out to obtain a pattern range containing the complete vehicle.
Referring to fig. 4, through area comparison, the area of the image area occupied by the vehicle 2 is larger than the area of the image area occupied by the vehicle 1, then only the multiple feature information of the pixel point of the area to be recognized (the image area occupied by the vehicle 2) with the largest area in the image 10 to be recognized is (R, G, B, D), the trained second convolutional neural network is input, and the track to be scratched of a vehicle (based on the image area occupied by the vehicle 2) is output through the second convolutional neural network. And carrying out fine matting on the pattern range containing the complete vehicle by using a second convolutional neural network so as to obtain a more accurate vehicle contour.
Referring to fig. 5, a closed figure region enclosed by a trajectory to be scratched is scratched from an image to be recognized 10.
The invention gives consideration to the speed and the accuracy of the image matting through the combined use of the first convolutional neural network and the second convolutional neural network which realize two functions, and can accurately matte the most obvious vehicle in the image from the image.
The problems to be solved by the invention are as follows: after background interference information is removed from the car images, the car images can be conveniently displayed and marketed on an internet platform, and browsing experience of users can be improved. Because the car picture structure is complicated, the mode of current manual work carrying out the car picture is scratched, and the treatment effeciency is low, and the cost of labor is comparatively expensive. The invention provides an automatic vehicle type picture matting method based on image instance segmentation and image matting technology, which realizes automatic matting of vehicle type pictures, can greatly save labor cost and improve intelligent image processing efficiency.
The invention discloses an automatic vehicle type picture matting method based on example segmentation and automatic image matting methods, which comprises the following steps of
The method comprises the following steps:
step one, constructing an instance segmentation model
The image salient example car body segmentation model provided by the invention comprises the steps of (1) generating a group of prototype masks and (2) predicting the mask coefficient of each example. Instance masks are then generated by linearly combining the prototype with the template coefficients. Since this process is not dependent on repooling, this method can produce very high quality masks. In addition, the emergent behavior of prototypes was analyzed and shown to be fully convoluted. Finally, by incorporating a deformable convolution into the backbone network, the predicted head is optimized using a better anchor scale and aspect ratio, and a novel fast masks re-scoring branch is added.
Step two, generating a ternary prime map
And generating a three-element map of the car map through morphological operation of the image based on the result of example segmentation. Specifically, first. Color or other replacement is performed on the background, and then byte and or operation is performed on the background and the background, so that the background and the background are combined into a new image. The edge of the background can also be subjected to erosion enode and expansion dilate operation for reducing or expanding the outline, the expansion and erosion variable region is used as an uncertain region of the ternary prime map, and the background and foreground parts of the unchanged region before and after the morphological operation are respectively used as a target region and a background region of the ternary prime map.
Step three, training the sectional drawing model
The purpose of the matting model is to make the contours of the uncertainty region fine and realistic, specifically, the loss function is a weighted combination of two loss: the method comprises the steps of (1) predicting alpha and actual alpha, wherein the predicted alpha is fused with corresponding background and foreground through a formula (1), and then is subjected to error with a ground truth RGB map, only the prediction error of an unknown region in the trimap is propagated backwards (the difference between the matting and background removal is realized, the matting needs the trimap as a user interaction interface, only the uncertain region in the matting needs to be predicted, and the background removal needs to propagate the error of the whole map in a reverse way), namely, the parts of the trimap, which are pure background and pure foreground, are not counted in a model prediction range, and only the two parts need to be directly copied to output alpha.
Step four, predictive generation of the main body part of the car map
In order to realize the high-quality automatic matting effect and set, a self-research algorithm is used for ensuring the integrity and the accuracy of local details of the matting object.
Step five, matting result output
And outputting a result picture with a background which is a pure color or transparent background according to a preset background requirement.
The method is used for constructing an example segmentation model for extracting the rough outline of the car image body based on the example segmentation data set. And generating a three-element diagram for image matting through morphological operation of the image. And then synthesizing a cutout model for accurate cutout of the car drawing based on the cutout data of the open source object, and inputting the ternary element drawing into the cutout model to produce a segmentation drawing with higher quality. Practice proves that the manual verification of the algorithm provided by the invention can reach 98% satisfaction rate, the result after the image matting can be returned within 1s on average, the automation efficiency is improved, and the labor cost is greatly saved.
Fig. 6 is a schematic structural diagram of a neural network-based vehicle matting system of the present invention. As shown in fig. 6, the neural network-based vehicle matting system 5 of the present invention includes:
the region extraction module 51 is configured to input the image to be recognized into the trained first convolutional neural network to extract multiple feature information of each pixel in the image information, where the multiple feature information at least includes color information and object identification information of each pixel;
the region judgment module 52 judges whether the object identification information is a pixel point of the vehicle, and if so, executes the form operation module;
the morphological operation module 53 is configured to perform morphological operation on the region where the object identification information is the pixel set of the vehicle, and obtain a region to be identified from the region where the edge of the region is corroded and expanded;
a matting track module 54, which inputs the multiple feature information of the pixel points in the region to be identified into a trained second convolutional neural network to output at least a matting track of the vehicle;
and the area matting module 55 is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
The vehicle matting system based on the neural network can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.
The embodiment of the invention also provides a vehicle matting device based on the neural network, which comprises a processor. A memory having stored therein executable instructions of the processor. Wherein the processor is configured to perform the steps of the neural network-based vehicle matting method via execution of executable instructions.
As shown above, the vehicle matting system based on the neural network according to the embodiment of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" platform.
Fig. 7 is a schematic structural diagram of a neural network-based vehicle matting device of the present invention. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 600 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of the present specification. For example, processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
Embodiments of the present invention also provide a computer-readable storage medium for storing a program, where the program implements the steps of the neural network-based vehicle matting method when executed. In some possible embodiments, the aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of this specification, when the program product is run on the terminal device.
As shown above, the vehicle matting system based on the neural network according to the embodiment of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
Fig. 8 is a schematic structural diagram of a computer-readable storage medium of the present invention. Referring to fig. 8, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the present invention aims to provide a method, a system, a device and a storage medium for vehicle matting based on a neural network, and the system for vehicle matting based on a neural network of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (10)
1. A vehicle matting method based on a neural network is characterized by comprising the following steps:
s110, inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
s120, judging whether the object identification information is a pixel point of the vehicle, if so, executing the step S130, and if not, ending;
s130, performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region obtained after corroding and expanding the edge of the region;
s140, inputting the multiple characteristic information of the pixel points in the area to be identified into a trained second convolutional neural network to output at least one track to be scratched of the vehicle;
s150, matting the closed graph area surrounded by the track to be matte from the image to be identified.
2. The neural network based vehicle matting method according to claim 1, characterized in that said step S110 includes the steps of:
s111, inputting an image to be recognized into a trained first convolution neural network for image segmentation based on the image;
s112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network; and
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
3. The neural network-based vehicle matting method according to claim 2, further comprising, before said step S110, the steps of:
acquiring a first preset number of learning images at least comprising vehicles and recognition results of all graphic areas in the learning images;
and inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of an original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
4. The neural network-based vehicle matting method according to claim 2, characterized in that said step S110 is preceded by further comprising: and training the second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, outputting the category of the vehicle and the track to be scratched of the vehicle by the second convolutional neural network, wherein the track to be scratched is a closed pattern.
5. The neural network-based vehicle matting method according to claim 1, wherein in the step S130, a background region and a foreground region, which are unchanged in region after morphological operation, are respectively used as the region to be identified and the background region.
6. The method as claimed in claim 5, wherein in step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
7. The method as claimed in claim 5, wherein in step S140, each region to be identified is input into the trained second convolutional neural network, so as to obtain the contour track of the vehicle in each region to be identified, and the contour track with the largest total area of pixels enclosed by the contour in the image is output as the trajectory to be subjected to matting.
8. A neural network-based vehicle matting system for implementing the neural network-based vehicle matting method according to claim 1, characterized by comprising:
the region extraction module is used for inputting an image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
the region judgment module is used for judging whether the object identification information is a pixel point of the vehicle, and if so, the form operation module is executed;
the morphological operation module is used for performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region after the edge of the region is corroded and expanded;
the matting track module is used for inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network and outputting at least one matting track of the vehicle;
and the area matting module is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
9. A neural network-based vehicle matting device, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the neural network-based vehicle matting method of any one of claims 1 to 7 via execution of the executable instructions.
10. A computer readable storage medium storing a program, wherein the program when executed implements the steps of the neural network-based vehicle matting method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110116600.4A CN112734778A (en) | 2021-01-28 | 2021-01-28 | Vehicle matting method, system, equipment and storage medium based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110116600.4A CN112734778A (en) | 2021-01-28 | 2021-01-28 | Vehicle matting method, system, equipment and storage medium based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112734778A true CN112734778A (en) | 2021-04-30 |
Family
ID=75594355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110116600.4A Pending CN112734778A (en) | 2021-01-28 | 2021-01-28 | Vehicle matting method, system, equipment and storage medium based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112734778A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387331A (en) * | 2022-01-14 | 2022-04-22 | 平安科技（深圳）有限公司 | Method, device and equipment for evaluating segmentation quality of vehicle image and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909016A (en) * | 2017-11-03 | 2018-04-13 | 车智互联（北京）科技有限公司 | A kind of convolutional neural networks generation method and the recognition methods of car system |
| CN107945204A (en) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | A kind of Pixel-level portrait based on generation confrontation network scratches drawing method |
| CN108399454A (en) * | 2018-03-05 | 2018-08-14 | 山东领能电子科技有限公司 | A kind of completely new sectional convolution neural network target recognition |
| CN108961279A (en) * | 2018-06-28 | 2018-12-07 | Oppo（重庆）智能科技有限公司 | Image processing method, device and mobile terminal |
| CN110889855A (en) * | 2019-10-28 | 2020-03-17 | 公安部交通管理科学研究所 | Certificate photo matting method and system based on end-to-end convolutional neural network |
| CN111126248A (en) * | 2019-12-20 | 2020-05-08 | 湖南千视通信息科技有限公司 | Method and device for identifying shielded vehicle |
| CN111383232A (en) * | 2018-12-29 | 2020-07-07 | Tcl集团股份有限公司 | Matting method, matting device, terminal equipment and computer-readable storage medium |
| WO2020186790A1 (en) * | 2019-03-15 | 2020-09-24 | 平安科技（深圳）有限公司 | Vehicle model identification method, device, computer apparatus, and storage medium |
-
2021
- 2021-01-28 CN CN202110116600.4A patent/CN112734778A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107945204A (en) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | A kind of Pixel-level portrait based on generation confrontation network scratches drawing method |
| CN107909016A (en) * | 2017-11-03 | 2018-04-13 | 车智互联（北京）科技有限公司 | A kind of convolutional neural networks generation method and the recognition methods of car system |
| CN108399454A (en) * | 2018-03-05 | 2018-08-14 | 山东领能电子科技有限公司 | A kind of completely new sectional convolution neural network target recognition |
| CN108961279A (en) * | 2018-06-28 | 2018-12-07 | Oppo（重庆）智能科技有限公司 | Image processing method, device and mobile terminal |
| CN111383232A (en) * | 2018-12-29 | 2020-07-07 | Tcl集团股份有限公司 | Matting method, matting device, terminal equipment and computer-readable storage medium |
| WO2020186790A1 (en) * | 2019-03-15 | 2020-09-24 | 平安科技（深圳）有限公司 | Vehicle model identification method, device, computer apparatus, and storage medium |
| CN110889855A (en) * | 2019-10-28 | 2020-03-17 | 公安部交通管理科学研究所 | Certificate photo matting method and system based on end-to-end convolutional neural network |
| CN111126248A (en) * | 2019-12-20 | 2020-05-08 | 湖南千视通信息科技有限公司 | Method and device for identifying shielded vehicle |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387331A (en) * | 2022-01-14 | 2022-04-22 | 平安科技（深圳）有限公司 | Method, device and equipment for evaluating segmentation quality of vehicle image and storage medium |
| CN114387331B (en) * | 2022-01-14 | 2024-07-16 | 平安科技（深圳）有限公司 | Method, device, equipment and storage medium for evaluating vehicle image segmentation quality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110176027B (en) | Video target tracking method, device, equipment and storage medium | |
| CN110503097A (en) | Training method, device and the storage medium of image processing model | |
| CN110245579B (en) | People flow density prediction method and device, computer equipment and readable medium | |
| CN112116599B (en) | Sputum smear tubercle bacillus semantic segmentation method and system based on weak supervised learning | |
| CN107909638B (en) | Rendering method, medium, system and electronic device of virtual object | |
| CN111563502A (en) | Image text recognition method and device, electronic equipment and computer storage medium | |
| CN112287941B (en) | License plate recognition method based on automatic character region perception | |
| CN112418216A (en) | Method for detecting characters in complex natural scene image | |
| CN112990331A (en) | Image processing method, electronic device, and storage medium | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113569852A (en) | Training method and device of semantic segmentation model, electronic equipment and storage medium | |
| CN112287144A (en) | Picture retrieval method, equipment and storage medium | |
| CN113762303A (en) | Image classification method and device, electronic equipment and storage medium | |
| CN117830537B (en) | Weak supervision 3D scene graph generation method, device, equipment and medium | |
| CN115100469A (en) | Target attribute identification method, training method and device based on segmentation algorithm | |
| CN116071557A (en) | Long tail target detection method, computer readable storage medium and driving device | |
| CN113223011B (en) | Small sample image segmentation method based on guide network and full-connection conditional random field | |
| CN111709762B (en) | Information matching degree evaluation method, device, equipment and storage medium | |
| CN111062388B (en) | Advertisement character recognition method, system, medium and equipment based on deep learning | |
| CN112734778A (en) | Vehicle matting method, system, equipment and storage medium based on neural network | |
| CN113744280A (en) | Image processing method, apparatus, device and medium | |
| CN111815748B (en) | Animation processing method and device, storage medium and electronic equipment | |
| CN112258416A (en) | Image processing method and device and electronic equipment | |
| CN112434591B (en) | Lane line determination method and device | |
| CN117121050A (en) | Segmenting and removing objects from media items |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |