CN112734778A - Vehicle matting method, system, equipment and storage medium based on neural network - Google Patents

Vehicle matting method, system, equipment and storage medium based on neural network Download PDF

Info

Publication number
CN112734778A
CN112734778A CN202110116600.4A CN202110116600A CN112734778A CN 112734778 A CN112734778 A CN 112734778A CN 202110116600 A CN202110116600 A CN 202110116600A CN 112734778 A CN112734778 A CN 112734778A
Authority
CN
China
Prior art keywords
neural network
vehicle
matting
image
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110116600.4A
Other languages
Chinese (zh)
Inventor
成丹妮
罗超
邵旻昊
邹宇
胡泓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ctrip Computer Technology Shanghai Co Ltd
Original Assignee
Ctrip Computer Technology Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ctrip Computer Technology Shanghai Co Ltd filed Critical Ctrip Computer Technology Shanghai Co Ltd
Priority to CN202110116600.4A priority Critical patent/CN112734778A/en
Publication of CN112734778A publication Critical patent/CN112734778A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/047Probabilistic or stochastic networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a vehicle matting method, a system, equipment and a storage medium based on a neural network, wherein the method comprises the following steps: inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point; when the object identification information is a pixel point of the vehicle, performing morphological operation on an area of which the object identification information is a pixel set of the vehicle, and corroding and expanding the edge of the area to obtain an area to be identified; inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network to output at least one track to be scratched of a vehicle; and carrying out matting on a closed graph area surrounded by a track to be matting in the image to be identified. The invention can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.

Description

Vehicle matting method, system, equipment and storage medium based on neural network
Technical Field
The invention relates to the field of picture identification, in particular to a vehicle matting method, a system, equipment and a storage medium based on a neural network.
Background
The automatic image matting technology is a branch of digital image processing and is an important image editing technology. The image cutout is used for extracting the outline of an interested target in an image, so that the interference of a disordered background is avoided, the effective display of the information of an article to be displayed can be ensured, the attractiveness of the image display is improved, and the image cutout has wide application in the internet image display.
The automatic cutout of vehicle model picture is an application field of the automatic cutout technology of the image. The difficulty of the cutout of the car-type figure is higher compared with the cutout of other objects such as a commodity picture, because the car-type figure is usually mixed with non-solid background information irrelevant to a car-type main body, such as a road, a sky, a background building and the like, besides displaying main car-type contents. In addition, some pictures may have other information interfering with the vehicle type, such as non-subject vehicle type, unrelated prominent persons, etc. In addition, the vehicle model contour curve is irregular, and the requirement for accurate contour extraction is high. Because the cutout processing complexity of the car-shaped drawing is higher, the manual PS processing of the car-shaped drawing is usually time-consuming and labor-consuming.
In view of this, the invention provides a vehicle matting method, a system, a device and a storage medium based on a neural network.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a vehicle matting method, a system, equipment and a storage medium based on a neural network, overcomes the difficulties in the prior art, can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.
The embodiment of the invention provides a vehicle matting method based on a neural network, which comprises the following steps:
s110, inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
s120, judging whether the object identification information is a pixel point of the vehicle, if so, executing the step S130, and if not, ending;
s130, performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region obtained after corroding and expanding the edge of the region;
s140, inputting the multiple characteristic information of the pixel points in the area to be identified into a trained second convolutional neural network to output at least one track to be scratched of the vehicle;
s150, matting the closed graph area surrounded by the track to be matte from the image to be identified.
Preferably, the step S110 includes the following steps:
s111, inputting an image to be recognized into a trained first convolution neural network for image segmentation based on the image;
s112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network; and
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
Preferably, before the step S110, the method further includes the following steps:
acquiring a first preset number of learning images at least comprising vehicles and recognition results of all graphic areas in the learning images;
and inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of an original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
Preferably, the step S110 further comprises: and training the second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, outputting the category of the vehicle and the track to be scratched of the vehicle by the second convolutional neural network, wherein the track to be scratched is a closed pattern.
Preferably, in the step S130, a background area and a foreground area, of which the areas are not changed after the morphological operation, are respectively used as the area to be identified and the background area.
Preferably, in the step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
Preferably, in the step S140, only each region to be recognized is input into the trained second convolutional neural network, so as to obtain a contour track of the vehicle in each region to be recognized, and a contour track with the maximum total area of pixels enclosed by the contour in the image is output as a track to be scratched.
The embodiment of the present invention further provides a vehicle matting system based on a neural network, which is used for implementing the above vehicle matting method based on the neural network, and the vehicle matting system based on the neural network includes:
the region extraction module is used for inputting an image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
the region judgment module is used for judging whether the object identification information is a pixel point of the vehicle, and if so, the form operation module is executed;
the morphological operation module is used for performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region after the edge of the region is corroded and expanded;
the matting track module is used for inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network and outputting at least one matting track of the vehicle;
and the area matting module is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
Embodiments of the present invention also provide a neural network-based vehicle matting apparatus, including:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the neural network-based vehicle matting method described above via execution of the executable instructions.
Embodiments of the present invention also provide a computer-readable storage medium storing a program that, when executed, implements the steps of the neural network-based vehicle matting method described above.
The invention aims to provide a vehicle matting method, a system, equipment and a storage medium based on a neural network, which can greatly improve the processing efficiency on large-scale image processing, ensure the matting quality and have higher application value.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, with reference to the accompanying drawings.
FIG. 1 is a flow chart diagram of a neural network-based vehicle matting method of the present invention.
Fig. 2 to 5 are process diagrams of a flow implementing the neural network-based vehicle matting method of the present invention.
Fig. 6 is a schematic structural diagram of a neural network-based vehicle matting system of the present invention.
Fig. 7 is a schematic structural diagram of a neural network-based vehicle matting device of the present invention.
Fig. 8 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus their repetitive description will be omitted.
FIG. 1 is a flow chart diagram of a neural network-based vehicle matting method of the present invention. As shown in fig. 1, an embodiment of the present invention provides a neural network-based vehicle matting method, including the following steps:
s110, inputting the image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point.
And S120, judging whether the object identification information is a pixel point of the vehicle, if so, executing a step S130, and if not, executing a step S160.
S130, performing morphological operation on the region with the object identification information as the pixel set of the vehicle, and obtaining the region to be identified from the region with the edge of the region subjected to corrosion and expansion.
S140, inputting the multiple characteristic information of the pixel points in the area to be identified into the trained second convolutional neural network to output at least one track to be scratched of the vehicle.
S150, matting the closed graph area surrounded by the track to be matting in the image to be identified.
And S160, ending.
In a preferred embodiment, step S110 includes the following steps:
and S111, inputting the image to be recognized into the trained first convolution neural network for image segmentation based on the image.
And S112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network. And
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
In a preferred embodiment, before step S110, the following steps are further included:
a first predetermined number of learning images including at least a vehicle and recognition results of respective pattern regions in the learning images are acquired.
And inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of the original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
In a preferred embodiment, a large number of pictures with subareas and object identifications are adopted to learn and train the first convolutional neural network, so that the first convolutional neural network can automatically perform area differentiation on the input pictures, perform image recognition on each area separately, and output the object identification of each area.
In a preferred embodiment, a second convolutional neural network is trained by using a large amount of point cloud information with multiple characteristic information, so that the second convolutional neural network can accurately output the category of the 3D target. In the present invention, the difference between training the first convolutional neural network and the second convolutional neural network is: the first convolutional neural network has the functions of regional differentiation and primary image identification, and the second convolutional neural network has no function of regional differentiation and only has the function of accurately identifying regions compared with the first convolutional neural network.
In a preferred embodiment, step S110 further includes: and training a second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, wherein the second convolutional neural network outputs the category of the vehicle and the track to be scratched of the vehicle, and the track to be scratched is a closed pattern.
In a preferred embodiment, in step S130, a background area and a foreground area, which are unchanged after morphological operations, are respectively used as the area to be identified and the background area.
In a preferred embodiment, in step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
In a preferred embodiment, in step S140, only the trained second convolutional neural network is input into each region to be recognized, so as to obtain the contour trajectory of the vehicle in each region to be recognized, and the contour trajectory with the largest total area of pixels enclosed by the contour in the image is output as the trajectory to be scratched.
Fig. 2 to 5 are process diagrams of a flow implementing the neural network-based vehicle matting method of the present invention. One embodiment of the present invention is described below with reference to fig. 2 to 5:
first, a first predetermined number of learning images including at least a vehicle and recognition results of respective pattern regions in the learning images are acquired. And inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of the original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
And training a second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, wherein the second convolutional neural network outputs the category of the vehicle and the track to be scratched of the vehicle, and the track to be scratched is a closed pattern. For example: and training a second convolutional neural network by adopting a large amount of point cloud information with multiple characteristic information, so that the second convolutional neural network can accurately output the category and the range of the 3D target in the image, and the multiple characteristic information at least comprises color information RGB (red, green and blue) of each pixel point and object identification information D.
Referring to fig. 2, an image 10 to be recognized including a vehicle 1 and a vehicle 2 parked on the ground is input into a trained first convolutional neural network to extract multi-feature information of each pixel in image information, where the multi-feature information at least includes color information of each pixel and object identification information. The identification is performed through the first convolutional neural network to add the object identification information to each pixel, for example, the multiple feature information of each pixel in the pattern where the vehicle 2 is located is (R, G, B, D), the object identification information D of this part of the pixels is the vehicle, similarly, the multiple feature information of each pixel in the pattern where the vehicle 1 is located is (R, G, B, D), the object identification information D of this part of the pixels is the vehicle, the multiple feature information of the rest of the pixels is (R, G, B, D), and the object identification information D of this part of the pixels is the ground. In this embodiment, the first convolutional neural network plays a role of quickly locating the position of the vehicle in the picture, which is equivalent to the operation of rough matting.
Referring to fig. 3, since there is a pixel point of the vehicle corresponding to the object identification information, morphological operation is performed on the region where the object identification information is the pixel set of the vehicle, and the region to be identified is obtained from the region where the edge of the region is corroded and expanded. In order to avoid the operation of rough matting of the first convolution neural network from damaging the edge outline of the vehicle, certain region expansion is carried out to obtain a pattern range containing the complete vehicle.
Referring to fig. 4, through area comparison, the area of the image area occupied by the vehicle 2 is larger than the area of the image area occupied by the vehicle 1, then only the multiple feature information of the pixel point of the area to be recognized (the image area occupied by the vehicle 2) with the largest area in the image 10 to be recognized is (R, G, B, D), the trained second convolutional neural network is input, and the track to be scratched of a vehicle (based on the image area occupied by the vehicle 2) is output through the second convolutional neural network. And carrying out fine matting on the pattern range containing the complete vehicle by using a second convolutional neural network so as to obtain a more accurate vehicle contour.
Referring to fig. 5, a closed figure region enclosed by a trajectory to be scratched is scratched from an image to be recognized 10.
The invention gives consideration to the speed and the accuracy of the image matting through the combined use of the first convolutional neural network and the second convolutional neural network which realize two functions, and can accurately matte the most obvious vehicle in the image from the image.
The problems to be solved by the invention are as follows: after background interference information is removed from the car images, the car images can be conveniently displayed and marketed on an internet platform, and browsing experience of users can be improved. Because the car picture structure is complicated, the mode of current manual work carrying out the car picture is scratched, and the treatment effeciency is low, and the cost of labor is comparatively expensive. The invention provides an automatic vehicle type picture matting method based on image instance segmentation and image matting technology, which realizes automatic matting of vehicle type pictures, can greatly save labor cost and improve intelligent image processing efficiency.
The invention discloses an automatic vehicle type picture matting method based on example segmentation and automatic image matting methods, which comprises the following steps of
The method comprises the following steps:
step one, constructing an instance segmentation model
The image salient example car body segmentation model provided by the invention comprises the steps of (1) generating a group of prototype masks and (2) predicting the mask coefficient of each example. Instance masks are then generated by linearly combining the prototype with the template coefficients. Since this process is not dependent on repooling, this method can produce very high quality masks. In addition, the emergent behavior of prototypes was analyzed and shown to be fully convoluted. Finally, by incorporating a deformable convolution into the backbone network, the predicted head is optimized using a better anchor scale and aspect ratio, and a novel fast masks re-scoring branch is added.
Step two, generating a ternary prime map
And generating a three-element map of the car map through morphological operation of the image based on the result of example segmentation. Specifically, first. Color or other replacement is performed on the background, and then byte and or operation is performed on the background and the background, so that the background and the background are combined into a new image. The edge of the background can also be subjected to erosion enode and expansion dilate operation for reducing or expanding the outline, the expansion and erosion variable region is used as an uncertain region of the ternary prime map, and the background and foreground parts of the unchanged region before and after the morphological operation are respectively used as a target region and a background region of the ternary prime map.
Step three, training the sectional drawing model
The purpose of the matting model is to make the contours of the uncertainty region fine and realistic, specifically, the loss function is a weighted combination of two loss: the method comprises the steps of (1) predicting alpha and actual alpha, wherein the predicted alpha is fused with corresponding background and foreground through a formula (1), and then is subjected to error with a ground truth RGB map, only the prediction error of an unknown region in the trimap is propagated backwards (the difference between the matting and background removal is realized, the matting needs the trimap as a user interaction interface, only the uncertain region in the matting needs to be predicted, and the background removal needs to propagate the error of the whole map in a reverse way), namely, the parts of the trimap, which are pure background and pure foreground, are not counted in a model prediction range, and only the two parts need to be directly copied to output alpha.
Step four, predictive generation of the main body part of the car map
In order to realize the high-quality automatic matting effect and set, a self-research algorithm is used for ensuring the integrity and the accuracy of local details of the matting object.
Step five, matting result output
And outputting a result picture with a background which is a pure color or transparent background according to a preset background requirement.
The method is used for constructing an example segmentation model for extracting the rough outline of the car image body based on the example segmentation data set. And generating a three-element diagram for image matting through morphological operation of the image. And then synthesizing a cutout model for accurate cutout of the car drawing based on the cutout data of the open source object, and inputting the ternary element drawing into the cutout model to produce a segmentation drawing with higher quality. Practice proves that the manual verification of the algorithm provided by the invention can reach 98% satisfaction rate, the result after the image matting can be returned within 1s on average, the automation efficiency is improved, and the labor cost is greatly saved.
Fig. 6 is a schematic structural diagram of a neural network-based vehicle matting system of the present invention. As shown in fig. 6, the neural network-based vehicle matting system 5 of the present invention includes:
the region extraction module 51 is configured to input the image to be recognized into the trained first convolutional neural network to extract multiple feature information of each pixel in the image information, where the multiple feature information at least includes color information and object identification information of each pixel;
the region judgment module 52 judges whether the object identification information is a pixel point of the vehicle, and if so, executes the form operation module;
the morphological operation module 53 is configured to perform morphological operation on the region where the object identification information is the pixel set of the vehicle, and obtain a region to be identified from the region where the edge of the region is corroded and expanded;
a matting track module 54, which inputs the multiple feature information of the pixel points in the region to be identified into a trained second convolutional neural network to output at least a matting track of the vehicle;
and the area matting module 55 is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
The vehicle matting system based on the neural network can greatly improve the processing efficiency on large-scale image processing, ensures the matting quality and has higher application value.
The embodiment of the invention also provides a vehicle matting device based on the neural network, which comprises a processor. A memory having stored therein executable instructions of the processor. Wherein the processor is configured to perform the steps of the neural network-based vehicle matting method via execution of executable instructions.
As shown above, the vehicle matting system based on the neural network according to the embodiment of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" platform.
Fig. 7 is a schematic structural diagram of a neural network-based vehicle matting device of the present invention. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 600 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of the present specification. For example, processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 630 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
Embodiments of the present invention also provide a computer-readable storage medium for storing a program, where the program implements the steps of the neural network-based vehicle matting method when executed. In some possible embodiments, the aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of this specification, when the program product is run on the terminal device.
As shown above, the vehicle matting system based on the neural network according to the embodiment of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
Fig. 8 is a schematic structural diagram of a computer-readable storage medium of the present invention. Referring to fig. 8, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the present invention aims to provide a method, a system, a device and a storage medium for vehicle matting based on a neural network, and the system for vehicle matting based on a neural network of the present invention can greatly improve processing efficiency in large-scale image processing, and ensure matting quality, and has a high application value.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims (10)

1. A vehicle matting method based on a neural network is characterized by comprising the following steps:
s110, inputting an image to be recognized into a trained first convolution neural network to extract multiple characteristic information of each pixel point in image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
s120, judging whether the object identification information is a pixel point of the vehicle, if so, executing the step S130, and if not, ending;
s130, performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region obtained after corroding and expanding the edge of the region;
s140, inputting the multiple characteristic information of the pixel points in the area to be identified into a trained second convolutional neural network to output at least one track to be scratched of the vehicle;
s150, matting the closed graph area surrounded by the track to be matte from the image to be identified.
2. The neural network based vehicle matting method according to claim 1, characterized in that said step S110 includes the steps of:
s111, inputting an image to be recognized into a trained first convolution neural network for image segmentation based on the image;
s112, obtaining object identification information D corresponding to each segmented image area in the image through a first convolutional neural network; and
s113, the multiple characteristic information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located.
3. The neural network-based vehicle matting method according to claim 2, further comprising, before said step S110, the steps of:
acquiring a first preset number of learning images at least comprising vehicles and recognition results of all graphic areas in the learning images;
and inputting the learning image and the recognition result of each graphic area into a convolution network, an expansion network and a Softmax layer of an original convolution neural network model, and training the original neural network model to obtain a first convolution neural network.
4. The neural network-based vehicle matting method according to claim 2, characterized in that said step S110 is preceded by further comprising: and training the second convolutional neural network by adopting a large number of vehicle images with multiple characteristic information, outputting the category of the vehicle and the track to be scratched of the vehicle by the second convolutional neural network, wherein the track to be scratched is a closed pattern.
5. The neural network-based vehicle matting method according to claim 1, wherein in the step S130, a background region and a foreground region, which are unchanged in region after morphological operation, are respectively used as the region to be identified and the background region.
6. The method as claimed in claim 5, wherein in step S140, only the area to be recognized with the largest area in the image to be recognized is input into the trained second convolutional neural network, so as to obtain the contour trajectory of the vehicle in the area to be recognized.
7. The method as claimed in claim 5, wherein in step S140, each region to be identified is input into the trained second convolutional neural network, so as to obtain the contour track of the vehicle in each region to be identified, and the contour track with the largest total area of pixels enclosed by the contour in the image is output as the trajectory to be subjected to matting.
8. A neural network-based vehicle matting system for implementing the neural network-based vehicle matting method according to claim 1, characterized by comprising:
the region extraction module is used for inputting an image to be recognized into the trained first convolution neural network to extract multiple characteristic information of each pixel point in the image information, wherein the multiple characteristic information at least comprises color information and object identification information of each pixel point;
the region judgment module is used for judging whether the object identification information is a pixel point of the vehicle, and if so, the form operation module is executed;
the morphological operation module is used for performing morphological operation on the region of the pixel set of the vehicle with the object identification information, and obtaining a region to be identified from the region after the edge of the region is corroded and expanded;
the matting track module is used for inputting the multiple characteristic information of the pixel points in the region to be identified into a trained second convolutional neural network and outputting at least one matting track of the vehicle;
and the area matting module is used for matting the closed graph area surrounded by the track to be matting from the image to be identified.
9. A neural network-based vehicle matting device, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the neural network-based vehicle matting method of any one of claims 1 to 7 via execution of the executable instructions.
10. A computer readable storage medium storing a program, wherein the program when executed implements the steps of the neural network-based vehicle matting method according to any one of claims 1 to 7.
CN202110116600.4A 2021-01-28 2021-01-28 Vehicle matting method, system, equipment and storage medium based on neural network Pending CN112734778A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110116600.4A CN112734778A (en) 2021-01-28 2021-01-28 Vehicle matting method, system, equipment and storage medium based on neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110116600.4A CN112734778A (en) 2021-01-28 2021-01-28 Vehicle matting method, system, equipment and storage medium based on neural network

Publications (1)

Publication Number Publication Date
CN112734778A true CN112734778A (en) 2021-04-30

Family

ID=75594355

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110116600.4A Pending CN112734778A (en) 2021-01-28 2021-01-28 Vehicle matting method, system, equipment and storage medium based on neural network

Country Status (1)

Country Link
CN (1) CN112734778A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114387331A (en) * 2022-01-14 2022-04-22 平安科技(深圳)有限公司 Method, device and equipment for evaluating segmentation quality of vehicle image and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107909016A (en) * 2017-11-03 2018-04-13 车智互联(北京)科技有限公司 A kind of convolutional neural networks generation method and the recognition methods of car system
CN107945204A (en) * 2017-10-27 2018-04-20 西安电子科技大学 A kind of Pixel-level portrait based on generation confrontation network scratches drawing method
CN108399454A (en) * 2018-03-05 2018-08-14 山东领能电子科技有限公司 A kind of completely new sectional convolution neural network target recognition
CN108961279A (en) * 2018-06-28 2018-12-07 Oppo(重庆)智能科技有限公司 Image processing method, device and mobile terminal
CN110889855A (en) * 2019-10-28 2020-03-17 公安部交通管理科学研究所 Certificate photo matting method and system based on end-to-end convolutional neural network
CN111126248A (en) * 2019-12-20 2020-05-08 湖南千视通信息科技有限公司 Method and device for identifying shielded vehicle
CN111383232A (en) * 2018-12-29 2020-07-07 Tcl集团股份有限公司 Matting method, matting device, terminal equipment and computer-readable storage medium
WO2020186790A1 (en) * 2019-03-15 2020-09-24 平安科技(深圳)有限公司 Vehicle model identification method, device, computer apparatus, and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107945204A (en) * 2017-10-27 2018-04-20 西安电子科技大学 A kind of Pixel-level portrait based on generation confrontation network scratches drawing method
CN107909016A (en) * 2017-11-03 2018-04-13 车智互联(北京)科技有限公司 A kind of convolutional neural networks generation method and the recognition methods of car system
CN108399454A (en) * 2018-03-05 2018-08-14 山东领能电子科技有限公司 A kind of completely new sectional convolution neural network target recognition
CN108961279A (en) * 2018-06-28 2018-12-07 Oppo(重庆)智能科技有限公司 Image processing method, device and mobile terminal
CN111383232A (en) * 2018-12-29 2020-07-07 Tcl集团股份有限公司 Matting method, matting device, terminal equipment and computer-readable storage medium
WO2020186790A1 (en) * 2019-03-15 2020-09-24 平安科技(深圳)有限公司 Vehicle model identification method, device, computer apparatus, and storage medium
CN110889855A (en) * 2019-10-28 2020-03-17 公安部交通管理科学研究所 Certificate photo matting method and system based on end-to-end convolutional neural network
CN111126248A (en) * 2019-12-20 2020-05-08 湖南千视通信息科技有限公司 Method and device for identifying shielded vehicle

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114387331A (en) * 2022-01-14 2022-04-22 平安科技(深圳)有限公司 Method, device and equipment for evaluating segmentation quality of vehicle image and storage medium
CN114387331B (en) * 2022-01-14 2024-07-16 平安科技(深圳)有限公司 Method, device, equipment and storage medium for evaluating vehicle image segmentation quality

Similar Documents

Publication Publication Date Title
CN110176027B (en) Video target tracking method, device, equipment and storage medium
CN110503097A (en) Training method, device and the storage medium of image processing model
CN110245579B (en) People flow density prediction method and device, computer equipment and readable medium
CN112116599B (en) Sputum smear tubercle bacillus semantic segmentation method and system based on weak supervised learning
CN107909638B (en) Rendering method, medium, system and electronic device of virtual object
CN111563502A (en) Image text recognition method and device, electronic equipment and computer storage medium
CN112287941B (en) License plate recognition method based on automatic character region perception
CN112418216A (en) Method for detecting characters in complex natural scene image
CN112990331A (en) Image processing method, electronic device, and storage medium
CN110852327A (en) Image processing method, image processing device, electronic equipment and storage medium
CN113569852A (en) Training method and device of semantic segmentation model, electronic equipment and storage medium
CN112287144A (en) Picture retrieval method, equipment and storage medium
CN113762303A (en) Image classification method and device, electronic equipment and storage medium
CN117830537B (en) Weak supervision 3D scene graph generation method, device, equipment and medium
CN115100469A (en) Target attribute identification method, training method and device based on segmentation algorithm
CN116071557A (en) Long tail target detection method, computer readable storage medium and driving device
CN113223011B (en) Small sample image segmentation method based on guide network and full-connection conditional random field
CN111709762B (en) Information matching degree evaluation method, device, equipment and storage medium
CN111062388B (en) Advertisement character recognition method, system, medium and equipment based on deep learning
CN112734778A (en) Vehicle matting method, system, equipment and storage medium based on neural network
CN113744280A (en) Image processing method, apparatus, device and medium
CN111815748B (en) Animation processing method and device, storage medium and electronic equipment
CN112258416A (en) Image processing method and device and electronic equipment
CN112434591B (en) Lane line determination method and device
CN117121050A (en) Segmenting and removing objects from media items

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination