CN109816659A - Image partition method, apparatus and system - Google Patents

Abstract

The present invention provides a kind of image partition methods, apparatus and system, are related to technical field of image processing, this method comprises: obtaining target image to be split；Target image is input to main coding network, target image is performed the encoding operation by main coding network, obtains fisrt feature figure；The size of fisrt feature figure is amplified into presupposition multiple, obtains amplified fisrt feature figure；Amplified fisrt feature figure is input to auxiliary coding network, amplified fisrt feature figure is performed the encoding operation by auxiliary coding network, obtains second feature figure；Fisrt feature figure and second feature figure are input to decoding network, fisrt feature figure and second feature figure are merged by decoding network, the first fusion feature figure is obtained, and operation is decoded to the first fusion feature figure, obtains the segmentation result of target image.The present invention can effectively promote the accuracy of image segmentation result.

Description

Image partition method, apparatus and system

Technical field

The present invention relates to technical field of image processing, more particularly, to a kind of image partition method, apparatus and system.

Background technique

Image segmentation is a core technology of computer vision.With popularizing for deep learning, in unmanned, machine There are important role in the applications such as people's navigation, image recognition.The main purpose of image segmentation is each in determining image The classification of pixel, to be come out each object segmentation in image based on Pixel-level.

Traditional image partition method is usually that the image progress resulting high dimensional feature of down-sampling is amplified to original image Then size is directly based upon amplified image and obtains image segmentation result.The classification ability to express of this image segmentation mode It is poor, and easily ignore the minutia of each object in image, the accuracy of image segmentation result is not high.

Summary of the invention

In view of this, the purpose of the present invention is to provide a kind of image partition method, apparatus and system, to promote image point Cut the accuracy of result.

To achieve the goals above, technical solution used in the embodiment of the present invention is as follows:

In a first aspect, the embodiment of the invention provides a kind of image partition methods, comprising: obtain target figure to be split Picture；The target image is input to main coding network, coding behaviour is carried out to the target image by the main coding network Make, obtains fisrt feature figure；The size of the fisrt feature figure is amplified into presupposition multiple, obtains amplified fisrt feature figure； The amplified fisrt feature figure is input to auxiliary coding network, by the auxiliary coding network to described amplified first Characteristic pattern performs the encoding operation, and obtains second feature figure；The fisrt feature figure and the second feature figure are input to decoding Network merges the fisrt feature figure and the second feature figure by the decoding network, obtains the first fusion feature figure, and Operation is decoded to the first fusion feature figure, obtains the segmentation result of the target image.

Further, the main coding network includes sequentially connected image scaling sub-network, main down-sampling sub-network and master Feature association sub-network；It is described that the target image is performed the encoding operation by the main coding network, obtain fisrt feature The step of figure, comprising: sub-network is scaled for the size scaling of the target image to specified size by described image；Pass through institute It states main down-sampling sub-network and down-sampling operation is carried out to the target image for zooming to specified size, obtain main down-sampling feature Figure；Full attended operation is carried out to the main down-sampling characteristic pattern by the main feature association sub-network, and will be adopted under the master Sample characteristic pattern is merged with the main down-sampling characteristic pattern after full attended operation, obtains fisrt feature figure.

Further, described image scaling sub-network includes at least one layer of convolutional layer；The main down-sampling sub-network includes one A or multiple master file product groups；Multiple master file product groups are sequentially connected, spy of each master file product group for will input to it Sign figure is reduced to specified characteristic dimension, and the different corresponding specified characteristic dimensions of master file product group is different；And each institute Stating master file product group includes multiple convolutional layers；

The main feature association sub-network includes the full articulamentum of sequentially connected master and master file lamination, further includes that principal point multiplies fortune Calculate layer；The full articulamentum of master and master file lamination in the main feature association sub-network are used to carry out the main down-sampling characteristic pattern Full attended operation, the principal point multiplication layer are used for the main down-sampling characteristic pattern and the main down-sampling after full attended operation Characteristic pattern is merged, and fisrt feature figure is obtained.

Further, the auxiliary coding network includes M auxiliary code sets；Wherein, M be it is preset be not less than 1 natural number；Institute It states and the amplified fisrt feature figure is performed the encoding operation by the auxiliary coding network, obtain the step of second feature figure Suddenly, comprising:

If m is equal to 1, the amplified fisrt feature figure is based on by the 1st auxiliary code set and is performed the encoding operation, is obtained Amplify the presupposition multiple to the 1st auxiliary coding characteristic figure, and by described 1st auxiliary coding characteristic figure, obtains amplified m A auxiliary coding characteristic figure；If m is greater than 1, by m-th auxiliary code set be based on amplified m-1 auxiliary coding characteristic figures into Row encoding operation obtains m-th of auxiliary coding characteristic figure, and m-th of auxiliary coding characteristic figure is amplified the presupposition multiple, is put M-th of auxiliary coding characteristic figure after big；Wherein, the value of m successively takes from 2 to M-1；If m is equal to M, pass through the auxiliary code set of m-th It is performed the encoding operation based on amplified m-1 auxiliary coding characteristic figures, obtains the auxiliary coding characteristic figure of m-th；By m-th of auxiliary volume Code characteristic pattern is determined as second feature figure；Wherein, the value of m successively takes from 1 to M.

Further, each auxiliary code set includes sequentially connected auxiliary down-sampling sub-network and auxiliary feature association subnet Network；

If m be equal to 1, it is described pass through the 1st auxiliary code set be based on the amplified fisrt feature figure carry out coding behaviour The step of making, obtaining the 1st auxiliary coding characteristic figure, comprising: be based on by the auxiliary down-sampling sub-network of the 1st auxiliary code set The amplified fisrt feature figure carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern；It is auxiliary by described 1st The auxiliary feature association sub-network of code set is based on described 1st auxiliary down-sampling characteristic pattern and carries out full attended operation, and by the described 1st A auxiliary down-sampling characteristic pattern is merged with the 1st auxiliary down-sampling characteristic pattern after full attended operation, obtains the 1st auxiliary coding Characteristic pattern；

If m is greater than 1, it is described pass through m-th of auxiliary code set and be based on amplified m-1 auxiliary coding characteristic figures compiled The step of code operates, and obtains m-th of auxiliary coding characteristic figure, comprising: pass through the auxiliary down-sampling sub-network of described m-th auxiliary code set Down-sampling operation is carried out based on amplified m-1 auxiliary coding characteristic figures, obtains m-th of auxiliary down-sampling characteristic pattern；Pass through institute The auxiliary feature association sub-network for stating m-th of auxiliary code set carries out full attended operation to described m-th auxiliary down-sampling characteristic pattern, and will M-th of auxiliary down-sampling characteristic pattern is merged with m-th of auxiliary down-sampling characteristic pattern after full attended operation, obtains m A auxiliary coding characteristic figure.

Further, the main down-sampling sub-network of the main coding network, which also exports, main intermediate features figure；The auxiliary coding The auxiliary down-sampling sub-network of group, which also exports, auxiliary intermediate features figure；

If m is equal to 1, the auxiliary down-sampling sub-network by the 1st auxiliary code set is based on described amplified The step of fisrt feature figure carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern, comprising: by the main intermediate features Figure and the amplified fisrt feature figure are spliced, and the 1st splicing characteristic pattern is obtained；Pass through the 1st auxiliary code set Auxiliary down-sampling sub-network to the 1st splicing characteristic pattern carry out down-sampling operation, obtain the 1st auxiliary down-sampling characteristic pattern；

If m is greater than 1, the auxiliary down-sampling sub-network by described m-th auxiliary code set is based on amplified m-1 The step of a auxiliary coding characteristic figure carries out down-sampling operation, obtains m-th of auxiliary down-sampling characteristic pattern, comprising: by m-1 auxiliary volumes The auxiliary intermediate features figure of code character output is spliced with amplified m-1 auxiliary coding characteristic figures, obtains m-th of splicing feature Figure；Down-sampling operation is carried out to m-th of splicing characteristic pattern by the auxiliary down-sampling sub-network of described m-th auxiliary code set, Obtain m-th of auxiliary down-sampling characteristic pattern；Wherein, the value of m successively takes from 2 to M.

Further, the auxiliary down-sampling sub-network includes one or more auxiliary convolution groups；Multiple auxiliary convolution groups are successively Connection, each auxiliary convolution group are used to that specified characteristic dimension will to be reduced to its characteristic pattern inputted, and different is described auxiliary The corresponding specified characteristic dimension of convolution group is different；And each auxiliary convolution group includes multiple convolutional layers；

The auxiliary feature association sub-network includes sequentially connected auxiliary full articulamentum and auxiliary convolutional layer, further includes auxiliary dot product fortune Calculate layer；Auxiliary full articulamentum and auxiliary convolutional layer in the auxiliary feature association sub-network are used to carry out the auxiliary down-sampling characteristic pattern Full attended operation, the auxiliary point multiplication operation layer are used for the auxiliary down-sampling characteristic pattern and the auxiliary down-sampling after full attended operation Characteristic pattern is merged, and second feature figure is obtained.

Further, the master file that the main down-sampling sub-network includes accumulates the quantity of group and each auxiliary code set includes The quantity of auxiliary convolution group is N；Wherein, N is the natural number not less than 1；It is described by the 1st auxiliary code set it is auxiliary under adopt The step of appearance network carries out down-sampling operation to the 1st splicing characteristic pattern, obtains the 1st auxiliary down-sampling characteristic pattern, packet It includes:

If n is equal to 1, after the output characteristic pattern of the 1st master file product group in the main coding network and the amplification Fisrt feature figure spliced, obtain the 1st son splicing characteristic pattern, pass through the 1st auxiliary volume in the 1st auxiliary code set Product group carries out down-sampling operation to the 1st son splicing characteristic pattern, by the 1st auxiliary convolution in the 1st auxiliary code set The output characteristic pattern of group is determined as the 1st auxiliary intermediate features figure of the 1st auxiliary code set；

If n is greater than 1, by the output characteristic pattern of (n-1)th auxiliary convolution group in the 1st auxiliary code set and the main coding The output characteristic pattern of n-th of master file product group in network is spliced, and is obtained the 1n son and is spliced characteristic pattern, passes through described the N-th of auxiliary convolution group in 1 auxiliary code set carries out down-sampling operation to the 1n son splicing characteristic pattern, by described the The output characteristic pattern of n-th of auxiliary convolution group in 1 auxiliary code set is determined as n-th of auxiliary centre of the 1st auxiliary code set Characteristic pattern；Wherein, the value of n successively takes from 2 to N；

The auxiliary intermediate output figure of the n-th of the 1st auxiliary code set is determined as the 1st auxiliary down-sampling characteristic pattern.

Further, the auxiliary down-sampling sub-network by described m-th auxiliary code set is to m-th of splicing characteristic pattern The step of carrying out down-sampling operation, obtaining m-th of auxiliary down-sampling characteristic pattern, comprising:

If n is equal to 1, by the output characteristic pattern of the 1st auxiliary convolution group in m-1 auxiliary code sets and amplified the M-1 auxiliary coding characteristic figures are spliced, and the m1 son splicing characteristic pattern are obtained, by described m-th auxiliary code set 1st auxiliary convolution group carries out down-sampling operation to the m1 son splicing characteristic pattern, will be in described m-th auxiliary code set The output characteristic pattern of 1st auxiliary convolution group is determined as the 1st auxiliary intermediate features figure of described m-th auxiliary code set；

If n is greater than 1, by the output characteristic pattern of n-th of auxiliary convolution group in the m-1 auxiliary code sets and described The output characteristic pattern of (n-1)th auxiliary convolution group in m-th of auxiliary code set is spliced, and the mn son splicing feature is obtained Figure；Down-sampling is carried out to the mn son splicing characteristic pattern by n-th of auxiliary convolution group in described m-th auxiliary code set Operation, is determined as described m-th auxiliary code set for the output characteristic pattern of n-th of auxiliary convolution group in described m-th auxiliary code set N-th of auxiliary intermediate features figure；

The auxiliary intermediate output figure of the n-th of described m-th auxiliary code set is determined as m-th of auxiliary down-sampling characteristic pattern.

Further, the decoding network includes that fusion sub-network conciliates numeral network；The fusion sub-network is used for institute It states fisrt feature figure and the second feature figure is amplified to specified size, by the amplified fisrt feature figure and amplified The second feature figure is merged, and the first fusion feature figure is obtained；The decoding sub-network is used for special to first fusion Sign figure is decoded operation, obtains the segmentation result of the target image.

Further, the fusion sub-network includes multiple up-sampling layers and addition without carry operation layer；It is described to up-sample the defeated of layer Enter for the fisrt feature figure or the second feature figure；The input of different up-sampling layers is different；Each up-sampling layer For specified size will to be amplified to its characteristic pattern inputted, amplified fisrt feature figure or amplified second feature are obtained Figure；The addition without carry operation layer be used for by the amplified fisrt feature figure and the amplified second feature figure carry out by Position add operation, obtains the first fusion feature figure.

Further, described that the fisrt feature figure and the second feature figure are merged by the decoding network, obtain the The step of one fusion feature figure, comprising: will be in the output characteristic pattern and each auxiliary code set of first master file product group The output characteristic pattern of first auxiliary convolution group be input to the decoding network；First institute is merged by the decoding network State the output characteristic pattern of master file product group, the output characteristic pattern of first auxiliary convolution group in each auxiliary code set, described the One characteristic pattern and the second feature figure, obtain the first fusion feature figure.

Second aspect, the embodiment of the invention provides a kind of image segmentation devices, comprising: target image obtains module, uses In the target image that acquisition is to be split；Main coding module, for the target image to be input to main coding network, by described Main coding network performs the encoding operation the target image, obtains fisrt feature figure；Size amplification module, for by described the The size of one characteristic pattern amplifies presupposition multiple, obtains amplified fisrt feature figure；Auxiliary coding module, being used for will be after the amplification Fisrt feature figure be input to auxiliary coding network, the amplified fisrt feature figure is compiled by the auxiliary coding network Code operation, obtains second feature figure；Decoder module, for the fisrt feature figure and the second feature figure to be input to decoding Network merges the fisrt feature figure and the second feature figure by the decoding network, obtains the first fusion feature figure, and Operation is decoded to the first fusion feature figure, obtains the segmentation result of the target image.

The third aspect, the embodiment of the invention provides a kind of image segmentation system, the system comprises: image collector It sets, processor and storage device；Described image acquisition device, for acquiring target image；Meter is stored on the storage device Calculation machine program, the computer program execute such as the described in any item methods of first aspect when being run by the processor.

Fourth aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage Computer program is stored on medium, the computer program is executed when being run by processor described in above-mentioned any one of first aspect Method the step of.

The embodiment of the invention provides a kind of image partition methods, apparatus and system, by constructing main coding network-auxiliary volume Code network-decoding network structure carries out first encoding operation to target image by main coding network first, obtains the first spy Sign figure；It is input to auxiliary coding network after fisrt feature figure is done enhanced processing again, by auxiliary coding network to amplified first Characteristic pattern carries out encoding operation again, obtains second feature figure；And then fisrt feature figure and second feature figure are input to decoding Network merges fisrt feature figure and second feature figure by decoding network, obtains the first fusion feature figure, then melt again to first It closes characteristic pattern and is decoded operation, obtain the segmentation result of target image.This mode is based on multiple coding networks (main coding net Network and auxiliary coding network) target image is carried out repeatedly to encode the macroscopic information that can effectively extract each object in image, it helps In promotion classification ability to express；And it is merged to resulting characteristic pattern (fisrt feature figure and second feature figure) is repeatedly encoded Afterwards, fusion feature figure is decoded, obtains segmentation result, facilitate the minutia for effectively going back each object in original image, from And it can effectively promote the accuracy of image segmentation.

Other features and advantages of the present invention will illustrate in the following description, also, partly become from specification It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention are in specification, claims And specifically noted structure is achieved and obtained in attached drawing.

To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate Appended attached drawing, is described in detail below.

Detailed description of the invention

It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art Embodiment or attached drawing needed to be used in the description of the prior art be briefly described, it should be apparent that, it is described below Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor It puts, is also possible to obtain other drawings based on these drawings.

Fig. 1 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention；

Fig. 2 is a kind of flow chart of image partition method provided in an embodiment of the present invention；

Fig. 3 is the structural schematic diagram of the first Image Segmentation Model provided in an embodiment of the present invention；

Fig. 4 is the structural schematic diagram of second of Image Segmentation Model provided in an embodiment of the present invention；

Fig. 5 is the structural schematic diagram of the third Image Segmentation Model provided in an embodiment of the present invention；

Fig. 6 is the structural schematic diagram of the 4th kind of Image Segmentation Model provided in an embodiment of the present invention；

Fig. 7 is the structural schematic diagram of the 5th kind of Image Segmentation Model provided in an embodiment of the present invention；

Fig. 8 is the structural schematic diagram of the 6th kind of Image Segmentation Model provided in an embodiment of the present invention；

Fig. 9 is a kind of structural schematic diagram of auxiliary feature association sub-network provided in an embodiment of the present invention；

Figure 10 is the structural schematic diagram of the 7th kind of Image Segmentation Model provided in an embodiment of the present invention；

Figure 11 is the structural schematic diagram of the 8th kind of Image Segmentation Model provided in an embodiment of the present invention；

Figure 12 is the structural schematic diagram of the 9th kind of Image Segmentation Model provided in an embodiment of the present invention；

Figure 13 is the structural schematic diagram of the provided in an embodiment of the present invention ten kind of Image Segmentation Model；

Figure 14 is a kind of structural schematic diagram of main feature association sub-network provided in an embodiment of the present invention；

Figure 15 is a kind of Contrast on effect schematic diagram provided in an embodiment of the present invention；

Figure 16 is another Contrast on effect schematic diagram provided in an embodiment of the present invention；

Figure 17 is a kind of structural block diagram of image segmentation device provided in an embodiment of the present invention.

Specific embodiment

In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention Technical solution be clearly and completely described, it is clear that described embodiments are some of the embodiments of the present invention, rather than Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise Under every other embodiment obtained, shall fall within the protection scope of the present invention.

Two key factors for influencing image segmentation result are the classification ability to express and space expression energy of neural network Power.The Main of existing image Segmentation Technology is to design the neural network comprising coding layer and decoding layer, to coding The image is encoded to one group of high dimensional feature by coding layer by layer one image of input, and high dimensional feature correspondence is originally inputted figure As the characteristic pattern after many times of down-sampling；Decoding layer reverts to the corresponding characteristic pattern of the high dimensional feature and original image size Output after identical.The classification ability to express of this image segmentation mode is poor, and the details for easily ignoring each object in image is special Sign, the accuracy of image segmentation result be not high.If better classification capacity may be implemented using deeper network structure, but Since too deep network structure frequently can lead to that characteristics of image resolution ratio is too low, and then lose the descriptive power in space.

Based on discussed above, the reliability of existing image partition method is poor, and the accuracy of image segmentation result is not high. To improve this problem, the embodiment of the invention provides a kind of image partition method, apparatus and system, which can be applied to nothing People drives, robot navigation, the image segmentation task in any type such as image recognition field, below to the embodiment of the present invention into Row is discussed in detail.

Embodiment one:

Firstly, describing a kind of image partition method for realizing the embodiment of the present invention, apparatus and system referring to Fig.1 Exemplary electronic device 100.

The structural schematic diagram of a kind of electronic equipment as shown in Figure 1, electronic equipment 100 include one or more processors 102, one or more storage devices 104, input unit 106, output device 108 and image collecting device 110, these components It is interconnected by bindiny mechanism's (not shown) of bus system 112 and/or other forms.It should be noted that electronic equipment shown in FIG. 1 100 component and structure be it is illustrative, and not restrictive, as needed, the electronic equipment also can have other Component and structure.

The processor 102 can use digital signal processor (DSP), field programmable gate array (FPGA), can compile At least one of journey logic array (PLA) example, in hardware realizes that the processor 102 can be central processing unit (CPU) or one or more of the processing unit of other forms with data-handling capacity and/or instruction execution capability Combination, and can control other components in the electronic equipment 100 to execute desired function.

The storage device 104 may include one or more computer program products, and the computer program product can To include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easy The property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non- Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage medium On can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter institute The client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meter Can also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/or The various data etc. generated.

The input unit 106 can be the device that user is used to input instruction, and may include keyboard, mouse, wheat One or more of gram wind and touch screen etc..

The output device 108 can export various information (for example, image or sound) to external (for example, user), and It and may include one or more of display, loudspeaker etc..

Described image acquisition device 110 can shoot the desired image of user (such as photo, video etc.), and will be clapped The image taken the photograph is stored in the storage device 104 for the use of other components.

Illustratively, for realizing image partition method according to an embodiment of the present invention, the exemplary electron of apparatus and system Equipment may be implemented as the intelligent terminals such as smart phone, tablet computer, computer, capture machine.

Embodiment two:

A kind of image partition method flow chart shown in Figure 2, this method can be set by the electronics that previous embodiment provides Standby to execute, this method specifically comprises the following steps:

Step S202 obtains target image to be split；It wherein, include target object to be split in target image. Such as, having been may include in the animals such as bird or cat to be identified or the target image in the target image to include Vehicle to be identified, pedestrian, house etc..

Target image is input to main coding network, is encoded by main coding network to target image by step S204 Operation, obtains fisrt feature figure.Wherein, which mainly includes dimensionality reduction (also referred to as, down-sampling) operation, the mesh of encoding operation Be by by target image dimensionality reduction, to extract the gross feature structure in target image, consequently facilitating to target image Region locating for middle different objects is divided.

The size of fisrt feature figure is amplified presupposition multiple, obtains amplified fisrt feature figure by step S206.Amplification Multiple can flexible setting according to demand, depend primarily on the ruler of the full size and target image of target image after dimensionality reduction It is very little, additionally depend on the accessible picture size of auxiliary coding network.

Amplified fisrt feature figure is input to auxiliary coding network by step S208, by auxiliary coding network to amplification after Fisrt feature figure perform the encoding operation, obtain second feature figure.Different from conventional image segmentation mode, the present embodiment is also set Auxiliary coding network has been set, auxiliary coding network can further execute encoding operation based on amplified fisrt feature figure, thus into Characteristic information in onestep extraction image, the rich of information extraction help to promote classification ability to express.This mode can be compared with The feature multiplexing of good realization network level, makes network obtain the tagsort ability of deep layer.

Fisrt feature figure and second feature figure are input to decoding network by step S210, pass through decoding network fusion first Characteristic pattern and second feature figure, obtain the first fusion feature figure, and be decoded operation to the first fusion feature figure, obtain target The segmentation result of image；Wherein, decoding operate mainly includes that a liter dimension (also referred to as, up-samples) operation, and the purpose of decoding operate is Replenish the microscale features in target image lost during performing the encoding operation to target image.

Above-mentioned image partition method provided in an embodiment of the present invention is based on multiple coding networks (main coding network and auxiliary volume Code network) target image is carried out repeatedly to encode the macroscopic information that can effectively extract each object in image, facilitate promotion point Class ability to express；And to repeatedly encoding after resulting characteristic pattern (fisrt feature figure and second feature figure) merges, to fusion Characteristic pattern is decoded, and obtains segmentation result, facilitates the minutia for effectively going back each object in original image, so as to effective Promote the accuracy of image segmentation.

In the specific implementation, Image Segmentation Model can be constructed in advance, and Image Segmentation Model can be based on above-mentioned image segmentation Method is realized.The structural schematic diagram of the first Image Segmentation Model shown in Figure 3 illustrates that the Image Segmentation Model includes Main coding network, auxiliary coding network and decoding network；First encoding operation output is carried out to target image by main coding network Fisrt feature figure does the fisrt feature figure that main coding network exports enhanced processing (in Fig. 3, indicating with symbol "×"), then will Input of the amplified fisrt feature figure as auxiliary coding network carries out amplified fisrt feature figure by auxiliary coding network Encoding operation again realizes the other coding of sub-pixel.The first Image Segmentation Model provided in an embodiment of the present invention takes net The mode of network feature multiplexing, the output after directly not amplifying the high dimensional feature obtained by coding as model, but It will further be input to next coding network after this feature amplification and (amplified fisrt feature figure is inputted into auxiliary coding net Network), it can further obtain the tagsort ability of deep layer.

For ease of understanding, aforementioned main coding network is described in detail first.Second of figure as shown in Figure 4 As the structural schematic diagram of parted pattern, a kind of structure of above-mentioned main coding network, the main coding are illustrated on the basis of Fig. 3 Network includes sequentially connected image scaling sub-network, main down-sampling sub-network and main feature association sub-network.

Based on this, target image is performed the encoding operation by main coding network in above-mentioned steps S204, obtains the first spy A kind of specific executive mode for levying figure can be with are as follows: by image scaling sub-network by the size scaling of target image to specified ruler It is very little；Down-sampling operation is carried out to the target image for zooming to specified size by main down-sampling sub-network, it is special to obtain main down-sampling Sign figure；Full attended operation carried out to main down-sampling characteristic pattern by main feature association sub-network, and by main down-sampling characteristic pattern with Main down-sampling characteristic pattern after full attended operation is merged, and fisrt feature figure is obtained.

When in view of practical application, target image to be split is usually big figure, above-mentioned side provided in an embodiment of the present invention Formula is first reduced target image based on the image scaling sub-network in main coding network, to reduce subsequent down-sampling, complete The calculation amount of the image processing process such as connection and mixing operation facilitates the speed for promoting image segmentation.Based on main coding network In main feature association sub-network main down-sampling characteristic pattern that main down-sampling sub-network is exported carry out full attended operation after, and will Main down-sampling characteristic pattern is merged with the main down-sampling characteristic pattern after full attended operation, can reinforce each feature in image Correlation.

For ease of understanding, this gives a kind of specific embodiment of auxiliary coding network, above-mentioned auxiliary coding networks It may include M auxiliary code sets；Wherein, M be it is preset be not less than 1 natural number；Based on this structure, with nature number variable m table Show that any auxiliary code set in M auxiliary code sets, the value range of m are 1≤m≤M.Pass through auxiliary coding net in above-mentioned steps S208 Network performs the encoding operation amplified fisrt feature figure, and obtaining second feature figure can refer to following manner implementation:

If m is equal to 1, amplified fisrt feature figure is based on by the 1st auxiliary code set and is performed the encoding operation, obtains the 1 auxiliary coding characteristic figure, and the 1st auxiliary coding characteristic figure is amplified into presupposition multiple, obtain amplified m-th auxiliary coding characteristic Figure；

If m is greater than 1, amplified m-1 auxiliary coding characteristic figures are based on by m-th of auxiliary code set and carry out coding behaviour Make, obtains m-th of auxiliary coding characteristic figure, and m-th of auxiliary coding characteristic figure is amplified into presupposition multiple, obtain amplified m-th Auxiliary coding characteristic figure；Wherein, the value of m successively takes from 2 to M-1；

If m is equal to M, amplified m-1 auxiliary coding characteristic figures are based on by the auxiliary code set of m-th and carry out coding behaviour Make, obtains the auxiliary coding characteristic figure of m-th；

M-th of auxiliary coding characteristic figure is determined as second feature figure；Wherein, the value of m successively takes from 1 to M.

The structural schematic diagram of the third Image Segmentation Model as shown in Figure 5 illustrates auxiliary coding on the basis of fig. 4 Illustrate that the auxiliary coding network includes the first auxiliary code set and the second auxiliary code set in a kind of specific structure namely Fig. 5 of network. Corresponding aforesaid way, the value of m are 1 and 2, are based on amplified fisrt feature figure by the first auxiliary code set and perform the encoding operation The 1st auxiliary coding characteristic figure is obtained, amplified 1st auxiliary coding characteristic figure is based on by the second code set and is performed the encoding operation Obtain the 2nd auxiliary coding characteristic figure；1st auxiliary coding characteristic figure and the 2nd auxiliary coding characteristic figure are determined as second feature Figure, namely second feature figure includes that the 1st auxiliary coding of the first auxiliary code set output is special in Fig. 5 provided in an embodiment of the present invention Sign figure and the 2nd auxiliary coding characteristic figure of the second auxiliary code set output.

In a kind of specific embodiment, each auxiliary code set includes sequentially connected auxiliary down-sampling sub-network and auxiliary Feature association sub-network, the structural schematic diagram of the 4th kind of Image Segmentation Model as shown in FIG. 6, specifically shows on the basis of Fig. 5 The the first auxiliary down-sampling sub-network and the first auxiliary feature association sub-network and the second auxiliary volume that first auxiliary code set of having anticipated out includes The the second auxiliary down-sampling sub-network and the second auxiliary feature association sub-network that code character includes.It is illustrated according to the value point situation of m, The specific executive mode that different auxiliary code sets obtains corresponding auxiliary coding characteristic figure is as follows:

If m is equal to 1, amplified fisrt feature figure is based on above by the 1st auxiliary code set and is performed the encoding operation, is obtained The step of to the 1st auxiliary coding characteristic figure, comprising: amplification is based on by the auxiliary down-sampling sub-network of the 1st auxiliary code set first Fisrt feature figure afterwards carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern；Then pass through the 1st auxiliary code set Auxiliary feature association sub-network is based on the 1st auxiliary down-sampling characteristic pattern and carries out full attended operation, and by the 1st auxiliary down-sampling characteristic pattern It is merged with the 1st auxiliary down-sampling characteristic pattern after full attended operation, obtains the 1st auxiliary coding characteristic figure.

If m is greater than 1, amplified m-1 auxiliary coding characteristic figures are based on above by m-th of auxiliary code set and are compiled The step of code operates, and obtains m-th of auxiliary coding characteristic figure, comprising: pass through the auxiliary down-sampling sub-network of m-th of auxiliary code set first Down-sampling operation is carried out based on amplified m-1 auxiliary coding characteristic figures, obtains m-th of auxiliary down-sampling characteristic pattern；Then lead to It crosses the auxiliary feature association sub-network of m-th of auxiliary code set and full attended operation is carried out to m-th of auxiliary down-sampling characteristic pattern, and by m A auxiliary down-sampling characteristic pattern is merged with m-th of auxiliary down-sampling characteristic pattern after full attended operation, obtains m-th of auxiliary coding Characteristic pattern.

In Fig. 6, the input of the first auxiliary down-sampling sub-network is amplified fisrt feature figure, the first auxiliary down-sampling subnet The output of network is the 1st auxiliary down-sampling characteristic pattern；The input of first auxiliary feature association sub-network is the 1st auxiliary down-sampling feature Figure, the output of the first auxiliary feature association sub-network are the 1st auxiliary coding characteristic figure.The input of second auxiliary down-sampling sub-network is to put The 1st auxiliary coding characteristic figure after big, the output of the second auxiliary down-sampling sub-network are the 2nd auxiliary down-sampling characteristic pattern；Second is auxiliary The input of feature association sub-network is the 2nd auxiliary down-sampling characteristic pattern, and the output of the second auxiliary feature association sub-network is the 2nd auxiliary Coding characteristic figure.

Further to promote the classification ability to express of image segmentation, and deeper network structure is avoided to influence feature resolution Rate, under the premise of not changing Image Segmentation Model structure, the embodiment of the present invention is on the basis of above-mentioned network characterization is multiplexed, also A kind of hierarchy characteristic multiplexing method is provided, the embodiment of the invention provides the images of a variety of applications hierarchy characteristic multiplexing method Parted pattern (the 5th kind to the tenth kind), is described as follows:

As shown in fig. 7, the embodiment of the invention provides the structures of the 5th kind of Image Segmentation Model, on the basis of Fig. 6 also Illustrate multiple splicing layers, in Fig. 7 withTo indicate that splicing layer, splicing layer are used for the characteristic pattern of different levels.Chief editor The main down-sampling sub-network of code network, which also exports, main intermediate features figure；The auxiliary down-sampling sub-network of above-mentioned auxiliary code set also exports There is auxiliary intermediate features figure.The main middle graph characteristic pattern and amplified first that splicing layer is then used to export main down-sampling sub-network The auxiliary intermediate features figure and amplified 1st auxiliary coding characteristic figure of characteristic pattern or the output of the first auxiliary down-sampling sub-network into Row is spliced to form corresponding splicing characteristic pattern, by the way that the merging features (also referred to as, connecting) of different levels are reused together, Such mode can increase the complexity of model while not increasing additional calculation amount, help to promote image segmentation mould The classification ability to express and spatial description ability of type, the splitting speed preferably balanced and accuracy.

Based on this, if m is equal to 1, it is special that amplified first is based on by the auxiliary down-sampling sub-network of the 1st auxiliary code set The step of sign figure carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern, comprising:

Main intermediate features figure and amplified fisrt feature figure are spliced, the 1st splicing characteristic pattern is obtained；Pass through The auxiliary down-sampling sub-network of 1 auxiliary code set carries out down-sampling operation to the 1st splicing characteristic pattern, obtains the 1st auxiliary down-sampling Characteristic pattern；

If m is greater than 1, amplified m-1 auxiliary codings are based on by the auxiliary down-sampling sub-network of m-th of auxiliary code set The step of characteristic pattern carries out down-sampling operation, obtains m-th of auxiliary down-sampling characteristic pattern, comprising: export m-1 auxiliary code sets Auxiliary intermediate features figure and amplified m-1 auxiliary coding characteristic figures spliced, obtain m-th of splicing characteristic pattern；Pass through The auxiliary down-sampling sub-network of m-th of auxiliary code set carries out down-sampling operation to m-th splicing characteristic pattern, obtain m-th it is auxiliary under adopt Sample characteristic pattern；Wherein, the value of m successively takes from 2 to M.

Further, the embodiment of the invention provides a kind of specific embodiments of hierarchy characteristic multiplexing.Above-mentioned image scaling Sub-network includes at least one layer of convolutional layer；Main down-sampling sub-network includes one or more master file product groups；Multiple master files product groups according to Secondary connection, each master file product group is for will be reduced to specified characteristic dimension, different master file product groups to its characteristic pattern inputted Corresponding specified characteristic dimension is different；And each master file product group includes multiple convolutional layers；Main feature association sub-network include according to The full articulamentum of the master of secondary connection and master file lamination further include principal point multiplication layer；Master in main feature association sub-network connects entirely Layer and master file lamination are used to carry out main down-sampling characteristic pattern full attended operation, and principal point multiplication layer is used for main down-sampling feature Figure is merged with the main down-sampling characteristic pattern after full attended operation, obtains fisrt feature figure.

Above-mentioned auxiliary down-sampling sub-network includes one or more auxiliary convolution groups；Multiple auxiliary convolution groups are sequentially connected, Mei Gefu Convolution group is used to that specified characteristic dimension, the different corresponding specified spies of auxiliary convolution group will to be reduced to its characteristic pattern inputted It is different to levy dimension；And each auxiliary convolution group includes multiple convolutional layers；Auxiliary feature association sub-network includes sequentially connected auxiliary connects entirely Layer and auxiliary convolutional layer are connect, further includes auxiliary point multiplication operation layer；Auxiliary full articulamentum and auxiliary convolutional layer in auxiliary feature association sub-network are used In carrying out full attended operation to auxiliary down-sampling characteristic pattern, auxiliary point multiplication operation layer is used to auxiliary down-sampling characteristic pattern connecting behaviour with through complete Auxiliary down-sampling characteristic pattern after work is merged, and second feature figure is obtained.

For ease of understanding, the embodiment of the invention provides the structural representations of the 6th kind of Image Segmentation Model as shown in Figure 8 Figure, the multiple master files product group illustrated in detail in main down-sampling sub-network on the basis of Fig. 7 (show three masters in Fig. 8 Convolution group, respectively the first master file accumulate group, the second master file product group and third master file product group), the first auxiliary down-sampling sub-network it is more A master file product group (shows three auxiliary convolution groups, convolution group 1-1, auxiliary convolution group 1-2 and auxiliary convolution group 1- supplemented by difference in Fig. 8 3), multiple auxiliary convolution groups of the second auxiliary down-sampling sub-network (show three auxiliary convolution groups, convolution group 2- supplemented by difference in Fig. 8 1, auxiliary convolution group 2-2 and auxiliary convolution group 2-3).In addition, the embodiment of the present invention is by taking auxiliary feature association sub-network as an example, it is detailed in Fig. 9 Carefully show a kind of structure of auxiliary feature association sub-network.

Based on the 6th kind of Image Segmentation Model shown in Fig. 8, in a kind of optional embodiment, main down-sampling sub-network The quantity for the auxiliary convolution group that the quantity and each auxiliary code set for the master file product group for including include is N；Wherein, N is not less than 1 Natural number.The specific execution that application level feature of the embodiment of the present invention is multiplexed thought is as follows:

For ease of understanding, any one master file in N number of master file product group (alternatively, auxiliary code set) is indicated with nature number variable n Product group (alternatively, auxiliary code set), the value range of n are 1≤n≤N.

(1) down-sampling is carried out to the 1st splicing characteristic pattern above by the auxiliary down-sampling sub-network of the 1st auxiliary code set Operation, obtaining the 1st auxiliary down-sampling characteristic pattern can refer under type implementation such as:

If n is equal to 1, by the output characteristic pattern of the 1st master file product group in main coding network and amplified first spy Sign figure is spliced, and the 1st son splicing characteristic pattern is obtained, by the 1st auxiliary convolution group in the 1st auxiliary code set to the 1st Son splicing characteristic pattern carries out down-sampling operation, and the output characteristic pattern of the 1st auxiliary convolution group in the 1st auxiliary code set is determined as 1st auxiliary intermediate features figure of the 1st auxiliary code set；

If n is greater than 1, by the output characteristic pattern of (n-1)th auxiliary convolution group in the 1st auxiliary code set and main coding network In n-th of master file product group output characteristic pattern spliced, obtain the 1n son splice characteristic pattern, pass through the 1st auxiliary volume N-th of auxiliary convolution group in code character carries out down-sampling operation to the 1n son splicing characteristic pattern, will be in the 1st auxiliary code set The output characteristic pattern of n-th of auxiliary convolution group is determined as n-th of auxiliary intermediate features figure of the 1st auxiliary code set；Wherein, the value of n is from 2 It successively takes to N；

The auxiliary intermediate output figure of the n-th of 1st auxiliary code set is determined as the 1st auxiliary down-sampling characteristic pattern.

In the specific implementation, reference can be made to the structural schematic diagram of the 6th kind of Image Segmentation Model shown in Fig. 8, the value of N are 3, then the value range of n is 1≤n≤3.Auxiliary convolution group 1-1 (that is, the 1st auxiliary convolution group in the 1st auxiliary code set) it is defeated Enter for the output characteristic pattern and amplified first of the first master file product group (that is, the 1st master file product group in main coding network) The resulting 1st son splicing characteristic pattern of characteristic pattern splicing, the output of auxiliary convolution group 1-1 is during the 1st of the 1st auxiliary code set is auxiliary Between characteristic pattern.The input of auxiliary convolution group 1-2 (that is, the 2nd auxiliary convolution group in the 1st auxiliary code set) is the second master file product group (that is, in main coding network the 2nd master file product group) output characteristic pattern and auxiliary convolution group 1-1 output characteristic pattern (that is, 1st auxiliary intermediate features figure of the 1st auxiliary code set) the resulting 12nd son splicing characteristic pattern of splicing, auxiliary convolution group 1-2's Output is the 2nd auxiliary intermediate features figure of the 1st auxiliary code set.Auxiliary convolution group 1-3 is (that is, the in the 1st auxiliary code set the 3rd A auxiliary convolution group) input be third master file product group (that is, in main coding network the 3rd master file product group) output characteristic pattern With the output characteristic pattern (that is, the 2nd auxiliary intermediate features figure of the 1st auxiliary code set) of auxiliary convolution group 1-2 splicing resulting the 13 son splicing characteristic patterns, the output of auxiliary convolution group 1-2 is the 3rd auxiliary intermediate features figure of the 1st auxiliary code set.

(2) down-sampling is carried out to m-th of splicing characteristic pattern above by the auxiliary down-sampling sub-network of m-th of auxiliary code set Operation, obtaining m-th of auxiliary down-sampling characteristic pattern can refer under type implementation such as:

If n is equal to 1, by the output characteristic pattern of the 1st auxiliary convolution group in m-1 auxiliary code sets and amplified the M-1 auxiliary coding characteristic figures are spliced, and are obtained the m1 son splicing characteristic pattern, are passed through the 1st in m-th of auxiliary code set Auxiliary convolution group carries out down-sampling operation to the m1 son splicing characteristic pattern, by the 1st auxiliary convolution group in m-th of auxiliary code set Output characteristic pattern be determined as the 1st auxiliary intermediate features figure of m-th of auxiliary code set；

It is if n is greater than 1, the output characteristic pattern of n-th of auxiliary convolution group in m-1 auxiliary code sets and m-th is auxiliary The output characteristic pattern of (n-1)th auxiliary convolution group in code set is spliced, and the mn son splicing characteristic pattern is obtained；Pass through N-th of auxiliary convolution group in m auxiliary code sets carries out down-sampling operation to the mn son splicing characteristic pattern, by m-th of auxiliary volume The output characteristic pattern of n-th of auxiliary convolution group in code character is determined as n-th of auxiliary intermediate features figure of m-th of auxiliary code set；

The auxiliary intermediate output figure of the n-th of m-th of auxiliary code set is determined as m-th of auxiliary down-sampling characteristic pattern.

6th kind of Image Segmentation Model as shown in Figure 8, auxiliary convolution group 2-1 is (that is, the 1st in the 2nd auxiliary code set Auxiliary convolution group) input supplemented by convolution group 1-1 (that is, the 1st auxiliary convolution group in the 1st auxiliary code set) output characteristic pattern With the amplified 1st auxiliary resulting 21st son splicing characteristic pattern of coding characteristic figure splicing, the output of auxiliary convolution group 2-1 is 1st auxiliary intermediate features figure of the 2nd auxiliary code set.Auxiliary convolution group 2-2 is (that is, the 2nd auxiliary volume in the 2nd auxiliary code set Product group) input supplemented by convolution group 1-2 (that is, in main coding network the 2nd master file product group) output characteristic pattern and auxiliary volume Resulting 22nd of output characteristic pattern (that is, the 1st auxiliary intermediate features figure of the 2nd auxiliary code set) splicing of product group 2-1 Son splicing characteristic pattern, the output of auxiliary convolution group 2-2 are the 2nd auxiliary intermediate features figure of the 2nd auxiliary code set.Auxiliary convolution group 2-3 Convolution group 1-3 is (that is, in main coding network supplemented by the input of (that is, the 3rd auxiliary convolution group in the 2nd auxiliary code set) 3rd master file product group) output characteristic pattern and auxiliary convolution group 2-2 output characteristic pattern (that is, the 2nd of the 2nd auxiliary code set Auxiliary intermediate features figure) the resulting 23rd son splicing characteristic pattern of splicing, the output of auxiliary convolution group 2-3 is the 2nd auxiliary code set The 3rd auxiliary intermediate features figure.

For ease of understanding, above-mentioned decoding network is described in detail in the embodiment of the present invention, and shown in Figure 10 The structural schematic diagram of seven kinds of Image Segmentation Models, above-mentioned decoding network include that fusion sub-network conciliates numeral network；Merge subnet Network is used to fisrt feature figure and second feature figure being amplified to specified size, by amplified fisrt feature figure and amplified the Two characteristic patterns are merged, and the first fusion feature figure is obtained；Decoding sub-network is for being decoded behaviour to the first fusion feature figure Make, obtains the segmentation result of target image.

As shown in Figure 10, merging includes multiple up-sampling layers (the respectively first up-sampling layer, the second up-sampling in sub-network Layer, third up-sample layer) and addition without carry operation layer；The input for up-sampling layer is fisrt feature figure or second feature figure；Different The input for up-sampling layer is different；Wherein, second feature figure includes the output characteristic pattern and second of the first auxiliary feature association sub-network The output characteristic pattern of auxiliary feature association sub-network.As shown in Figure 10, the input of the first up-sampling layer is the second auxiliary feature association The input of the output characteristic pattern of network, the second up-sampling layer is the output characteristic pattern of the first auxiliary feature association sub-network, in third The input of sample level is the output characteristic pattern namely fisrt feature figure of main feature association sub-network.Each up-sampling layer is used for will The characteristic pattern inputted to it is amplified to specified size, when it is implemented, the side such as deconvolution, bilinear interpolation can be used in up-sampling layer The characteristic pattern of its opposite input of formula carries out up-sampling operation, to obtain specified size is amplified to its characteristic pattern inputted Amplified fisrt feature figure or amplified second feature figure；Addition without carry operation layer be used for amplified fisrt feature figure and Amplified second feature figure carries out step-by-step add operation, obtains the first fusion feature figure.

To further increase the complexity of decoding network to promote the space expressive faculty of image segmentation result feature, this hair Bright embodiment additionally provides in a kind of above-mentioned steps S210 and merges fisrt feature figure and second feature figure by decoding network, obtains The specific embodiment of first fusion feature figure is as follows:

The output of first auxiliary convolution group in the output characteristic pattern and each auxiliary code set of first master file product group is special Sign figure is input to decoding network；Pass through the output characteristic pattern of first master file product group of decoding network fusion, each auxiliary code set In first auxiliary convolution group output characteristic pattern, fisrt feature figure and second feature figure, obtain the first fusion feature figure.

The acquisition pattern of corresponding above-mentioned first fusion feature figure, referring to Figure 11, the embodiment of the invention provides the 8th kind of figures As the structural schematic diagram of parted pattern, illustrated in figure by the first master file product group (that is, above-mentioned first master file accumulates group) Export characteristic pattern, auxiliary convolution group 1-1 (namely first auxiliary convolution group in the 1st auxiliary code set) in the first auxiliary code set Export characteristic pattern, auxiliary convolution group 2-1 (namely first auxiliary convolution group in the 2nd auxiliary code set) in the second auxiliary code set Export characteristic pattern, the output characteristic pattern (that is, fisrt feature figure) of main feature association sub-network, the first auxiliary feature association sub-network Decoding network is input to the output characteristic pattern (that is, second feature figure) of the second auxiliary feature association sub-network merge To the first fusion feature figure.

The fusion specifically, structural schematic diagram of the 9th kind of Image Segmentation Model shown in Figure 12, in decoding network Sub-network includes 6 up-sampling layers, is respectively up-sampled with the second auxiliary feature association sub-network output characteristic pattern corresponding first Layer up-samples layer with the first auxiliary feature association sub-network output characteristic pattern corresponding second, exports with main feature association sub-network The corresponding third up-sampling layer of characteristic pattern, the 4th up-sampling layer corresponding with auxiliary convolution group 2-1 output characteristic pattern and auxiliary convolution group 1-1 exports the corresponding 5th up-sampling layer of characteristic pattern and the 6th up-sampling corresponding with the first master file product group output characteristic pattern Layer.Addition without carry operation layer in Figure 12 is used to 6 up-sampling layers in fused sub-network respectively corresponding the amplified of output First master file accumulates the output feature of the output characteristic pattern organized, first auxiliary convolution group in amplified each auxiliary code set Figure, amplified fisrt feature figure and amplified second feature figure carry out step-by-step add operation, obtain the first fusion feature figure.

For existing complex model, above-mentioned image corresponding with image partition method provided in this embodiment Parted pattern structure is smaller, and the coding network and decoding network structure of building are relatively simple, will not be when the mesh to be split received Image segmentation performance is reduced when logo image is big figure；And by encoding-operation process use network level feature multiplexing and/or The mode of hierarchy characteristic multiplexing, does not increase additional calculation amount, enhances the complexity of model while guaranteeing fast speed Degree.Moreover, the embodiment of the present invention can splice the intermediate features generated in an encoding process, again to spliced feature Secondary coding, is capable of the performance of more effective lifting feature description, namely promotes the classification ability to express of image segmentation and space is retouched State ability；In addition, amplified characteristic patterns multiple in coding network are merged by decoding network, further enhance decoding Ability improves the performance of Image Segmentation Model.It is greatly improved based on above-mentioned image partition method and image partition method The speed and accuracy of image segmentation.

Embodiment three:

Based on the image partition method that embodiment two provides, the embodiment of the present invention is to the target having a size of 1024 × 1024 For image is split, it is described in detail.Referring first to Figure 13, the embodiment of the invention provides the tenth kind of image segmentation moulds The structural schematic diagram of type, on the basis of Figure 12 it is detailed illustrate in Image Segmentation Model structure applied by each section and Its dimension for exporting characteristic pattern.

Specifically, comprising conv1 (that is, earlier figures in main coding network (that is, backbone frame shown in Figure 13) As scaling sub-network), enc2 (that is, aforementioned first master file accumulate group), enc3 (that is, aforementioned second master file accumulates group), enc4 ( That is, aforementioned third master file product group) and fc attention (that is, aforementioned main feature association sub-network).Wherein, practical application When, the realization of Xception module can be used in enc2, enc3 and enc4.Xception module is also referred to as the revoluble volume module of depth, The dimension that its cardinal principle is the convolution operation by multiple 1*1 to reduce feature.In one embodiment, the present invention is implemented Example provides two kinds of design parameter information (respectively ginsengs of XceptionA of the convolution kernel in conv1, enc2, enc3 and enc4 The parameter information of number information and XceptionB), as shown in table 1 below:

Table 1

In practical application, the target image of 1024 × 1024 sizes is input to main coding network first, through main coding Conv1 in network is contracted to 512 × 512 × 64 characteristic pattern；The characteristic pattern exported from conv1 is input to main coding net again Enc2 obtains 256 × 256 × 48 characteristic pattern after down-sampling dimensionality reduction in network；The characteristic pattern that enc2 in main coding network is exported It is input to the characteristic pattern that enc3 in main coding network obtains 128 × 128 × 96 after down-sampling dimensionality reduction, it will be in main coding network The characteristic pattern of enc3 output is input to the characteristic pattern that enc4 in main coding network obtains 64 × 64 × 192 after down-sampling dimensionality reduction, The enc4 characteristic pattern exported is input in the fc attention of main coding network again, after full attended operation and mixing operation Output obtains the fisrt feature figure (64 × 64 × 192) of main coding network.

Referring to Figure 14, the embodiment of the invention also provides a kind of structural schematic diagrams of main feature association sub-network, based on figure Image Segmentation Model structure shown in 13, the main feature association sub-network include fc layers (that is, the full articulamentums of aforementioned master), Conv (that is, aforementioned master file lamination) and principal point multiplication layer (as shown in Figure 14).Wherein, fc layers of dimension is 1000, conv parameter is 1 × 1 × 192.By fc layer in the fc attention of main coding network and conv layers to chief editor After the characteristic pattern of enc4 output carries out full attended operation in code network, then pass through principal point multiplication layer for enc4 in main coding network The characteristic pattern of output is merged with the characteristic pattern of the enc4 output after full attended operation, obtains fisrt feature figure.In addition, such as Shown in Figure 13, fisrt feature figure is amplified into presupposition multiple (4 times).

Further, as shown in figure 13, auxiliary coding network includes two auxiliary code sets, include in each auxiliary code set it is auxiliary under adopt Appearance network and auxiliary feature association sub-network include three auxiliary convolution groups in each auxiliary down-sampling sub-network.In practical applications, Master file product group and main feature association in the structural parameters information of auxiliary convolution group and auxiliary feature association sub-network and main coding network The structural parameters information that subnet is fallen can be consistent, and specific application process can refer to the reality of the image partition method in embodiment two It applies, no longer illustrates herein.

Further, as shown in figure 13, decoding network (that is, decoder frame in Figure 13) includes 6 up-sampling layers, (is divided Wei conv × 1 (a), conv × 2, conv × 3, conv × 4, conv × 8 and conv × 16), for decoding network will to be inputted In different size of characteristic pattern be amplified to uniform sizes.Specifically, conv × 1 (a) is for exporting enc2 in main coding network Characteristic pattern amplify 1 times；Conv × 2 are used to the characteristic pattern of the enc2 output in the 1st auxiliary code set amplifying 2 times；conv×3 For the characteristic pattern of the enc2 output in the 2nd auxiliary code set to be amplified 3 times；Conv × 4 are used for the fc in main coding network The characteristic pattern of attention output amplifies 4 times；Conv × 8 are used for the fc attention output in the 1st auxiliary code set Characteristic pattern amplifies 8 times；Conv × 16 are used for the characteristic pattern amplification 16 of the fc attention output in the 2nd auxiliary code set Times.When practical application, corresponding amplification factor can also be directly based upon by above-mentioned different size of characteristic pattern and be amplified to uniform sizes, The embodiment of the present invention preferably uses above-mentioned up-sampling layer, can enhance the generalization ability of decoding network, helps to promote image The segmentation performance of parted pattern.

In addition, the decoding network in Figure 13 further includes two addition without carry operation layers, one of them be used for conv × 1 (a), The image that conv × 2 and conv × 3 are exported is merged, and the first intermediate fusion feature figure is obtained；Another is used for conv × 4, the image that conv × 8 and conv × 16 are exported is merged, and obtains the second intermediate fusion feature figure, and will be melted among first It closes characteristic pattern and the second intermediate fusion feature figure is merged to obtain aforementioned first fusion feature figure.Based on such structure, originally Need to merge 6 kinds of characteristic patterns can be divided into two parts by inventive embodiments, after two parts are merged simultaneously, then to this two parts The result of fusion output is merged, and can effectively be shortened the calculating time of decoding process, be helped to improve image segmentation Speed.Further, intermediate treatment convolutional layer is additionally provided in decoding network shown in Figure 13 (that is, shown in Figure 13 Conv × 1 (b)), conv × 1 (b) is set between two addition without carry operation layers, by first as shown in figure 13 from left to right The intermediate fusion feature figure of addition without carry operation layer output is first defeated with second addition without carry operation layer again after conv × 1 (b) processing Intermediate fusion feature figure out is merged, and the general of decoding network can be further enhanced while not generating extra computation amount Change ability promotes the segmentation performance of Image Segmentation Model, is a kind of preferred embodiment, so that the effect of image segmentation result More preferably.

To show effect caused by the embodiment of the present invention, on general database (cityscapes), the present invention is real It applies example and has carried out parameter selection and test.By parameter regulation, the embodiment of the present invention using three layers feature multiplexing (that is, Main coding network, the first auxiliary code set and the second auxiliary code set).Method (DeepLab) compared to baseline, the present invention are real Applying example realizes 200 times of acceleration and 15% performance boost, compared to real time method for segmenting best before (BiSeNet1) embodiment of the present invention realizes 3.5 times of acceleration and 3% performance boost.Specifically with other parted patterns Contrast on effect explanation, it is as shown in table 2 below:

Table 2

Wherein, the DFANetA in table 2 and DFANetB is Image Segmentation Model provided in an embodiment of the present invention；Specifically , DFANet1 is model designed by above-mentioned XceptionA structural parameters based on the embodiment of the present invention, and DFANet1 is Model designed by above-mentioned XceptionB structural parameters based on the embodiment of the present invention.

In addition, the embodiment of the invention also provides two kinds of effect contrast figures.A kind of Contrast on effect signal as shown in figure 15 Scheme, the corresponding relationship in table 2 between the transmission frame number per second of each model and accuracy (average to hand over and than %) is shown in Figure 15. As shown in Figure 15, above-mentioned Image Segmentation Model provided in an embodiment of the present invention can effectively keep splitting speed and accuracy it Between balance, may be implemented to image carry out fast accurate segmentation.

Referring to Figure 16, the embodiment of the invention provides another Contrast on effect schematic diagram, shown to three not in Figure 16 With the process that is split of image, above-mentioned Image Segmentation Model based on the embodiment of the present invention (comprising main coding network, First auxiliary code set, the second auxiliary code set).Each column image shown in Figure 16 is followed successively by former target image, amplified master The characteristic pattern of coding network output, the characteristic pattern of amplified first auxiliary code set output, amplified second auxiliary code set are defeated Characteristic pattern out and the image segmentation result manually demarcated.It can be seen that with the increase of the network number of plies, the image exported point It cuts result classification ability to express and feature space descriptive power is more and more stronger, finally more connect with the segmentation result manually demarcated Closely.It can be seen that above-mentioned Image Segmentation Model provided in an embodiment of the present invention is able to ascend the accuracy of image segmentation.

Example IV:

For image partition method provided in embodiment two, the embodiment of the invention provides a kind of image segmentation dresses It sets, a kind of structural block diagram of image segmentation device shown in Figure 17, the device comprises the following modules:

Target image obtains module 1702, for obtaining target image to be split；

Main coding module 1704, for target image to be input to main coding network, by main coding network to target figure As performing the encoding operation, fisrt feature figure is obtained；

Size amplification module 1706 obtains amplified first for the size of fisrt feature figure to be amplified presupposition multiple Characteristic pattern；

Auxiliary coding module 1708 passes through auxiliary coding net for amplified fisrt feature figure to be input to auxiliary coding network Network performs the encoding operation amplified fisrt feature figure, obtains second feature figure；

Decoder module 1710 passes through decoding network for fisrt feature figure and second feature figure to be input to decoding network Fisrt feature figure and second feature figure are merged, obtains the first fusion feature figure, and operation is decoded to the first fusion feature figure, Obtain the segmentation result of target image.

Above-mentioned image segmentation device provided in an embodiment of the present invention is based on multiple coding networks (main coding network and auxiliary volume Code network) target image is carried out repeatedly to encode the macroscopic information that can effectively extract each object in image, facilitate promotion point Class ability to express；And to repeatedly encoding after resulting characteristic pattern (fisrt feature figure and second feature figure) merges, to fusion Characteristic pattern is decoded, and obtains segmentation result, facilitates the minutia for effectively going back each object in original image, so as to effective Promote the accuracy of image segmentation.

In one embodiment, main coding network includes sequentially connected image scaling sub-network, main down-sampling subnet Network and main feature association sub-network；Above-mentioned main coding module 1704 is further used for target image through image scaling sub-network Size scaling to specified size；Down-sampling behaviour is carried out to the target image for zooming to specified size by main down-sampling sub-network Make, obtains main down-sampling characteristic pattern；Full attended operation is carried out to main down-sampling characteristic pattern by main feature association sub-network, and will Main down-sampling characteristic pattern is merged with the main down-sampling characteristic pattern after full attended operation, obtains fisrt feature figure.

In a kind of embodiment kind, image scaling sub-network includes at least one layer of convolutional layer；Main down-sampling sub-network includes One or more master file product groups；Multiple master file product groups are sequentially connected, and each master file product group is for dropping the characteristic pattern inputted to it Down to specified characteristic dimension, the different corresponding specified characteristic dimensions of master file product group is different；And each master file product group includes Multiple convolutional layers；Main feature association sub-network includes the full articulamentum of sequentially connected master and master file lamination, further includes that principal point multiplies fortune Calculate layer；The full articulamentum of master and master file lamination in main feature association sub-network are used to carry out main down-sampling characteristic pattern full connection behaviour To make, principal point multiplication layer is used to merge main down-sampling characteristic pattern with the main down-sampling characteristic pattern after full attended operation, Obtain fisrt feature figure.

In one embodiment, auxiliary coding network includes M auxiliary code sets；Wherein, M be it is preset be not less than 1 from So number；If above-mentioned auxiliary coding module 1708 is further used for m equal to 1, amplified first is based on by the 1st auxiliary code set Characteristic pattern performs the encoding operation, and obtains the 1st auxiliary coding characteristic figure, and the 1st auxiliary coding characteristic figure is amplified described default times Number, obtains amplified m-th auxiliary coding characteristic figure；If m is greater than 1, amplified m- is based on by m-th of auxiliary code set 1 auxiliary coding characteristic figure performs the encoding operation, and obtains m-th of auxiliary coding characteristic figure, and m-th of auxiliary coding characteristic figure is amplified in advance If multiple, amplified m-th auxiliary coding characteristic figure is obtained；Wherein, the value of m successively takes from 2 to M-1；If m is equal to M, pass through The auxiliary code set of m-th is based on amplified m-1 auxiliary coding characteristic figures and performs the encoding operation, and obtains the auxiliary coding characteristic of m-th Figure；M-th of auxiliary coding characteristic figure is determined as second feature figure；Wherein, the value of m successively takes from 1 to M.

In one embodiment, each auxiliary code set includes that sequentially connected auxiliary down-sampling sub-network and auxiliary feature are closed Join sub-network；If m is equal to 1, above-mentioned auxiliary coding module 1708 is further used for auxiliary down-sampling by the 1st auxiliary code set Network is based on amplified fisrt feature figure and carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern；It is auxiliary by the 1st The auxiliary feature association sub-network of code set is based on the 1st auxiliary down-sampling characteristic pattern and carries out full attended operation, and by the 1st it is auxiliary under adopt Sample characteristic pattern is merged with the 1st auxiliary down-sampling characteristic pattern after full attended operation, obtains the 1st auxiliary coding characteristic figure； If m is greater than 1, above-mentioned auxiliary coding module 1708 is further used for being based on by the auxiliary down-sampling sub-network of m-th of auxiliary code set Amplified m-1 auxiliary coding characteristic figures carry out down-sampling operation, obtain m-th of auxiliary down-sampling characteristic pattern；It is auxiliary by m-th The auxiliary feature association sub-network of code set carries out full attended operation to m-th of auxiliary down-sampling characteristic pattern, and by m-th of auxiliary down-sampling Characteristic pattern is merged with m-th of auxiliary down-sampling characteristic pattern after full attended operation, obtains m-th of auxiliary coding characteristic figure.

In one embodiment, the main down-sampling sub-network of main coding network, which also exports, main intermediate features figure；Auxiliary volume The auxiliary down-sampling sub-network of code character, which also exports, auxiliary intermediate features figure；If m is equal to 1, above-mentioned auxiliary coding module 1708 is further For splicing main intermediate features figure and amplified fisrt feature figure, the 1st splicing characteristic pattern is obtained；Pass through the 1st The auxiliary down-sampling sub-network of auxiliary code set carries out down-sampling operation to the 1st splicing characteristic pattern, obtains the 1st auxiliary down-sampling feature Figure；If m is greater than 1, above-mentioned auxiliary coding module 1708 is further used for the auxiliary intermediate features figure of m-1 auxiliary code set output Spliced with amplified m-1 auxiliary coding characteristic figures, obtains m-th of splicing characteristic pattern；Pass through m-th auxiliary code set Auxiliary down-sampling sub-network carries out down-sampling operation to m-th of splicing characteristic pattern, obtains m-th of auxiliary down-sampling characteristic pattern；Wherein, m Value successively take from 2 to M.

In one embodiment, auxiliary down-sampling sub-network includes one or more auxiliary convolution groups；Multiple auxiliary convolution groups according to Secondary connection, each auxiliary convolution group are used to that specified characteristic dimension, different auxiliary convolution groups will to be reduced to its characteristic pattern inputted Corresponding specified characteristic dimension is different；And each auxiliary convolution group includes multiple convolutional layers；Auxiliary feature association sub-network include according to The auxiliary full articulamentum of secondary connection and auxiliary convolutional layer further include auxiliary point multiplication operation layer；Auxiliary full connection in auxiliary feature association sub-network Layer and auxiliary convolutional layer are used to carry out auxiliary down-sampling characteristic pattern full attended operation, and auxiliary point multiplication operation layer is used for auxiliary down-sampling feature Figure is merged with the auxiliary down-sampling characteristic pattern after full attended operation, obtains second feature figure.

In one embodiment, the master file that main down-sampling sub-network includes accumulates the quantity of group and each auxiliary code set includes The quantity of auxiliary convolution group be N；Wherein, N is the natural number not less than 1；Above-mentioned auxiliary coding module 1708 be further used for as Fruit n is equal to 1, and the output characteristic pattern of the 1st master file product group in main coding network is spelled with amplified fisrt feature figure It connects, obtains the 1st son splicing characteristic pattern, by the 1st auxiliary convolution group in the 1st auxiliary code set to the 1st son splicing feature Figure carries out down-sampling operation, and the output characteristic pattern of the 1st auxiliary convolution group in the 1st auxiliary code set is determined as the 1st auxiliary volume 1st auxiliary intermediate features figure of code character；It is if n is greater than 1, the output of (n-1)th auxiliary convolution group in the 1st auxiliary code set is special Sign figure and the output characteristic pattern of n-th of master file product group in main coding network are spliced, and the 1n son splicing feature is obtained Figure carries out down-sampling operation to the 1n son splicing characteristic pattern by n-th of auxiliary convolution group in the 1st auxiliary code set, will The output characteristic pattern of n-th of auxiliary convolution group in 1st auxiliary code set is determined as n-th of auxiliary intermediate spy of the 1st auxiliary code set Sign figure；Wherein, the value of n successively takes from 2 to N；By the auxiliary intermediate output figure of the n-th of the 1st auxiliary code set be determined as the 1st it is auxiliary under Sample characteristic pattern.

In one embodiment, if above-mentioned auxiliary coding module 1708 is further used for n equal to 1, by m-1 auxiliary volumes The output characteristic pattern of the 1st auxiliary convolution group in code character and amplified m-1 auxiliary coding characteristic figures are spliced, and obtain the M1 son splicing characteristic pattern, by the 1st auxiliary convolution group in m-th of auxiliary code set to the m1 son splice characteristic pattern into The operation of row down-sampling, is determined as m-th of auxiliary code set for the output characteristic pattern of the 1st auxiliary convolution group in m-th of auxiliary code set The 1st auxiliary intermediate features figure；If n is greater than 1, by the output characteristic pattern of n-th of auxiliary convolution group in m-1 auxiliary code sets And the output characteristic pattern of (n-1)th auxiliary convolution group in m-th of auxiliary code set is spliced, and it is special to obtain the mn son splicing Sign figure；Down-sampling operation is carried out to the mn son splicing characteristic pattern by n-th of auxiliary convolution group in m-th of auxiliary code set, The output characteristic pattern of n-th of auxiliary convolution group in m-th of auxiliary code set is determined as to n-th of auxiliary centre of m-th of auxiliary code set Characteristic pattern；The auxiliary intermediate output figure of the n-th of m-th of auxiliary code set is determined as m-th of auxiliary down-sampling characteristic pattern.

In one embodiment, decoding network includes that fusion sub-network conciliates numeral network；Fusion sub-network is used for will Fisrt feature figure and second feature figure are amplified to specified size, by amplified fisrt feature figure and amplified second feature figure It is merged, obtains the first fusion feature figure；Decoding sub-network obtains mesh for being decoded operation to the first fusion feature figure The segmentation result of logo image.

In one embodiment, fusion sub-network includes multiple up-sampling layers and addition without carry operation layer；Up-sample layer Input is fisrt feature figure or the second feature figure；The input of different up-sampling layers is different；Each up-sampling layer is used for will The characteristic pattern inputted to it is amplified to specified size, obtains amplified fisrt feature figure or amplified second feature figure；It presses Position plus operation layer are used to amplified fisrt feature figure and amplified second feature figure carrying out step-by-step add operation, obtain first Fusion feature figure.

In one embodiment, above-mentioned decoder module 1710 is further also used to the output of first master file product group is special The output characteristic pattern of first auxiliary convolution group in sign figure and each auxiliary code set is input to decoding network；Pass through decoding network Merge the output characteristic pattern of first master file product group, the output characteristic pattern of first auxiliary convolution group in each auxiliary code set, the One characteristic pattern and second feature figure obtain the first fusion feature figure.

Embodiment five:

Corresponding to method and apparatus provided by previous embodiment, the embodiment of the invention also provides a kind of image segmentation systems System, which includes image collecting device, processor and storage device；Image collecting device, for acquiring target image；Storage Computer program is stored on device, computer program executes any one provided such as embodiment two when being run by processor Method.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description Specific work process, can be with reference to the corresponding process in previous embodiment, and details are not described herein.

Further, the present embodiment additionally provides a kind of computer readable storage medium, the computer readable storage medium On be stored with computer program, the computer program executes any one institute that above-described embodiment two provides when being run by processor The step of method stated.

The computer program product of image partition method, apparatus and system provided by the embodiment of the present invention, including storage The computer readable storage medium of program code, the instruction that said program code includes can be used for executing previous methods embodiment Described in method, specific implementation can be found in embodiment of the method, details are not described herein.

In addition, in the description of the embodiment of the present invention unless specifically defined or limited otherwise, term " installation ", " phase Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can To be mechanical connection, it is also possible to be electrically connected；It can be directly connected, can also can be indirectly connected through an intermediary Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition Concrete meaning in invention.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention. And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.

In the description of the present invention, it should be noted that term " center ", "upper", "lower", "left", "right", "vertical", The orientation or positional relationship of the instructions such as "horizontal", "inner", "outside" be based on the orientation or positional relationship shown in the drawings, merely to Convenient for description the present invention and simplify description, rather than the device or element of indication or suggestion meaning must have a particular orientation, It is constructed and operated in a specific orientation, therefore is not considered as limiting the invention.In addition, term " first ", " second ", " third " is used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance.

Finally, it should be noted that embodiment described above, only a specific embodiment of the invention, to illustrate the present invention Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair It is bright to be described in detail, those skilled in the art should understand that: anyone skilled in the art In the technical scope disclosed by the present invention, it can still modify to technical solution documented by previous embodiment or can be light It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover in protection of the invention Within the scope of.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.

Claims (15)

1. a kind of image partition method characterized by comprising

Obtain target image to be split；

The target image is input to main coding network, coding behaviour is carried out to the target image by the main coding network Make, obtains fisrt feature figure；

The size of the fisrt feature figure is amplified into presupposition multiple, obtains amplified fisrt feature figure；

The amplified fisrt feature figure is input to auxiliary coding network, by the auxiliary coding network to described amplified Fisrt feature figure performs the encoding operation, and obtains second feature figure；

The fisrt feature figure and the second feature figure are input to decoding network, merge described the by the decoding network One characteristic pattern and the second feature figure, obtain the first fusion feature figure, and be decoded behaviour to the first fusion feature figure Make, obtains the segmentation result of the target image.

2. the method according to claim 1, wherein the main coding network includes sequentially connected image scaling Sub-network, main down-sampling sub-network and main feature association sub-network；

The described the step of target image is performed the encoding operation, obtains fisrt feature figure by the main coding network, packet It includes:

Sub-network is scaled by the size scaling of the target image to specified size by described image；

Down-sampling operation is carried out to the target image for zooming to specified size by the main down-sampling sub-network, is led Down-sampling characteristic pattern；

Full attended operation is carried out to the main down-sampling characteristic pattern by the main feature association sub-network, and will be adopted under the master Sample characteristic pattern is merged with the main down-sampling characteristic pattern after full attended operation, obtains fisrt feature figure.

3. according to the method described in claim 2, it is characterized in that, described image scaling sub-network includes at least one layer of convolution Layer；

The main down-sampling sub-network includes one or more master file product groups；Multiple master file product groups are sequentially connected, Mei Gesuo Master file product group is stated for specified characteristic dimension will to be reduced to its characteristic pattern inputted, different master file product groups is corresponding Specified characteristic dimension is different；And each master file product group includes multiple convolutional layers；

The main feature association sub-network includes the full articulamentum of sequentially connected master and master file lamination, further includes principal point multiplication Layer；The full articulamentum of master and master file lamination in the main feature association sub-network are used to carry out the main down-sampling characteristic pattern complete Attended operation, the principal point multiplication layer are used for the main down-sampling characteristic pattern and the main down-sampling after full attended operation is special Sign figure is merged, and fisrt feature figure is obtained.

4. according to the method described in claim 2, it is characterized in that, the auxiliary coding network includes M auxiliary code sets；Wherein, M For it is preset be not less than 1 natural number；

It is described that the amplified fisrt feature figure is performed the encoding operation by the auxiliary coding network, obtain second feature figure The step of, comprising:

If m is equal to 1, the amplified fisrt feature figure is based on by the 1st auxiliary code set and is performed the encoding operation, obtains the 1 auxiliary coding characteristic figure, and described 1st auxiliary coding characteristic figure is amplified into the presupposition multiple, obtain amplified m-th it is auxiliary Coding characteristic figure；

If m are greater than 1, amplified m-1 auxiliary coding characteristic figures are based on by m-th of auxiliary code set and are performed the encoding operation, M-th of auxiliary coding characteristic figure is obtained, and m-th of auxiliary coding characteristic figure is amplified into the presupposition multiple, is obtained amplified m-th Auxiliary coding characteristic figure；Wherein, the value of m successively takes from 2 to M-1；

If m are equal to M, amplified m-1 auxiliary coding characteristic figures are based on by the auxiliary code set of m-th and are performed the encoding operation, Obtain the auxiliary coding characteristic figure of m-th；

M-th of auxiliary coding characteristic figure is determined as second feature figure；Wherein, the value of m successively takes from 1 to M.

5. according to the method described in claim 4, it is characterized in that, each auxiliary code set include it is sequentially connected it is auxiliary under Sample sub-network and auxiliary feature association sub-network；

If m be equal to 1, it is described pass through the 1st auxiliary code set be based on the amplified fisrt feature figure perform the encoding operation, obtain The step of to the 1st auxiliary coding characteristic figure, comprising:

The amplified fisrt feature figure, which is based on, by the auxiliary down-sampling sub-network of the 1st auxiliary code set carries out down-sampling Operation, obtains the 1st auxiliary down-sampling characteristic pattern；

Described 1st auxiliary down-sampling characteristic pattern is based on by the auxiliary feature association sub-network of the 1st auxiliary code set to carry out entirely Attended operation, and the 1st auxiliary down-sampling characteristic pattern and the 1st auxiliary down-sampling characteristic pattern after full attended operation are carried out Fusion, obtains the 1st auxiliary coding characteristic figure；

If m is greater than 1, it is described pass through m-th of auxiliary code set and be based on amplified m-1 auxiliary coding characteristic figures carry out coding behaviour The step of making, obtaining m-th of auxiliary coding characteristic figure, comprising:

Amplified m-1 auxiliary coding characteristic figures are based on by the auxiliary down-sampling sub-network of described m-th auxiliary code set to carry out Down-sampling operation, obtains m-th of auxiliary down-sampling characteristic pattern；

Described m-th auxiliary down-sampling characteristic pattern is connected entirely by the auxiliary feature association sub-network of described m-th auxiliary code set Operation is connect, and described m-th auxiliary down-sampling characteristic pattern and m-th of auxiliary down-sampling characteristic pattern after full attended operation are melted It closes, obtains m-th of auxiliary coding characteristic figure.

6. according to the method described in claim 5, it is characterized in that, the main down-sampling sub-network of the main coding network also exports There is main intermediate features figure；The auxiliary down-sampling sub-network of the auxiliary code set, which also exports, auxiliary intermediate features figure；

If m is equal to 1, the auxiliary down-sampling sub-network by the 1st auxiliary code set is based on described amplified first The step of characteristic pattern carries out down-sampling operation, obtains the 1st auxiliary down-sampling characteristic pattern, comprising:

The main intermediate features figure and the amplified fisrt feature figure are spliced, the 1st splicing characteristic pattern is obtained；

Down-sampling operation is carried out to the 1st splicing characteristic pattern by the auxiliary down-sampling sub-network of the 1st auxiliary code set, Obtain the 1st auxiliary down-sampling characteristic pattern；

If m is greater than 1, it is a auxiliary that the auxiliary down-sampling sub-network by described m-th auxiliary code set is based on amplified m-1 The step of coding characteristic figure carries out down-sampling operation, obtains m-th of auxiliary down-sampling characteristic pattern, comprising:

The auxiliary intermediate features figure of m-1 auxiliary code set output is spliced with amplified m-1 auxiliary coding characteristic figures, Obtain m-th of splicing characteristic pattern；

Down-sampling operation is carried out to m-th of splicing characteristic pattern by the auxiliary down-sampling sub-network of described m-th auxiliary code set, Obtain m-th of auxiliary down-sampling characteristic pattern；Wherein, the value of m successively takes from 2 to M.

7. according to the method described in claim 6, it is characterized in that, the auxiliary down-sampling sub-network includes one or more auxiliary volumes Product group；Multiple auxiliary convolution groups are sequentially connected, and each auxiliary convolution group is used to that finger will to be reduced to its characteristic pattern inputted Fixed characteristic dimension, the corresponding specified characteristic dimension of the different auxiliary convolution group are different；And each auxiliary convolution group packet Include multiple convolutional layers；

The auxiliary feature association sub-network includes sequentially connected auxiliary full articulamentum and auxiliary convolutional layer, further includes auxiliary point multiplication operation Layer；Auxiliary full articulamentum and auxiliary convolutional layer in the auxiliary feature association sub-network are used to carry out the auxiliary down-sampling characteristic pattern complete Attended operation, the auxiliary point multiplication operation layer are used for the auxiliary down-sampling characteristic pattern and the auxiliary down-sampling after full attended operation is special Sign figure is merged, and second feature figure is obtained.

8. the method according to the description of claim 7 is characterized in that the number for the master file product group that the main down-sampling sub-network includes The quantity for the auxiliary convolution group that amount and each auxiliary code set include is N；Wherein, N is the natural number not less than 1；

The auxiliary down-sampling sub-network by the 1st auxiliary code set carries out down-sampling to the 1st splicing characteristic pattern The step of operating, obtaining the 1st auxiliary down-sampling characteristic pattern, comprising:

If n is equal to 1, by the output characteristic pattern and described amplified the of the 1st master file product group in the main coding network One characteristic pattern is spliced, and is obtained the 1st son splicing characteristic pattern, is passed through the 1st auxiliary convolution group in the 1st auxiliary code set Down-sampling operation is carried out to the 1st son splicing characteristic pattern, by the 1st auxiliary convolution group in the 1st auxiliary code set Output characteristic pattern is determined as the 1st auxiliary intermediate features figure of the 1st auxiliary code set；

If n is greater than 1, by the output characteristic pattern of (n-1)th auxiliary convolution group in the 1st auxiliary code set and the main coding network In n-th of master file product group output characteristic pattern spliced, obtain the 1n son splice characteristic pattern, pass through described 1st N-th of auxiliary convolution group in auxiliary code set carries out down-sampling operation to the 1n son splicing characteristic pattern, by described 1st The output characteristic pattern of n-th of auxiliary convolution group in auxiliary code set is determined as n-th of auxiliary intermediate features of the 1st auxiliary code set Figure；Wherein, the value of n successively takes from 2 to N；

The auxiliary intermediate output figure of the n-th of the 1st auxiliary code set is determined as the 1st auxiliary down-sampling characteristic pattern.

9. the method according to the description of claim 7 is characterized in that the auxiliary down-sampling by described m-th auxiliary code set The step of sub-network carries out down-sampling operation to m-th of splicing characteristic pattern, obtains m-th of auxiliary down-sampling characteristic pattern, comprising:

If n is equal to 1, by the output characteristic pattern and amplified m-1 of the 1st auxiliary convolution group in m-1 auxiliary code sets A auxiliary coding characteristic figure is spliced, and is obtained the m1 son splicing characteristic pattern, is passed through the 1st in described m-th auxiliary code set A auxiliary convolution group carries out down-sampling operation to the m1 son splicing characteristic pattern, by the 1st in described m-th auxiliary code set The output characteristic pattern of a auxiliary convolution group is determined as the 1st auxiliary intermediate features figure of described m-th auxiliary code set；

If n is greater than 1, by the output characteristic pattern and the m of n-th of auxiliary convolution group in the m-1 auxiliary code sets The output characteristic pattern of (n-1)th auxiliary convolution group in a auxiliary code set is spliced, and the mn son splicing characteristic pattern is obtained；It is logical N-th of the auxiliary convolution group crossed in described m-th auxiliary code set carries out down-sampling operation to the mn son splicing characteristic pattern, The output characteristic pattern of n-th of auxiliary convolution group in described m-th auxiliary code set is determined as the n-th of described m-th auxiliary code set A auxiliary intermediate features figure；

The auxiliary intermediate output figure of the n-th of described m-th auxiliary code set is determined as m-th of auxiliary down-sampling characteristic pattern.

10. the method according to claim 1, wherein the decoding network includes that fusion sub-network conciliates numeral Network；

The fusion sub-network is used to the fisrt feature figure and the second feature figure being amplified to specified size, after amplification The fisrt feature figure and the amplified second feature figure merged, obtain the first fusion feature figure；

The decoding sub-network obtains the segmentation of the target image for being decoded operation to the first fusion feature figure As a result.

11. according to the method described in claim 10, it is characterized in that, the fusion sub-network include multiple up-sampling layers and by Position plus operation layer；

The input of the up-sampling layer is the fisrt feature figure or the second feature figure；The input of different up-sampling layers is not Together；

Each up-sampling layer is used to that specified size will to be amplified to its characteristic pattern inputted, obtains amplified fisrt feature Figure or amplified second feature figure；

The addition without carry operation layer is used to carry out in the amplified fisrt feature figure and the amplified second feature figure Step-by-step add operation obtains the first fusion feature figure.

12. the method according to the description of claim 7 is characterized in that described merge first spy by the decoding network The step of sign schemes and the second feature figure, obtains the first fusion feature figure, comprising:

By the defeated of first auxiliary convolution group in the output characteristic pattern and each auxiliary code set of first master file product group Characteristic pattern is input to the decoding network out；

The output characteristic pattern of first master file product group, the in each auxiliary code set are merged by the decoding network The output characteristic pattern of one auxiliary convolution group, the fisrt feature figure and the second feature figure, obtain the first fusion feature figure.

13. a kind of image segmentation device characterized by comprising

Target image obtains module, for obtaining target image to be split；

Main coding module, for the target image to be input to main coding network, by the main coding network to the mesh Logo image performs the encoding operation, and obtains fisrt feature figure；

Size amplification module obtains amplified fisrt feature for the size of the fisrt feature figure to be amplified presupposition multiple Figure；

Auxiliary coding module passes through the auxiliary coding net for the amplified fisrt feature figure to be input to auxiliary coding network Network performs the encoding operation the amplified fisrt feature figure, obtains second feature figure；

Decoder module passes through the decoding for the fisrt feature figure and the second feature figure to be input to decoding network Fisrt feature figure described in the network integration and the second feature figure obtain the first fusion feature figure, and special to first fusion Sign figure is decoded operation, obtains the segmentation result of the target image.

14. a kind of image segmentation system, which is characterized in that the system comprises: image collecting device, processor and storage dress It sets；

Described image acquisition device, for acquiring target image；

Computer program is stored on the storage device, the computer program is executed when being run by the processor as weighed Benefit requires 1 to 12 described in any item methods.

15. a kind of computer readable storage medium, computer program, feature are stored on the computer readable storage medium The step of being, the described in any item methods of the claims 1 to 12 executed when the computer program is run by processor.