CN110490238A - A kind of image processing method, device and storage medium - Google Patents

Abstract

The embodiment of the invention discloses a kind of image processing method, device and storage medium, the embodiment of the present invention, which passes through, obtains image to be processed, and the label detection region in image to be processed is extracted by the first image recognition algorithm；Intercept label detection Area generation target image to be processed；Feature extraction is carried out to target image to be processed by the second image recognition algorithm, determines corresponding target identification classification results；Corresponding processing result image is determined according to target identification classification results.With this, the label detection region comprising default mark can be carried out by the first image recognition algorithm scratching figure generation target image to be processed, the target identification classification results of the default mark in target image to be processed are determined using the second image recognition algorithm of accurate knowledge figure, cooperation processing is carried out in conjunction with the advantages of two image recognition algorithms, so that image processing effect more fully and accurately, is greatly improved the comprehensive and accuracy rate of image procossing.

Description

A kind of image processing method, device and storage medium

Technical field

The present invention relates to fields of communication technology, and in particular to a kind of image processing method, device and storage medium.

Background technique

With the development of network and the extensive use of computer, the popularization master for paying the bill to launch promotion message can put down promoting Impression information on platform often includes vivid image in promotion message, comes to promote the product of oneself well The effect promoted is promoted, and Extension Software Platform needs to carry out audit processing to the content in image in order to avoid other products of encroaching right.

In the prior art, Extension Software Platform needs before being launched promotion message to the image packet in promotion message The identification information contained carries out qualification, if promoting the dispensing qualification of the main identification information for not having and identifying, promotes Extension Software Platform of advocating peace will undertake liability for tort, and Extension Software Platform generally can carry out first trial screening using simple detection network, then Qualification is carried out to image by the way of manual examination and verification.

In the research and practice process to the prior art, it was found by the inventors of the present invention that in the prior art, although providing Simple detection network carries out first trial screening, but can only identify the mark letter of single form due to simply detecting network Breath, treatment effect is often not comprehensive enough, is easy to omit the identification information in image.

Summary of the invention

The embodiment of the present invention provides a kind of image processing method, device and storage medium, it is intended to promote the complete of image procossing Face property and accuracy rate.

In order to solve the above technical problems, the embodiment of the present invention the following technical schemes are provided:

A kind of image processing method, comprising:

Image to be processed is obtained, and the label detection in the image to be processed is extracted by the first image recognition algorithm Region；

Intercept the label detection Area generation target image to be processed；

Feature extraction is carried out to target image to be processed by the second image recognition algorithm, determines corresponding target Identify classification results；

Corresponding processing result image is determined according to target identification classification results.

A kind of image processing apparatus, comprising:

First extraction unit for obtaining image to be processed, and is extracted by the first image recognition algorithm described wait locate Manage the label detection region in image；

Interception unit, for intercepting the label detection Area generation target image to be processed；

Second extraction unit is mentioned for carrying out feature to target image to be processed by the second image recognition algorithm It takes, determines corresponding target identification classification results；

Determination unit, for determining corresponding processing result image according to target identification classification results.

In some embodiments, the interception unit, is used for:

The output layer of target detection network after the training is connect with the input layer of the residual error network after training；

The label detection region of target detection network output after the training is intercepted, target figure to be processed is obtained Picture, and target image to be processed is inputted to the input layer of the residual error network after the training.

In some embodiments, the mark subelement, is used for:

The characteristic information and depth residual information are subjected to full connection group and are merged into row normalized, obtains corresponding mesh Mark mark classification results；

The determination unit, is used for:

The label detection region and corresponding target identification classification results are labeled on the image to be processed.

In some embodiments, second extraction unit further include:

First acquisition subelement, for acquiring the first training image, first training image includes default mark and phase The identification information answered；

Subelement is intercepted, includes the default target training region identified for intercepting, and target training region is true It is set to first object training image, the first object training image is associated with the identification information；

Second acquisition subelement, include for acquiring the second target training image, in the second target training image with The similarity of default mark is greater than the non-default mark of preset threshold；

Training subelement, for the first object training image and the second target training image input is described residual Poor network is trained, the residual error network after being trained.

In some embodiments, the interception subelement, is used for:

Interception includes the training region of default mark, and carries out the processing of data augmentation and/or more rulers to the trained region Change process is spent, target training region is obtained；

Target training region is determined as first object training image.

The embodiment of the present invention extracts image to be processed by the first image recognition algorithm by obtaining image to be processed In label detection region；Intercept label detection Area generation target image to be processed；By the second image recognition algorithm to mesh It marks image to be processed and carries out feature extraction, determine corresponding target identification classification results；It is true according to target identification classification results Fixed corresponding processing result image.With this, the default label detection area identified can will be included by the first image recognition algorithm Domain carries out scratching figure generation target image to be processed, determines target figure to be processed using the second image recognition algorithm of accurate knowledge figure The target identification classification results of default mark as in, carry out cooperation processing in conjunction with the advantages of two image recognition algorithms, so that Image processing effect more fully and accurately, is greatly improved the comprehensive and accuracy rate of image procossing.

Detailed description of the invention

To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.

Fig. 1 is the schematic diagram of a scenario of image processing system provided in an embodiment of the present invention；

Fig. 2 a is the flow diagram of image processing method provided in an embodiment of the present invention；

Fig. 2 b is the structural schematic diagram of target detection network provided in an embodiment of the present invention；

Fig. 2 c is a structural schematic diagram of target detection network provided in an embodiment of the present invention；

Fig. 2 d is the structural schematic diagram of 50 layers of residual error network provided in an embodiment of the present invention；

Fig. 3 is a flow diagram of image processing method provided in an embodiment of the present invention；

Fig. 4 is the application scenarios schematic diagram of image processing method provided in an embodiment of the present invention；

Fig. 5 a is the structural schematic diagram of image processing apparatus provided in an embodiment of the present invention；

Fig. 5 b is a structural schematic diagram of image processing apparatus provided in an embodiment of the present invention；

Fig. 5 c is a structural schematic diagram of image processing apparatus provided in an embodiment of the present invention；

Fig. 6 is the structural schematic diagram of server provided in an embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts Example, shall fall within the protection scope of the present invention.

The embodiment of the present invention provides a kind of image processing method, device and storage medium.

Referring to Fig. 1, Fig. 1 is the schematic diagram of a scenario of image processing system provided by the embodiment of the present invention, comprising: terminal A, and server (image processing system can also include other terminals in addition to terminal A, and the specific number of terminal is here not Limit), can be connected by communication network between terminal A and server, the communication network, may include wireless network and Cable network, wherein wireless network includes in wireless wide area network, WLAN, wireless MAN and private wireless network One or more combinations.Include router, gateway etc. network entity in network, is not illustrated in figure.Terminal A can lead to It crosses communication network and server carries out information exchange, for example terminal A needs are launched on the corresponding Extension Software Platform of server When information is promoted, server needs to carry out the image to be processed in promotion message qualification to avoid other classes of encroaching right Product, therefore, terminal A needs in real time send pending image to be processed in server, and server can be to reception The image to be processed arrived carries out aptitude checking.

The image processing system may include image processing apparatus, which specifically can integrate in server In, in some embodiments, which can also be integrated in the terminal with operational capability, in the present embodiment In, be illustrated in the server so that the image processing apparatus is integrated, as shown in Figure 1, server receiving terminal A send to The image to be processed of audit, is identified identification to the image to be processed by the first image recognition algorithm, extracts this wait locate The label detection region in image is managed, includes default mark in the label detection region, that is, completes to determine default mark Position intercepts the label detection Area generation target image to be processed, completes that the stingy graphic operation of default mark will be needed to carry out The region of identification individually intercepts and comes out, and carries out feature extraction, root to target image to be processed by the second image recognition algorithm Corresponding target identification classification results are determined according to the feature of extraction, such as identify the identified category of default mark, finally, can be with Corresponding processing result image is determined according to the target identification classification results, it such as can be by the target identification classification results and mark Know detection zone to be labeled on image to be processed, auditor quickly audited according to the processing result image, It is further to improve review efficiency.

Terminal A can install the application that various users need, such as instant messaging application, media in the image processing system Using and browser application etc., as the user volume that instant messaging is applied is growing day by day, promotion message can be thrown by promoting master It is put into instant messaging application platform and promotes oneself product, i.e., popularization master can initiate to promote request, by pending wait locate Reason image, which uploads in the server that instant messaging is applied, to be audited.

It should be noted that the schematic diagram of a scenario of image processing system shown in FIG. 1 is only an example, the present invention is real The image processing system and scene of applying example description are the technical solutions in order to more clearly illustrate the embodiment of the present invention, not The restriction for technical solution provided in an embodiment of the present invention is constituted, those of ordinary skill in the art are it is found that with image procossing The differentiation of system and the appearance of new business scene, technical solution provided in an embodiment of the present invention is for similar technical problem, together Sample is applicable in.

It is described in detail separately below.It should be noted that the serial number of following embodiment is not as preferably suitable to embodiment The restriction of sequence.

In the present embodiment, it will be described from the angle of image processing apparatus, which can specifically collect At have storage element and microprocessor is installed and with operational capability computer equipment in, computer equipment can be Server or terminal are illustrated so that computer equipment is server as an example in the present embodiment.

A kind of image processing method, comprising: obtain image to be processed, and extracted by the first image recognition algorithm wait locate Manage the label detection region in image；Intercept label detection Area generation target image to be processed；It is calculated by the second image recognition Method carries out feature extraction to target image to be processed, determines corresponding target identification classification results；Classified according to target identification As a result corresponding processing result image is determined.It is understood that this method is executed by computer equipment.

Fig. 2 a is please referred to, Fig. 2 a is the flow diagram of image processing method provided in an embodiment of the present invention.At the image Reason method includes:

In a step 101, image to be processed is obtained, and is extracted in image to be processed by the first image recognition algorithm Label detection region.

Artificial intelligence (Artificial Intelligence, AI) is to utilize digital computer or digital computer control Machine simulation, extension and the intelligence for extending people of system, perception environment obtain knowledge and the reason using Knowledge Acquirement optimum By, method, technology and application system.In other words, artificial intelligence is a complex art of computer science, it attempts to understand The essence of intelligence, and produce a kind of new intelligence machine that can be made a response in such a way that human intelligence is similar.Artificial intelligence The design principle and implementation method for namely studying various intelligence machines make machine have the function of perception, reasoning and decision.

Artificial intelligence technology is an interdisciplinary study, is related to that field is extensive, and the technology of existing hardware view also has software layer The technology in face.Artificial intelligence basic technology generally comprise as sensor, Special artificial intelligent chip, cloud computing, distributed storage, The technologies such as big data processing technique, operation/interactive system, electromechanical integration.Artificial intelligence software's technology mainly includes computer Several general orientation such as vision technique, voice processing technology, natural language processing technique and machine learning/deep learning.

Computer vision technique (Computer Vision, CV) computer vision is how a research makes machine " seeing " Science further just refer to and the machines such as replace human eye to be identified, tracked to target with video camera and computer and measured Device vision, and graphics process is further done, so that computer is treated as the image for being more suitable for eye-observation or sending instrument detection to. As a branch of science, the relevant theory and technology of computer vision research, it is intended to which foundation can be from image or multidimensional number According to the middle artificial intelligence system for obtaining information.Computer vision technique generally includes image procossing, image recognition, image, semantic reason Solution, image retrieval, optical character identification (Optical Character Recognition, OCR), video processing, video semanteme Understanding, video content/Activity recognition, three-dimension object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map structure It the technologies such as builds, further includes the biometrics identification technologies such as common recognition of face, fingerprint recognition.

Scheme provided by the embodiments of the present application is related to the technologies such as the computer vision technique of artificial intelligence, especially by as follows Embodiment is illustrated:

It is understood that server side needs before it will promote main promotion message and launched in promotion message Identification information on image carries out aptitude checking, belongs to infringement if the no relevant qualification of popularization master, server side with Liability for tort will be undertaken by promoting master, and since current identification information is more and more, and every kind of brand all includes variform Identification information, such as text form identification information or areal shape identification information, auditor is impossible to remember So more identification informations, although some manufacturers provide simple detection network and are screened, simple detection net Network only supports the identification of some common identification informations, and detection effect is often not comprehensive enough, is easy to omit some marks in image Information is known, so that later period audit effect is poor.

Wherein, the image to be processed in the application can be popularization Your Majesty and reach in server, the figure audited Picture, the format of the image to be processed can be basic multilingual plane (BitMaP, BMP) format, graphic interchange format (Graphics Interchange Forma, GIF) or joint photographic experts group (Joint Photographic Experts Group, JPEG) etc., specially to the algorithm for presetting mark progress frame choosing in image, i.e., which can be First image recognition algorithm can extract the characteristic information in image to be processed, be identified region according to this feature information The qualified default corresponding regional frame of mark is elected, determines the default corresponding label detection region of mark by classification, Image to be processed is identified by the first image recognition algorithm, can be completed for the default mark in image to be processed The label detection region identified is intercepted out by the stingy graphic operation in region, so that subsequent can finely be identified.

In some embodiments, the label detection in the image to be processed should be extracted by the first image recognition algorithm The step of region may include:

(1) the characteristics map information in the image to be processed is extracted by the target detection network after training；

(2) this feature cartographic information is analyzed, determines corresponding target identification candidate region；

(3) the target identification candidate region is determined as label detection region；Or

(4) determine that the target identification candidate region identifies classification results accordingly, and according to the target identification candidate region Corresponding mark classification results are adjusted first object mark candidate region, obtain label detection region.

Wherein, the detection of the identification information can carry out preliminary screening by the help of computer, the target detection network (Faster RCNN) can be thus achieved target detection (object detection), complete to examine the mark comprising default mark It surveys region progress frame choosing to be referred to together to preferably describe the present embodiment incorporated by reference to Fig. 2 b and Fig. 2 c, which is this The structural schematic diagram for the target detection network that inventive embodiments provide, Fig. 2 c are target detection net provided in an embodiment of the present invention Another structural schematic diagram of network.The target detection network 10 can be broadly divided into 4 parts:

Basic part convolutional network 12 (Conv layers), the part are a kind of convolutional neural networks, such as 13 convolution (conv)+13, layer+4, line rectification function (relu) layer pond layer (pooling) layer is constituted, and is mainly used for extracting to be processed Characteristics map information 13 (feature maps) in image 11.

Area generation network 14 (Region Proposal Networks, RPN), the Area generation network 14 is for generating It identifies candidate region (region proposals), specifically by normalized function (softmax) characteristic of division cartographic information Anchor (anchors) in 13 obtains actively classification (positive) information and negative sort (negative) information, will actively divide Category information is determined as identifying candidate region, and the frame for calculating anchor returns (bounding box regression) offset, root Offset is returned according to the frame to be adjusted the mark candidate region, obtains target identification candidate region 15 to the end (proposal), while too small and target identification candidate region 15 more than boundary is rejected, realizes the posting of default mark Choosing.In one embodiment, which can be determined directly as label detection region.

Interest pond layer 16 (ROI pooling), the layer are responsible for collecting target identification candidate region 15 and characteristics map letter Breath 13, and calculate the eligible provincial characteristics cartographic information of size (proposal feature maps) be sent into succeeding layer into Row processing.

Classifier 17 (Classifier), the layer may include at full articulamentum (full connection) and normalization Layer is managed, which is combined provincial characteristics cartographic information by full articulamentum and normalized layer, calculates this Provincial characteristics map identifies classification results accordingly, while can be according to the mark classification results to the target identification candidate region 15 are finely adjusted, and the target identification candidate region 15 after fine tuning is determined as label detection region.

In some embodiments, it is extracted in the image to be processed characteristically by the target detection network after training Before the step of figure information, may include:

(1) training image is acquired, includes default mark and corresponding identification information in the training image；

(2) interception includes the target training region of default mark, and target training region is determined as target training figure Picture, the target training image are associated with the identification information；

(3) the target training image is inputted the target detection network to be trained, the target detection net after being trained Network.

Wherein, the training image is acquired, includes default mark and corresponding identification information, the pre- bidding in the training image Knowing is the mark audited, which is the default corresponding systematic name of mark.

Further, in order to increase trained efficiency, intercepting includes the default target training region identified in training image, And target training region is determined as target training image, which is associated with the identification information, target training Image is positive sample, which is inputted the target detection network and is trained, so that the target inspection after training Survey grid network has the function of identifying default identified areas and systematic name in image, but due in the target detection network Network depth is than shallower, so the classification accuracy of the target detection network after the training is not high, can have many classification and miss Sentence, this will will greatly affect the review efficiency in later period.

In a step 102, label detection Area generation target image to be processed is intercepted.

Wherein, identify include the label detection region of default identified areas when, for the ease of subsequent fine point Class needs individually to handle the label detection region for including default identified areas, and it is raw to intercept the label detection region At target image to be processed.

It in some embodiments, can also processing figure to be detected to target in generation target processing image to be detected As carrying out image sharpening processing, that is, the profile of image is compensated, enhances the edge of image and the part of Gray Level Jump, becomes image Clearly, it is divided into spatial domain processing and frequency domain handles two classes, so that subsequent identification is more accurate.

In step 103, feature extraction is carried out to target image to be processed by the second image recognition algorithm, determines phase The target identification classification results answered.

Wherein, which can be the algorithm specially classified to target image to be processed, and should The algorithm complexity of second image recognition algorithm certainly will be greater than the algorithm complexity of the first image recognition algorithm, can extract the The minutia information that can not be extracted in one image recognition algorithm, so the classification accuracy of second image recognition algorithm is certain Greater than the classification accuracy of the first image recognition algorithm.Based on this, which can extract the target and wait locating The characteristic information and the minutia information that can not extract of the first image recognition algorithm of default mark in reason image, by should Characteristic information and minutia information are combined identification, the corresponding target identification of default mark point of available precise classification Class result.

In some embodiments, feature extraction should be carried out to target image to be processed by the second image recognition algorithm, The step of determining corresponding target identification classification results may include:

(1) characteristic information and residual information of target image to be processed are extracted by second image recognition algorithm；

(2) this feature information and residual information is combined to obtain target identification classification results.

Wherein, second image recognition algorithm can for training after residual error network (ResNet), common neural network, It the problems such as having gradient explosion and gradient disappearance with the depth increase of network, becomes increasingly difficult to train.Residual error network Proposition so that training depth network become to be more easier.His principle is the nonlinear model after the linear block of a certain layer Increase the output of front layer, i.e. study residual error before block, so that stack layer learns to arrive newly on the basis of input feature vector Feature, to possess better performance.

Further, the characteristic information and residual information in target image to be processed can be extracted by the residual error network, The residual information is the characteristic information that the first image recognition algorithm can not extract, and this feature information and residual information are tied It closes, it can be deduced that target identification classification results more more accurate than the first image recognition algorithm.

Therefore, the label detection region comprising default mark is directly being extracted by the first image recognition algorithm Afterwards, which can directly carry out sophisticated category on the basis of the label detection region, avoid to be processed to whole Image is identified, the efficiency and accuracy rate of residual error network are seriously affected.

In some embodiments, feature should be carried out to target image to be processed by the second image recognition algorithm to mention The step of taking, determining corresponding target identification classification results can also include:

(1.1) characteristic information of target image to be processed is extracted by the residual error network after training and depth residual error is believed Breath；

(1.2) target identification classification results are obtained according to this feature information and depth residual information.

Wherein, the residual error network after the training can be 50 layers of residual error network after training, in order to preferably describe this reality Example is applied, is referred to together incorporated by reference to Fig. 2 d, which is the structural representation of 50 layers of residual error network provided in an embodiment of the present invention Figure, 50 layers of residual error network 20 extract the characteristic information in target image to be processed by convolution module Conv1, and by residual Difference module Conv2_x, Conv3_x, Conv4_x and Conv5_x extract the depth residual error letter of multilayer in target image to be processed Breath.

Further, this feature information and depth residual information are subjected to pond by average pond layer, by Chi Huahou's Characteristic information and depth residual information normalize layer by the full articulamentum of 1000 dimensions and index and carry out full connection combination and normalization Processing, obtains corresponding target identification classification results, which carries out due to joined depth residual information Processing, the accuracy rate of identification are greatly promoted.

In some embodiments, the characteristic information of target image to be processed should be extracted by the residual error network after training Before the step of depth residual information, can also include:

(2.1) the first training image is acquired, which includes default mark and corresponding identification information；

(2.2) interception includes the target training region of default mark, and target training region is determined as first object Training image, the first object training image are associated with the identification information；

(2.3) the second target training image is acquired, includes the similarity with default mark in the second target training image Greater than the non-default mark of preset threshold；

(2.4) the first object training image and the second target training image the residual error network is inputted to be trained, Residual error network after being trained.

Wherein, the first training image is acquired, includes to preset mark and corresponding identification information in first training image, it should Default mark is the mark audited, which is the default corresponding systematic name of mark.

Further, in order to increase trained efficiency, intercepting in the first training image includes default mark target training center Domain, and target training region is determined as first object training image, which is associated with the identification information, The first object training image is positive sample.

Correspondingly, the second target training image can also be acquired, include and default mark in the second target training image Similarity be greater than the non-default mark of preset threshold, i.e., this it is non-default be identified as similar to default mark, but be not default The interference of mark identifies, so the second target training image is negative sample.By the positive sample of the first object training image It inputs in the residual error network and is trained with the negative sample of the second target training image, know so that the residual error network after training has The function of the identification information identified Chu not be preset in image, simultaneously as the training of negative sample is introduced, the residual error after the training Network, which also has the function of identifying, interferes mark, and since the network depth in the residual error network is deeper, so after the training Residual error network classification accuracy it is high, the low disadvantage of target detection network class accuracy rate after compensating for aforementioned training.

In some embodiments, the interception includes the target training region of default mark, and the target is trained region The step of being determined as first object training image may include:

(3.1) interception includes the training region of default mark, and carries out data augmentation processing and/or more to the training region Dimensional variation processing obtains target training region；

(3.2) target training region is determined as first object training image.

Wherein, in order to enhance the robustness of the residual error network after training, can interception include default mark training center When domain, the processing of data augmentation carried out to the training region, the mode of the data augmentation can be rotated to training region, mould Paste etc. variation, increases the quantity in training region.

Optionally, multiple dimensioned change process can also be carried out to training region, the mode of the multiple dimensioned change process can be with To carry out the cutting of multiple sizes to training region or resetting the modes such as size to change, the quantity in increase training region, And by data add lustre to and/or multiple dimensioned change process after training region be determined as target training region, by the target training center Domain is determined as first object training image, realizes the various dimensions extension of training data, the Shandong of the residual error network after enhancing training Stick.

In some embodiments, the step of interception label detection Area generation target image to be processed may include:

(4.1) output layer of the target detection network after the training is connect with the input layer of the residual error network after training；

(5.2) the label detection region by the target detection network output after the training intercepts, and obtains target and waits locating Image is managed, and target image to be processed is inputted to the input layer of the residual error network after the training.

Wherein it is possible to which the input layer of 50 layers of residual error network after training to be connect to the defeated of the target detection network after the training Out on layer, target detection network output identification detection zone after the training, and automatically by the label detection region to be processed Interception comes out on image, obtains target image to be processed, and target image to be processed is input to the residual error net after the training The input layer of network divides the residual error network after training finely just for the label detection region comprising default mark Class avoids and carries out traversal identification to whole image to be processed, influences the efficiency and accuracy rate of residual error network.

At step 104, corresponding processing result image is determined according to target identification classification results.

Wherein, which can be the identification information in each label detection region, due to the target mark Know classification results be for label detection region precise classification, so the recognition accuracy of the identification information be it is high, because This, can determine corresponding processing result image according to the result of the precise classification, for example, will identify that each mark letter come Breath is placed into and is shown together, generate processing result image, can be according to accurately identifying out so that auditor is subsequent Identification information quickly determines whether the identification information in the image to be processed encroaches right, be greatly saved time of auditor with Energy.

In some embodiments, the step of this determines corresponding processing result image according to target identification classification results, It may include that the label detection region and corresponding target identification classification results are labeled on the image to be processed.

Wherein it is possible to which the label detection region is labeled in the corresponding position of image to be processed in the form of rectangle frame On, it include default mark to prompt the user position, and mark will be preset in rectangle frame accordingly around the rectangle frame Target identification classification results shown with written form so that user can be with according to the corresponding text of the rectangle frame and periphery Quickly know the default mark in image to be processed and corresponding identification information, further improves the accuracy of image procossing With the efficiency of follow-up checks.

It can be seen from the above, the embodiment of the present invention is extracted by obtaining image to be processed, and by the first image recognition algorithm Label detection region in image to be processed out；Intercept label detection Area generation target image to be processed；Pass through the second image Recognizer carries out feature extraction to target image to be processed, determines corresponding target identification classification results；According to target mark Know classification results and determines corresponding processing result image.With this, default mark can will be included by the first image recognition algorithm Label detection region carry out scratch figure generate target image to be processed, determined using the second image recognition algorithm of accurate knowledge figure The target identification classification results of default mark in target image to be processed are matched in conjunction with the advantages of two image recognition algorithms Conjunction processing, so that image processing effect more fully and accurately, is greatly improved the comprehensive and accuracy rate of image procossing.

Citing, is described in further detail by the method in conjunction with described in above-described embodiment below.

In the present embodiment, it will be illustrated so that the image processing apparatus specifically integrates in the server as an example, the present invention Embodiment is illustrated by taking the identification scene of product identification as an example, referring in particular to following explanation.

Referring to Fig. 3, Fig. 3 is another flow diagram of image processing method provided in an embodiment of the present invention.This method Process may include:

In step 201, server obtains image to be processed, extracts figure to be processed by the target detection network after training Characteristics map information as in.

In some embodiments, server can acquire training image, including default mark and accordingly in the training image Identification information, which is product identification, can also become Product Logo (LOGO), which is product Title.For the efficiency of training for promotion, the target training region in training image comprising product identification part is intercepted, and by the mesh It marks training region and is determined as target training image, the target training image related product title.

It should be strongly noted that current target detection network once trains selected sample number (batch_size) Be 256 because the region in picture containing product identification is less, if if the selected sample number of primary training is 256 meeting so that The ratio of positive negative sample is 1 to 10 even more big, causes the loss of target detection network training mainly to be controlled by negative sample, does not do Method learns the information to positive sample well, causes e-learning effect bad, so in this application, by the primary trained institute The sample number of selection is set as 32, and network is allowed preferably to learn the information to positive sample.Based on this, by target training image It is input to target detection network with the quantity that the selected sample number of once training is 32 to be trained, so that the target after training Detection network has the function of identifying that product identification and name of product in image, the product identification can be multiple business mens Product identification.

Wherein, which can reach in server to promote Your Majesty, and the image audited such as is promoted Main the image audited to be needed to be sent to server by terminal side, server is when receiving the image to be processed, Ke Yitong The target detection network crossed after training extracts the characteristics map information in image to be processed.

For example, as shown in figure 4, Fig. 4 is the application scenarios schematic diagram of image processing method provided in an embodiment of the present invention, clothes Device be engaged in after receiving image 11 to be processed, image 11 to be processed is subjected to size compression processing, so that after size compression processing Image to be processed 11 meet processing requirement, by 13 layers of convolutional layer in the target detection network after training add 13 it is linear whole Stream function layer adds 4 pond layers successively to carry out characteristic processing to the image to be processed 11 after the size compression, extracts characteristically Figure information 13, this feature cartographic information 13 are the stacking of multiple treated two-dimension pictures, can be right with image 11 to be processed It should get up.

In step 202, server passes through the Area generation network analysis in the target detection network after training characteristically Figure information obtains the positive classification information and negative sort information in characteristics map information, positive classification information is determined as marking Know candidate region, the offset returned according to the frame of mark candidate region is adjusted mark candidate region, obtains target Identify candidate region.

Wherein, include Area generation network 14 in the target detection network, include user pass in this feature cartographic information The region that the region of note and user are not concerned with, the Area generation network 14 are used to complete the region by user's concern from characteristics map The function of coming, the corresponding region of product identification for such as wanting to pay close attention to by user constituency from characteristics map information are selected in information Function out.I.e. server is obtained by Area generation network analysis this feature cartographic information in the target detection network Some classification informations, the classification information are not that mark is which kind of to belong to, but export sentencing for each region in determinating area Definite value p, the p value range are p ∈ [0,1], and the regional determination by decision content p more than or equal to 0.5 is the region of user's concern, i.e., Positive classification information, the regional determination by decision content p less than 0.5 are the region that user is not concerned with, i.e. negative sort information.

Further, the region due to decision content p more than or equal to 0.5 is the region of user's concern, so actively by this Classification information is determined as identifying candidate region, and obtains the inclined of corresponding frame recurrence according to the anchor coordinate of the mark candidate region Shifting amount is adjusted the mark candidate region according to the offset that the frame returns, and obtains more accurate target identification and waits Favored area, while rejecting too small and target identification candidate region beyond boundary.

For example, as shown in figure 4, the Area generation network 14 is divided to for two processing lines, above a processing line pass through normalizing The anchor coordinate changed in function category this feature cartographic information obtains positive classification information and negative sort information, the positive classification letter In breath include the interested product identification of user, and in the negative sort information include the uninterested classification of user, as road, The background informations such as desk, closet.The positive classification information is determined as to identify candidate region, and passes through following processing line meter The offset returned for the frame of anchor coordinate is calculated, the offset returned according to the frame is adjusted mark candidate region, Accurate target identification candidate region 15 is obtained, includes product identification in the target identification candidate region 15.

In step 203, target identification candidate region is determined as label detection region by server.

Wherein, due to including product identification in the target identification candidate region, it can pass through target identification candidate Region realizes that the function of scratch figure for the product identification in image to be processed is realized, so server can be directly by the target Mark candidate region is determined as label detection region,

For example, as shown in figure 4, the target identification candidate region 15 directly can be determined as label detection region by server 18, which is to pass through rectangle frame for the region of the progress frame choosing comprising product identification.

In step 204, server is by the residual error network after the output layer of the target detection network after training and training The label detection region of target detection network output after training is intercepted, obtains target figure to be processed by input layer connection Picture, and the input layer of the residual error network by target image input to be processed after trained.

Wherein, the residual error network after the training is a kind of depth network of profound level, is wrapped in the residual error network after the training Contain residual error module, so that stack layer may learn new feature on the basis of input feature vector, thus relative to training Target detection network afterwards has better detection performance, but the residual error network after the training does not have region labeling function, So the residual error network after the training must rely on the region labeling function of the target detection network after training.

Further, in order to which that both realizes is used cooperatively, server is by the output layer of the target detection network after training It is connect with the input layer of the residual error network after training, and the label detection region of the target detection network output after training is carried out Interception, obtains target image to be processed, by the input layer of the residual error network after target image input training to be processed, so that should Residual error network after training only needs to carry out Classification and Identification for the product identification paid close attention to comprising user, does not need to be not related to other It infuses region and carries out Classification and Identification, the classification feature of network will not be tied down, make it difficult to restrain, and save the time of Classification and Identification.

For example, as shown in figure 4, the output layer 15 of target detection network after training is connected to the residual error after training by server The input layer Conv1 at network, and the label detection region 18 of the target detection network output after the training is intercepted, it generates Two target images to be processed, and in the input layer Conv1 at the residual error network that target image to be processed is input to after training.

In step 205, the first training image of collection of server, interception include the training region of default mark, and to instruction Practice region and carry out the processing of data augmentation and/or multiple dimensioned change process, obtains target training region, and target training region is true It is set to first object training image.

Wherein, server can acquire the first training image, which includes product identification and corresponding product Title, the product identification are the mark audited, and in order to increase trained efficiency, interception includes the instruction of product identification Practice region, and training region is rotated, obscures and/or the cutting of multiple sizes is carried out to training region and resets ruler It is very little, sample size more target training region is obtained, and target training region is determined as first object training image, it should First object training image is positive sample data.

In step 206, the second target of collection of server training image instructs first object training image and the second target Practice image input residual error network to be trained, the residual error network after being trained.

Wherein, the second target of collection of server training image includes and product identification in the second target training image The non-product that similarity is greater than preset threshold identifies, as negative sample data, in order to avoid subsequent residual error network should be with product Identify similar non-product mark wrong identification, it is possible to by the first object training image and the second target training image one And input in the residual error network and be trained, so that the residual error network after training has the ProductName of product identification in identification image The function of title, simultaneously as introducing the training of negative sample data, the residual error network after the training can also be excluded and product mark The function that non-product as sensible identifies.And since the network depth in the residual error network is deeper, so the residual error after the training The classification accuracy of network is high, the low disadvantage of the target detection network class accuracy rate after can compensating for aforementioned training.

In step 207, server by training after residual error network extract target image to be processed characteristic information and Depth residual information.

Wherein, server by training after residual error network extract target image to be processed in characteristic information and depth it is residual Poor information, this feature information are the characteristic information extracted by convolutional layer, and the depth residual information is by residual error network The depth residual information that extracts of residual error module, which is the new feature learnt, is the mesh after training Mark detection network can not extract.

For example, as shown in figure 4, server can pass through the convolutional layer in the target detection network after 50 layers of the training Conv1 extracts the characteristic information in target image 18 to be processed, by residual error module Conv2_x, Conv3_x, the Conv4_x and Conv5_x extracts the depth residual information in target image 18 to be processed.

In a step 208, characteristic information and depth residual information are carried out full connection group and are merged into capable normalization by server Processing, obtains corresponding target identification classification results.

Wherein, server carries out this feature information and depth residual information carrying out full connection group after pondization operates being merged into Row normalized integrates this feature information and depth residual information, and the information after the synthesis can show The target identification classification results of target image to be processed out, it can identify the ProductName of each target image to be processed Claim.

For example, as shown in figure 4, server by the Conv1 characteristic information extracted and by residual error module Conv2_x, The depth residual information that Conv3_x, Conv4_x and Conv5_x are extracted carries out pond processing by average pond layer, obtains size Satisfactory characteristic information and depth residual information, by the satisfactory characteristic information of the size and depth residual information It is attached and is normalized by the 1000 full articulamentums of dimension, obtain the product letter of each target image to be processed Breath.

In step 209, label detection region and corresponding target identification classification results are labeled in be processed by server On image.

Wherein, server can will identify after the target identification classification results for identifying each target image to be processed Detection zone is labeled on corresponding product identification in the form of rectangle frame, and will be produced on the side of the rectangle frame with written form Product information labeling aside, audits subsequent auditor directly according to the processing result image, greatly Audit time is saved, and mode through the foregoing embodiment can greatly be compressed the ratio of audit amount.Some In embodiment, the target detection network after the training could alternatively be Cascade R-CNN network, the residual error net after the training Network could alternatively be SENet network, not illustrate herein.

For example, as shown in figure 4, label detection region 18 is labeled in image 11 to be processed by server in the form of rectangle frame On, and by the name of product in each label detection region 18, such as mark 1 and mark 2 are shown in label detection in the form of text 18 side of region, auditor is by the processing result image, the product identification that can be directly determined as in the image to be processed Whether encroach right, is greatly saved the time.

It can be seen from the above, the embodiment of the present invention obtains image to be processed by server, and pass through the target inspection after training Survey grid network extracts the label detection region in image to be processed, after the output layer of the target detection network after training and training Residual error network input layer connection, by after training target detection network output label detection region intercept, obtain Target image to be processed, the residual error net by the input layer of the residual error network after target image input training to be processed, after the training Network is by positive sample data and negative sample data while to be trained to obtain, and extracts target by the residual error network after training and waits for The characteristic information and depth residual information for handling image, characteristic information and depth residual information are connected entirely and normalize place Reason, obtain corresponding target identification classification results, by label detection region and corresponding target identification classification results mark to It handles on image.With this, the label detection region comprising default mark can be carried out by the target detection network after training It scratches figure and generates target image to be processed, determined in target image to be processed using the residual error network after the training of accurate knowledge figure The target identification classification results of default mark, carry out cooperation processing in conjunction with the advantages of two networks, so that image processing effect is more Be it is comprehensive and accurate, be greatly improved the comprehensive and accuracy rate of image procossing.

For convenient for better implementation image processing method provided in an embodiment of the present invention, the embodiment of the present invention also provides one kind Device based on above-mentioned image processing method.Wherein the meaning of noun is identical with above-mentioned image processing method, and specific implementation is thin Section can be with reference to the explanation in embodiment of the method.

Fig. 5 a is please referred to, Fig. 5 a is the structural schematic diagram of image processing apparatus provided in an embodiment of the present invention, wherein the figure As processing unit may include the first extraction unit 301, interception unit 302, the second extraction unit 303 and determination unit 304 Deng.

First extraction unit 301 extracts this wait locate for obtaining image to be processed, and by the first image recognition algorithm Manage the label detection region in image.

Wherein, the image to be processed in the application is to promote Your Majesty to reach in server, and the image audited should The format of image to be processed can be BMP format, and GIF or JPEG etc., which can be for specially to figure The algorithm that mark carries out frame choosing is preset as in, i.e. first image recognition algorithm can extract the letter of the feature in image to be processed Breath, the first extraction unit 301 are identified territorial classification according to this feature information, by the qualified default corresponding area of mark Domain frame is elected, and determines the default corresponding label detection region of mark, i.e., by the first image recognition algorithm to figure to be processed As being identified, the stingy graphic operation for the default identified areas in image to be processed can be completed, by what is identified Label detection region intercepts out, so that subsequent can finely be identified.

In some embodiments, as shown in Figure 5 b, first extraction unit 301 may include:

Subelement 3011 is extracted, for obtaining image to be processed, extracting by the target detection network after training should be wait locate Manage the characteristics map information in image；

It analyzes subelement 3012 and determines corresponding target identification candidate region for analyzing this feature cartographic information；

Subelement 3013 is determined, for the target identification candidate region to be determined as label detection region；Or

Subelement 3014 is adjusted, for determining that the target identification candidate region identifies classification results accordingly, and according to this Target identification candidate region identifies classification results accordingly and is adjusted to the target identification candidate region, obtains label detection area Domain.

Wherein, the detection of the identification information can carry out preliminary screening by the help of computer, the target detection network Target detection can be thus achieved, complete to carry out frame choosing to the label detection region comprising default mark, in order to preferably describe this Embodiment is referred to together incorporated by reference to Fig. 2 b and Fig. 2 c, which is target detection network provided in an embodiment of the present invention Structural schematic diagram, Fig. 2 c are another structural schematic diagram of target detection network provided in an embodiment of the present invention.The target detection Network 10 can be broadly divided into 4 parts:

Basic 12 part of convolutional network, i.e. extraction subelement 3011, the part are a kind of convolutional neural networks, such as 13 volumes + 4, the line rectification function layer pond layer of lamination+13 is constituted, and is mainly used for extracting the characteristics map letter in image 11 to be processed Breath 13.

Area generation network 14, the Area generation network 14 analyze subelement 3012 and identify candidate region for generating, Positive classification information and passive point are obtained specifically by the anchor (anchors) in normalized function characteristic of division cartographic information 13 Positive classification information is determined as identifying candidate region by category information, and the frame for calculating anchor returns offset, is returned according to the frame Return offset to be adjusted the mark candidate region, obtain target identification candidate region 15 to the end, at the same reject it is too small and More than the target identification candidate region 15 on boundary, the posting choosing of default mark is realized.In one embodiment, determine that son is single The target identification candidate region 15 can be determined directly as label detection region by member 3013.

Interest pond layer 16, which is responsible for collecting target identification candidate region 15 and characteristics map information 13, and calculates The eligible provincial characteristics cartographic information of size is sent into succeeding layer and is handled.

Classifier 17, i.e. adjustment subelement 3014, which may include full articulamentum and normalized layer, the classifier 17 are combined provincial characteristics cartographic information by full articulamentum and normalized layer, calculate the provincial characteristics map phase The mark classification results answered, while the target identification candidate region 15 can be finely adjusted according to the mark classification results, it will Target identification candidate region 15 after fine tuning is determined as label detection region.

In some embodiments, the analysis subelement 3012, is used for: by the target detection network after the training Area generation network analysis this feature cartographic information obtains the positive classification information and negative sort letter in this feature cartographic information Breath；The positive classification information is determined as to identify candidate region；The offset pair returned according to the frame of the mark candidate region The mark candidate region is adjusted, and obtains target identification candidate region.

Interception unit 302, for intercepting the label detection Area generation target image to be processed.

Wherein, interception unit 302 identify include the label detection region of default identified areas when, for the ease of rear Continuous sophisticated category needs individually to handle the label detection region for including default identified areas, intercepts the mark Detection zone generates target image to be processed.

In some embodiments, in generation target processing image to be detected, interception unit 302 can also wait for target Detection processing image carries out image sharpening processing, that is, compensates the profile of image, enhance the edge of image and the part of Gray Level Jump, It is apparent from image, is divided into spatial domain processing and frequency domain handles two classes, so that subsequent identification is more accurate.

In some embodiments, the interception unit 302, is used for: by the output layer of the target detection network after the training It is connect with the input layer of the residual error network after training；The label detection region of target detection network output after the training is carried out Interception, obtains target image to be processed, and target image to be processed is inputted to the input layer of the residual error network after the training.

Wherein, the input layer of 50 layers of residual error network after training can be connect the target after the training by interception unit 302 On the output layer for detecting network, the target detection network output identification detection zone after the training, and automatically by the label detection Region intercepts from image to be processed and comes out, and obtains target image to be processed, and target image to be processed is input to the instruction The input layer of residual error network after white silk allows the residual error network after training just for the label detection area comprising default mark Domain carries out sophisticated category, avoids and carries out traversal identification to whole image to be processed, influences the efficiency and accuracy rate of residual error network.

Second extraction unit 303 is mentioned for carrying out feature to target image to be processed by the second image recognition algorithm It takes, determines corresponding target identification classification results.

Wherein, which can be the algorithm specially classified to target image to be processed, and should The algorithm complexity of second image recognition algorithm certainly will be greater than the algorithm complexity of the first image recognition algorithm, can extract the The minutia information that can not be extracted in one image recognition algorithm, so the classification accuracy of second image recognition algorithm is certain Greater than the classification accuracy of the first image recognition algorithm.Based on this, which can extract the target and wait locating The minutia information that the characteristic information and the first image recognition algorithm of default mark in reason image can not extract, second extracts Unit 303 is by being combined identification, the default mark of available precise classification for this feature information and minutia information Corresponding target identification classification results.

In some embodiments, as shown in Figure 5 c, second extraction unit 303, is used for: passing through second image recognition Algorithm extracts the characteristic information and residual information of target image to be processed；Target is obtained in conjunction with this feature information and residual information Identify classification results.

Wherein, which can be the residual error network after training, common neural network, with network Depth increase can exist gradient explosion and gradient disappear the problems such as, become increasingly difficult to train.The it is proposed of residual error network makes Depth network must be trained to become to be more easier.His principle is increased before nonlinear block after the linear block of a certain layer Adding the output of front layer, i.e. study residual error, so that stack layer learns on the basis of input feature vector to new feature, thus Possess better performance.

Further, the second extraction unit 303 can extract the feature in target image to be processed by the residual error network Information and residual information, which is the characteristic information that the first image recognition algorithm can not extract, by this feature information It is combined with residual information, it can be deduced that target identification classification results more more accurate than the first image recognition algorithm.

Therefore, the second extraction unit 303 will include the default label detection area identified by the first image recognition algorithm After domain directly extracts, which can directly carry out sophisticated category on the basis of the label detection region, avoid Whole image to be processed is identified, seriously affects the efficiency and accuracy rate of residual error network.

In some embodiments, as shown in Figure 5 c, second extraction unit 303 may include:

First acquisition subelement 3031, for acquiring the first training image, first training image include default mark and Corresponding identification information；

Interception subelement 3032 includes the default target training region identified for intercepting, and the target is trained region It is determined as first object training image, which is associated with the identification information；

Second acquisition subelement 3033, is used to acquire the second target training image, includes in the second target training image It is greater than the non-default mark of preset threshold with the similarity of default mark；

Training subelement 3034, for the first object training image and the second target training image to be inputted the residual error Network is trained, the residual error network after being trained；

Subelement 3035 is extracted, for extracting the characteristic information of target image to be processed by the residual error network after training With depth residual information；

Subelement 3036 is identified, for obtaining target identification classification results according to this feature information and depth residual information.

Wherein, the first acquisition subelement 3031 acquires the first training image, includes default mark in first training image With corresponding identification information, which is the mark audited, which is that default mark is corresponding Systematic name.

Further, in order to increase trained efficiency, it includes default that interception subelement 3032, which intercepts in the first training image, Target training region is identified, and target training region is determined as first object training image, the first object training image It is associated with the identification information, which is positive sample.

Correspondingly, the second acquisition subelement 3033 can also acquire the second target training image, second target training figure Non-default mark in picture comprising the similarity with default mark greater than preset threshold, i.e. non-default be identified as are identified with default It is similar, but be not the interference mark of default mark, so the second target training image is negative sample.Training subelement 3034 by the negative sample of the positive sample of the first object training image and the second target training image input in the residual error network into Row training, so that the residual error network after training has the function of identifying the identification information that mark is preset in image, simultaneously as The training of negative sample is introduced, the residual error network after the training also has the function of identifying interference mark, and due to the residual error Network depth in network is deeper, so the classification accuracy of the residual error network after the training is high, after compensating for aforementioned training The low disadvantage of target detection network class accuracy rate.

Residual error network after the training can be 50 layers of residual error network after training, in order to preferably describe the present embodiment, It is referred to together incorporated by reference to Fig. 2 d, which is the structural schematic diagram of 50 layers of residual error network provided in an embodiment of the present invention, is mentioned It takes subelement 3035 to extract the characteristic information in target image to be processed by convolution module Conv1, and passes through residual error module Conv2_x, Conv3_x, Conv4_x and Conv5_x extract the depth residual information of multilayer in target image to be processed.

Further, this feature information and depth residual information are carried out pond by average pond layer by mark subelement 3036 Change, the characteristic information of Chi Huahou and depth residual information are normalized into layer by the full articulamentum of 1000 dimensions and index and connected entirely Combination and normalized, obtain corresponding target identification classification results, the target identification classification results are due to joined depth Residual information is handled, and the accuracy rate of identification is greatly promoted.

In some embodiments, the interception subelement 3032, is used for: interception includes the training region of default mark, and The processing of data augmentation and/or multiple dimensioned change process are carried out to the training region, obtain target training region；By target training Region is determined as first object training image.

Wherein, in order to enhance the robustness of the residual error network after training, interception subelement 3032 can include pre- in interception When being marked with the training region known, the processing of data augmentation is carried out to the training region, the mode of the data augmentation can be for training Region rotated, fuzzy etc. variation, increases the quantity in training region.

Further, interception subelement 3032 can also carry out multiple dimensioned change process, the multiple dimensioned change to training region The mode for changing processing can increase to carry out the cutting of multiple sizes to training region or resetting the modes such as size to change Training region quantity, and by data add lustre to and/or multiple dimensioned change process after training region be determined as target training region, Target training region is determined as first object training image, the various dimensions extension of training data is realized, after enhancing training Residual error network robustness.

In some embodiments, mark subelement 3036, for carrying out this feature information and depth residual information Full connection group is merged into row normalized, obtains respective objects mark classification results.

Determination unit 304, for determining corresponding processing result image according to target identification classification results.

Wherein, which can be the identification information in each label detection region, due to the target mark Know classification results be for label detection region precise classification, so the recognition accuracy of the identification information be it is high, because This, determination unit 304 can determine corresponding processing result image according to the result of the precise classification, come for example, will identify that Each identification information be placed into and shown together, generate processing result image can be according to essence so that auditor is subsequent The identification information really identified quickly determines whether the identification information in the image to be processed encroaches right, and is greatly saved audit The time of personnel and energy.

In some embodiments, the determination unit 304, is used for: by the label detection region and corresponding target identification Classification results are labeled on the image to be processed.

Wherein it is determined that the label detection region can be labeled in the image to be processed by unit 304 in the form of rectangle frame It include default mark to prompt the user position on corresponding position, and will be pre- in rectangle frame around the rectangle frame Bidding is known corresponding target identification classification results and is shown with written form, so that user is corresponding according to the rectangle frame and periphery Text can quickly know default mark and corresponding identification information in image to be processed, further improve at image The accuracy of reason and the efficiency of follow-up checks.

The specific implementation of above each unit can be found in the embodiment of front, and details are not described herein.

It can be seen from the above, the embodiment of the present invention obtains image to be processed by the first extraction unit 301, and pass through the first figure The label detection region in image to be processed is extracted as recognizer；Interception unit 302 intercepts label detection Area generation mesh Mark image to be processed；Second extraction unit 303 carries out feature extraction to target image to be processed by the second image recognition algorithm, Determine corresponding target identification classification results；Determination unit 304 determines at corresponding image according to target identification classification results Manage result.With this, the label detection region comprising default mark can be carried out by the first image recognition algorithm scratching figure generation Target image to be processed determines the default mark in target image to be processed using the second image recognition algorithm of accurate knowledge figure Target identification classification results, cooperation processing is carried out in conjunction with the advantages of two image recognition algorithms, so that image processing effect is more Be it is comprehensive and accurate, be greatly improved the comprehensive and accuracy rate of image procossing.

The embodiment of the present invention also provides a kind of computer equipment, as shown in fig. 6, it illustrates involved by the embodiment of the present invention Server structural schematic diagram, specifically:

The computer equipment may include the processor 401, one or one of one or more than one processing core with The components such as memory 402, power supply 403 and the input unit 404 of upper computer readable storage medium.Those skilled in the art can be with Understand, computer equipment structure shown in Fig. 6 do not constitute the restriction to computer equipment, may include than illustrate it is more or Less component perhaps combines certain components or different component layouts.Wherein:

Processor 401 is the control centre of the computer equipment, is set using various interfaces and the entire computer of connection Standby various pieces, by running or executing the software program and/or module that are stored in memory 402, and calling storage Data in memory 402 execute the various functions and processing data of computer equipment, to carry out to computer equipment whole Body monitoring.Optionally, processor 401 may include one or more processing cores；Optionally, processor 401 can integrate at Manage device and modem processor, wherein the main processing operation system of application processor, user interface and application program etc. are adjusted Demodulation processor processed mainly handles wireless communication.It is understood that above-mentioned modem processor can not also integrate everywhere It manages in device 401.

Memory 402 can be used for storing software program and module, and processor 401 is stored in memory 402 by operation Software program and module, thereby executing various function application and data processing.Memory 402 can mainly include storage journey Sequence area and storage data area, wherein storing program area can the (ratio of application program needed for storage program area, at least one function Such as sound-playing function, image player function) etc.；Storage data area, which can be stored, uses created data according to server Deng.In addition, memory 402 may include high-speed random access memory, it can also include nonvolatile memory, for example, at least One disk memory, flush memory device or other volatile solid-state parts.Correspondingly, memory 402 can also include Memory Controller, to provide access of the processor 401 to memory 402.

Computer equipment further includes the power supply 403 powered to all parts, and optionally, power supply 403 can pass through power supply pipe Reason system and processor 401 are logically contiguous, to realize management charging, electric discharge and power managed by power-supply management system Etc. functions.Power supply 403 can also include one or more direct current or AC power source, recharging system, power failure inspection The random components such as slowdown monitoring circuit, power adapter or inverter, power supply status indicator.

Computer equipment may also include input unit 404, which can be used for receiving the number or character of input Information, and generate keyboard related with user setting and function control, mouse, operating stick, optics or trackball signal Input.

Although being not shown, computer equipment can also be including display unit etc., and details are not described herein.Specifically in the present embodiment In, the processor 401 in computer equipment can be according to following instruction, by the process pair of one or more application program The executable file answered is loaded into memory 402, and the application journey being stored in memory 402 is run by processor 401 Sequence, thus realize the various method and steps that previous embodiment provides, it is as follows:

Image to be processed is obtained, and the label detection area in the image to be processed is extracted by the first image recognition algorithm Domain；Intercept the label detection Area generation target image to be processed；By the second image recognition algorithm to target figure to be processed As carrying out feature extraction, corresponding target identification classification results are determined；Corresponding figure is determined according to target identification classification results As processing result.

In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, it may refer to the detailed description above with respect to image processing method, details are not described herein again.

It can be seen from the above, the computer equipment of the embodiment of the present invention can be by obtaining image to be processed, and pass through first Image recognition algorithm extracts the label detection region in image to be processed；Intercept label detection Area generation target figure to be processed Picture；Feature extraction is carried out to target image to be processed by the second image recognition algorithm, determines corresponding target identification classification As a result；Corresponding processing result image is determined according to target identification classification results.With this, the first image recognition algorithm can be passed through Label detection region comprising default mark is carried out to scratch figure generation target image to be processed, uses accurate the second image for knowing figure Recognizer determines the target identification classification results of the default mark in target image to be processed, calculates in conjunction with two image recognitions The advantages of method, carries out cooperation processing, so that image processing effect more fully and accurately, is greatly improved the complete of image procossing Face property and accuracy rate.

It will appreciated by the skilled person that all or part of the steps in the various methods of above-described embodiment can be with It is completed by instructing, or relevant hardware is controlled by instruction to complete, which can store computer-readable deposits in one In storage media, and is loaded and executed by processor.

For this purpose, the embodiment of the present invention provides a kind of storage medium, wherein being stored with a plurality of instruction, which can be processed Device is loaded, to execute the step in any image processing method provided by the embodiment of the present invention.For example, the instruction can To execute following steps:

Image to be processed is obtained, and the label detection area in the image to be processed is extracted by the first image recognition algorithm Domain；Intercept the label detection Area generation target image to be processed；By the second image recognition algorithm to target figure to be processed As carrying out feature extraction, corresponding target identification classification results are determined；Corresponding figure is determined according to target identification classification results As processing result.

The specific implementation of above each operation can be found in the embodiment of front, and details are not described herein.

Wherein, which may include: read-only memory (ROM, Read Only Memory), random access memory Body (RAM, Random Access Memory), disk or CD etc..

By the instruction stored in the storage medium, can execute at any image provided by the embodiment of the present invention Step in reason method, it is thereby achieved that achieved by any image processing method provided by the embodiment of the present invention Beneficial effect is detailed in the embodiment of front, and details are not described herein.

It is provided for the embodiments of the invention a kind of image processing method, device, storage medium above and has carried out detailed Jie It continues, used herein a specific example illustrates the principle and implementation of the invention, and the explanation of above embodiments is only It is to be used to help understand method and its core concept of the invention；Meanwhile for those skilled in the art, according to the present invention Thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be construed as Limitation of the present invention.

Claims (15)

1. a kind of image processing method characterized by comprising

Image to be processed is obtained, and the label detection area in the image to be processed is extracted by the first image recognition algorithm Domain；

Intercept the label detection Area generation target image to be processed；

Feature extraction is carried out to target image to be processed by the second image recognition algorithm, determines corresponding target identification Classification results；

Corresponding processing result image is determined according to target identification classification results.

2. image processing method according to claim 1, which is characterized in that described to pass through the extraction of the first image recognition algorithm The step of label detection region in the image to be processed out, comprising:

The characteristics map information in the image to be processed is extracted by the target detection network after training；

The characteristics map information is analyzed, determines corresponding target identification candidate region；

The target identification candidate region is determined as label detection region；Or

Determine that the target identification candidate region identifies classification results accordingly, and corresponding according to the target identification candidate region Mark classification results the target identification candidate region is adjusted, obtain label detection region.

3. image processing method according to claim 2, which is characterized in that the analysis characteristics map information, really The step of making corresponding mark candidate region, comprising:

By characteristics map information described in the Area generation network analysis in the target detection network after the training, obtain described Positive classification information and negative sort information in characteristics map information；

The positive classification information is determined as to identify candidate region；

The mark candidate region is adjusted according to the offset that the frame of the mark candidate region returns, obtains target Identify candidate region.

4. image processing method according to any one of claims 1 to 3, which is characterized in that described to pass through the knowledge of the second image The step of other algorithm carries out feature extraction to target image to be processed, determines corresponding target identification classification results, packet It includes:

The characteristic information and residual information of target image to be processed are extracted by second image recognition algorithm；

Target identification classification results are obtained in conjunction with the characteristic information and residual information.

5. image processing method according to any one of claims 2 to 3, which is characterized in that described to pass through the knowledge of the second image The step of other algorithm carries out feature extraction to target image to be processed, determines corresponding target identification classification results, packet It includes:

The characteristic information and depth residual information of target image to be processed are extracted by the residual error network after training；

Target identification classification results are obtained according to the characteristic information and depth residual information.

6. image processing method according to claim 5, which is characterized in that the interception label detection Area generation The step of target image to be processed, comprising:

The output layer of target detection network after the training is connect with the input layer of the residual error network after training；

The label detection region of target detection network output after the training is intercepted, target image to be processed is obtained, And target image to be processed is inputted to the input layer of the residual error network after the training.

7. image processing method according to claim 5, which is characterized in that described residual according to the characteristic information and depth Poor information obtains the step of target identification classification results, comprising:

The characteristic information and depth residual information are subjected to full connection group and are merged into row normalized, obtains corresponding target Identify classification results；

Described the step of determining corresponding processing result image according to target identification classification results, comprising:

The label detection region and corresponding target identification classification results are labeled on the image to be processed.

8. image processing method according to claim 5, which is characterized in that the residual error network by after training extracts Before the step of characteristic information and depth residual information of the target image to be processed, further includes:

The first training image is acquired, first training image includes default mark and corresponding identification information；

Interception includes the target training region of default mark, and target training region is determined as first object training figure Picture, the first object training image are associated with the identification information；

The second target training image is acquired, includes being greater than to preset with the similarity of default mark in the second target training image The non-default mark of threshold value；

The first object training image and the second target training image are inputted the residual error network to be trained, obtained Residual error network after training.

9. image processing method according to claim 8, which is characterized in that the interception includes the target instruction of default mark Practice region, and the step of target training region is determined as first object training image, comprising:

Interception includes the training region of default mark, and carries out the processing of data augmentation and/or multiple dimensioned change to the trained region Change processing obtains target training region；

Target training region is determined as first object training image.

10. a kind of image processing apparatus characterized by comprising

First extraction unit extracts the figure to be processed for obtaining image to be processed, and by the first image recognition algorithm Label detection region as in；

Interception unit, for intercepting the label detection Area generation target image to be processed；

Second extraction unit, for carrying out feature extraction to target image to be processed by the second image recognition algorithm, really Make corresponding target identification classification results；

Determination unit, for determining corresponding processing result image according to target identification classification results.

11. processing unit according to claim 10, which is characterized in that first extraction unit, comprising:

Subelement is extracted, for obtaining image to be processed, the image to be processed is extracted by the target detection network after training In characteristics map information；

It analyzes subelement and determines corresponding target identification candidate region for analyzing the characteristics map information；

Subelement is determined, for the target identification candidate region to be determined as label detection region；Or

Subelement is adjusted, for determining that the target identification candidate region identifies classification results accordingly, and according to the target Mark candidate region identifies classification results accordingly and is adjusted to the target identification candidate region, obtains label detection area Domain.

12. processing unit according to claim 11, which is characterized in that the analysis subelement is used for:

By characteristics map information described in the Area generation network analysis in the target detection network after the training, obtain described Positive classification information and negative sort information in characteristics map information；

The positive classification information is determined as to identify candidate region；

The mark candidate region is adjusted according to the offset that the frame of the mark candidate region returns, obtains target Identify candidate region.

13. 0 to 12 described in any item processing units according to claim 1, which is characterized in that second extraction unit is used In:

The characteristic information and residual information of target image to be processed are extracted by second image recognition algorithm；

Target identification classification results are obtained in conjunction with the characteristic information and residual information.

14. processing unit according to any one of claims 11 to 12, which is characterized in that second extraction unit, packet It includes:

Subelement is extracted, for extracting the characteristic information and depth of target image to be processed by the residual error network after training Residual information；

Subelement is identified, for obtaining target identification classification results according to the characteristic information and depth residual information.

15. a kind of storage medium, which is characterized in that the storage medium is stored with a plurality of instruction, and described instruction is suitable for processor It is loaded, the step in 1 to 9 described in any item image processing methods is required with perform claim.