CN110210490A

CN110210490A - Image processing method, device, computer equipment and storage medium

Info

Publication number: CN110210490A
Application number: CN201810167102.0A
Authority: CN
Inventors: 李毅; 张伟辰; 王红法; 肖磊; 薛伟
Original assignee: Shenzhen Tencent Computer Systems Co Ltd
Current assignee: Shenzhen Tencent Computer Systems Co Ltd
Priority date: 2018-02-28
Filing date: 2018-02-28
Publication date: 2019-09-06
Anticipated expiration: 2038-02-28
Also published as: CN110210490B

Abstract

This application involves a kind of image processing methods, device, computer equipment and storage medium, the described method includes: obtaining the prediction area-of-interest of characteristic spectrum, obtain the inclination information of prediction area-of-interest, obtain the dimensional information of object pool matrix, the corresponding inclination information of pond window is determined according to the inclination information of prediction area-of-interest, prediction area-of-interest is divided to obtain according to dimensional information and pond window corresponding inclination information corresponding multiple to pond region, pond is carried out to pond region to each according to default pond algorithm, it obtains each to the corresponding pond result in pond region, it is obtained according to dimensional information each to the corresponding pond result in pond region, form object pool matrix.Prediction area-of-interest is divided to obtain according to the inclination information of prediction area-of-interest multiple to pond region, carries out pond to pond region to multiple, obtain more accurate pond as a result, tilting the feature extraction accuracy of area-of-interest with raising.

Description

Image processing method, device, computer equipment and storage medium

Technical field

This application involves field of computer technology, set more particularly to a kind of image processing method, device, computer Standby and storage medium.

Background technique

With the development of computer technology, there is image detection identification technology, when carrying out image detection identification, due to The problems such as shooting angle, causes the area-of-interest (Region Of Interest, ROI) in the image of shooting to occur tilting Problem, and traditional image-recognizing method, general area-of-interest pond process will tilt area-of-interest and be aligned by coordinate Rectangular window processing, it is understood that there may be it is emerging to be unable to accurate description sense for the phenomenon that feature missing or feature redundancy or feature are misaligned The feature extraction accuracy of the inclination conditions in interesting region, area-of-interest is low.

Summary of the invention

Based on this, it is necessary in view of the above technical problems, provide it is a kind of according to prediction area-of-interest inclination information and The dimensional information of object pool matrix carries out pond to prediction area-of-interest, obtains more accurate pond as a result, inclining to improve Image processing method, device, computer equipment and the storage medium of the feature extraction accuracy of oblique area-of-interest.

A kind of image processing method, comprising:

The prediction area-of-interest of characteristic spectrum is obtained, the inclination information of the prediction area-of-interest is obtained；

Obtain the dimensional information of object pool matrix；

The corresponding inclination information of pond window is determined according to the inclination information of the prediction area-of-interest；

Area is carried out to the prediction area-of-interest according to the dimensional information and the pond window corresponding inclination information Domain divides to obtain corresponding multiple to pond region；

According to default pond algorithm to it is each it is described to pond region carry out pond, obtain with it is each described to pond region Corresponding pond result；

According to the dimensional information obtain it is each it is described to the corresponding pond in pond region as a result, forming the object pool Matrix.

A kind of image data processing system, comprising:

Obtain the dimensional information of object pool matrix；

A kind of computer readable storage medium is stored with computer program, when the computer program is executed by processor, So that the processor executes following steps:

Obtain the dimensional information of object pool matrix；

A kind of computer equipment, including memory and processor, the memory are stored with computer program, the calculating When machine program is executed by the processor, so that the processor executes following steps:

Obtain the dimensional information of object pool matrix；

Above-mentioned image processing method, device, computer equipment and storage medium, the prediction sense for obtaining characteristic spectrum are emerging Interesting region obtains the inclination information of prediction area-of-interest, obtains the dimensional information of object pool matrix；It is interested according to predicting The inclination information in region determines the corresponding inclination information of pond window；According to dimensional information and the corresponding inclination information of pond window to pre- Survey area-of-interest carry out region division obtain it is corresponding multiple to pond region, according to default pond algorithm to each to pond Region carries out pond, obtain with it is each to the corresponding pond in pond region as a result, each to Chi Huaqu according to dimensional information acquisition The corresponding pond in domain is as a result, composition object pool matrix.According to the prediction inclination information of area-of-interest, object pool matrix Dimensional information to prediction area-of-interest carry out region division, obtain it is multiple more acurrate to pond region, to each to Chi Huaqu Domain carries out pond operation, obtains more accurate pond as a result, to improve the feature extraction accuracy of inclination area-of-interest.

Detailed description of the invention

Fig. 1 is the applied environment figure of image processing method in one embodiment；

Fig. 2 is the flow diagram of image processing method in one embodiment；

Fig. 3 is in one embodiment to the schematic diagram in prediction area-of-interest pond；

Fig. 4 is the flow diagram that characteristic spectrum generates in one embodiment；

Fig. 5 is the structure chart of depth convolutional neural networks in one embodiment；

Fig. 6 is the schematic diagram in characteristic pattern pond in a specific embodiment；

Fig. 7 is the flow diagram that inclination information is obtained in one embodiment；

Fig. 8 is the schematic diagram that area-of-interest is predicted in one embodiment；

Fig. 9 is the flow diagram of region division in another embodiment；

Figure 10 is the flow diagram in pond in one embodiment；

Figure 11 is the flow diagram of image processing method in another embodiment；

Figure 12 is the result schematic diagram of image processing method in one embodiment；

Figure 13 is the flow diagram of image processing method in further embodiment；

Figure 14 is the flow diagram in pond in another embodiment；

Figure 15 is the flow diagram of image processing method in a specific embodiment；

Figure 16 is the structural block diagram of image data processing system in one embodiment；

Figure 17 is the structural block diagram of characteristic spectrum generation module in one embodiment；

Figure 18 is the structural block diagram of region division module in one embodiment；

Figure 19 is the structural block diagram of pond module in one embodiment；

Figure 20 is the structural block diagram of image data processing system in another embodiment；

Figure 21 is the structural block diagram of image data processing system in further embodiment；

Figure 22 is the structural block diagram of pond module in another embodiment；

Figure 23 is the structural block diagram of computer equipment in one embodiment.

Specific embodiment

It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and It is not used in restriction the application.

Fig. 1 is the applied environment figure of image processing method in one embodiment.Referring to Fig.1, including 110 He of terminal Server 120.Terminal 110 and server 120 pass through network connection.Terminal 110 specifically can be terminal console or mobile terminal, Mobile terminal specifically can be at least one of mobile phone, tablet computer, laptop etc..Server 120 can be with independent The server cluster of server either multiple servers composition is realized.Terminal 110 obtain image data, to image data into Row processing obtains the prediction area-of-interest of characteristic spectrum, predicts the inclination information of area-of-interest and the dimension of object pool matrix Information is spent, the inclination information of pond window is determined according to inclination information, according to dimensional information and the inclination information of pond window to prediction Area-of-interest carry out region division obtain it is corresponding multiple to pond region, treat pond region progress pond obtain it is corresponding Pond is further processed to obtain processing result image as a result, determine object pool matrix according to pond result to pond matrix Result data is sent to server 120 by data.Server 120 obtains the image data that terminal 110 uploads, to image data Handled the prediction area-of-interest for obtaining characteristic spectrum, the inclination information for predicting area-of-interest and object pool matrix Dimensional information determines the inclination information of pond window according to inclination information, according to dimensional information and the inclination information of pond window to pre- It surveys area-of-interest and carries out region division and obtain corresponding multiple to pond region, treat pond region progress pond and corresponded to Pond as a result, determine object pool matrix according to pond result, to pond matrix to obtaining image procossing for further processing Result data is sent to terminal 110 by result data.

As shown in Fig. 2, in one embodiment, providing a kind of image processing method.The present embodiment is mainly with this Method is applied to the terminal 110 (or server 120) in above-mentioned Fig. 1 to illustrate.Referring to Fig. 2, the image real time transfer side Method specifically comprises the following steps:

Step S202 obtains the prediction area-of-interest of characteristic spectrum, obtains the inclination information of prediction area-of-interest.

Wherein, characteristic spectrum is the image for obtain after feature extraction to image to be processed, and characteristic spectrum includes single-pass Road image and multichannel image, the port number of characteristic spectrum and the port number of image to be processed are consistent, e.g., image to be processed For RGB image, RGB image is 3 channel images, then carrying out the characteristic spectrum that feature extraction obtains by RGB image is 3 channel figures Picture.Pay close attention to region when area-of-interest is image procossing, pay close attention to region can with include but is not limited to box, The modes such as circle, oval or irregular polygon mark.Image only carry out to area-of-interest in data handling procedure Processing, can save the processing time to non-key area, to improve image processing efficiency, avoid feature in non-key area Interference.Predict that area-of-interest includes but is not limited to object area, character area etc..

Specifically, terminal or server obtain the prediction area-of-interest of characteristic spectrum, and predict inclining for area-of-interest Oblique information.Wherein, predict that the inclination information of area-of-interest can be the tilt angle of prediction area-of-interest, or prediction feels emerging The coordinate position and/or geometry side length and/or Geometric center coordinates information in interesting region, by the coordinate bit for predicting area-of-interest It sets and/or the tilt angle of prediction area-of-interest can be calculated in geometry side length and/or Geometric center coordinates information.

Step S204 obtains the dimensional information of object pool matrix.

Wherein, object pool matrix is to carry out the matrix that pond operation obtains to prediction area-of-interest.Object pool square The dimensional information of battle array refers to the elevation information and width information and/or channel information of matrix, and wherein elevation information can use row table Show, width information can be indicated with column.If image is 3 channel images, then convolved image is 3 channel images, obtained Chi Huaju Battle array is 3 access matrixs.Terminal or server obtain the dimensional information of customized object pool matrix, as object pool matrix is One 2 row, 3 column, 3 access matrix, then its dimensional information includes row 2, column 3 and port number 3, is represented by (2,3,3).

In one embodiment, the dimensional information or characteristic spectrum of the dimensional information of object pool matrix and image to be processed Dimensional information it is directly proportional.Image to be processed is 3 channel images of 64 rows 64 column, and object pool matrix is 8 rows 8 column 3 access matrixs.

In one embodiment, object pool matrix is determined according to the dimensional information of prediction area-of-interest and/or area Dimensional information.Predict that area-of-interest is the image-region of 32 rows 64 column, object pool matrix is 4 rows 8 column Matrix.

Step S206 determines the corresponding inclination information of pond window according to the inclination information of prediction area-of-interest.

Wherein, pond (Pooling) is to carry out data sieve to convolution characteristic on the basis of convolution feature extraction Choosing reduces the data handling procedure of convolution characteristic dimension.Pond window is one for carrying out area to prediction area-of-interest The closure frame that domain divides, a prediction area-of-interest can correspond to one or more pond windows.One prediction area-of-interest pair When answering a pond window, region division, different sliding step, to pre- are carried out to prediction area-of-interest by sliding pond window Area-of-interest progress region division is surveyed to obtain dividing region difference.

Specifically, the inclination information of pond window is determined according to the inclination information of prediction area-of-interest, if prediction sense The inclination information in interest region is tilt angle, then the inclination information of pond window is inclination angle identical with prediction area-of-interest Degree, if the inclination information of prediction area-of-interest be the coordinate position and/or geometry side length and/or geometry of prediction area-of-interest The geological informations such as centre coordinate information are then calculated according to the inclination information of prediction area-of-interest and survey inclining for area-of-interest Rake angle, to obtain the tilt angle of pond window.

Step S208 carries out region to prediction area-of-interest according to dimensional information and the corresponding inclination information of pond window and draws Get corresponding multiple to pond region.

Specifically, it is the image-region that will be used to carry out pondization calculating to pond region, can be and pass through to pond region To the image-region that is divided of prediction area-of-interest, a prediction area-of-interest comprising multiple to pond region, It is each between the region of pond there are overlapping region or be not present overlapping region.Region division is carried out to prediction area-of-interest to obtain To multiple to pond region, the dimensional information each to pond region is identical as the dimensional information of pond window, each to Chi Huaqu The tilt angle in domain is identical as the tilt angle of pond window.Wherein, the dimensional information of pond window is according to prediction area-of-interest Dimensional information determine, such as predict area-of-interest be a rectangular area, then according to prediction area-of-interest side length believe The side length information and/or area information of pond window is calculated in breath and/or area information.

Step S210 carries out pond to pond region to each according to default pond algorithm, obtains with each to Chi Huaqu The corresponding pond result in domain.

Specifically, presetting pond algorithm is the fixed pond algorithm before carrying out pond operation, and different pondizations is calculated Method extracts different characteristics of image.Wherein presetting pond algorithm includes but is not limited to averagely pond algorithm (Average Pooling), maximum value pond algorithm (Max Pooling), random pool algorithm (Stochastic Pooling), norm pond Change algorithm etc..To carry out average pond to pond region be by each characteristic value to corresponding pixel in the region of pond to each Average calculating operation is carried out, using the result obtained after average calculating operation as pond as a result, treating pond region using average pond algorithm The feature of overall data can be retained by carrying out pondization, preferably protrude background information.Maximum pond is carried out to pond region to each Be from it is each in the corresponding characteristic value of pixel corresponding in the region of pond choose maximum eigenvalue as pond as a result, using Maximum value pond algorithm, which treats pond region progress pondization, can preferably retain the textural characteristics of image.

Step S212, according to dimensional information obtain it is each to the corresponding pond in pond region as a result, form object pool square Battle array.

Specifically, by it is above-mentioned it is each to the corresponding pond result in pond region according to custom order and object pool matrix It is corresponding.It is if the dimensional information of object pool matrix includes the row information and column information of matrix, then corresponding to pond region by each Pond result it is successively corresponding with the ranks of matrix.

In one embodiment, mesh is sequentially written according to customized to the corresponding pond result of pond pool area by each It marks in the matrix of pond.Such as will be to pond region from left to right, sortord from top to bottom is to each corresponding to pond pool area Pond result be ranked up, according to from left to right, sequence from top to bottom is sequentially written in object pool matrix.Such as default pond Changing algorithm is maximum pond algorithm, by figure in prediction area-of-interest first of the lower left corner to the corresponding pond in pond region As a result at the corresponding position of write-in first, object pool matrix upper left corner matrix element, according to from left to right, from top to bottom Sequence is successively in the pond result write-in pond matrix by each after the pool area of pond.It is indicated with reference to Fig. 3,010 in Fig. 3 Predict that area-of-interest, lattice 040 represent pixel, a lattice represents a pixel.030 frame of dotted line frame in figure Region firmly is one to pond region, each to the corresponding pond in pond region as a result, altogether comprising 6 to pond in figure Region carries out pond to pond region to 6 and obtains corresponding pond result 5,7,8,6,9,5.With first of the lower left corner to At the corresponding position of first matrix element in the upper left corner in the corresponding pond result write-in pond matrix 020 in pond region, According to from left to right, successively pond matrix 020 is written to the corresponding pond result in pond region by each in sequence from top to bottom In, obtaining pond matrix is [5,7,8；6,9,5].

In one embodiment, according to the corresponding relationship of the pixel of characteristic spectrum and the pixel of image to be processed, Position coordinates corresponding with the position coordinates of each pixel to pond region on characteristic spectrum are searched in image to be processed, and Position coordinates corresponding with the position coordinates of each pixel to pond region.

It is interested to obtain prediction by obtaining the prediction area-of-interest of characteristic spectrum for above-mentioned image processing method The inclination information in region obtains the dimensional information of object pool matrix, determines pond according to the inclination information of prediction area-of-interest Change the corresponding inclination information of window, region is carried out to prediction area-of-interest according to dimensional information and the corresponding inclination information of pond window Division obtains corresponding multiple to pond region, carries out pond to pond region to each according to default pond algorithm, obtain with It is each to the corresponding pond in pond region as a result, according to dimensional information obtain it is each to the corresponding pond in pond region as a result, group At object pool matrix.The inclination information that pond window is determined according to the inclination information of prediction area-of-interest, according to inclination information The multiple inclination informations to pond region divided with the dimensional information of object pool matrix are inclined with survey area-of-interest Oblique information is identical, and what the rectangular window processing existing characteristics missing or feature redundancy or feature that reduction is aligned by coordinate were misaligned asks Topic obtains more accurate pond region, improves pond result precision, accurate with the feature extraction for improving inclination area-of-interest Degree.

As shown in figure 4, further include the generation of characteristic spectrum before step S202 in one embodiment, characteristic spectrum Generate the following steps are included:

Step S402 obtains image to be processed, and image to be processed is inputted in neural network model.

Specifically, image to be processed can be the image that capture apparatus is shot, or the image got from file, Wherein image includes the neural network model image or image collection to be trained for training, or for the detection in detection image Image or image collection.Image to be processed is inputted in neural network model, which can be mind to be trained Through network model or the neural network model trained.

Step S404 carries out feature extraction and region of interest to image to be processed by the convolutional layer in neural network model Domain prediction, obtains characteristic spectrum, and characteristic spectrum includes prediction area-of-interest.

Specifically, the convolutional layer in neural network model is used to carry out feature extraction to the image to be processed of input and feel emerging Interesting regional prediction, wherein the feature extraction of convolutional layer mainly carries out convolution algorithm to image to be processed, after convolution algorithm Obtained image is predicted to obtain prediction area-of-interest by the area-of-interest to image as characteristic spectrum, wherein Prediction is exactly to position to area-of-interest, obtains the location information of area-of-interest.

Above-mentioned image processing method, further includes: characteristic spectrum is inputted into pond layer, enters step S202, pond layer It is connect with the convolutional layer in neural network model.

Wherein, in neural network model, convolutional layer is connect with pond layer, and the output of convolutional layer is as the defeated of pond layer Enter.As shown in figure 5, Fig. 5 be a depth convolutional neural networks structure chart, the network structure for input image to be processed into 2 tunnel convolution operation of row, every road include five concatenated convolution units, and the study of image advanced features is completed by convolution unit, Full articulamentum is accessed later.Wherein convolution unit includes convolutional layer (conv), pond layer (pool) and regularization layer (rnorm). Conv i-j indicates that the jth road convolutional layer of i-th of convolution unit, pool i-1 indicate the jth Lu Chihua of i-th of convolution unit Layer, rnorm i-1 indicate that the jth road regularization floor of i-th convolution unit, fc are full articulamentum, and full articulamentum is defeated per all the way Enter all comprising upper one layer per output all the way, full articulamentum is used to carry out the feature that convolution unit is extracted comprehensive operation, Quan Lian It connects layer and connects output layer later.

Specifically, characteristic spectrum is that convolutional layer carries out the convolved image obtained after feature extraction to image to be processed.Chi Hua Layer can be for carrying out pond operation when pond layer carries out pond operation to characteristic spectrum to characteristic spectrum to entire feature Map carries out pond, is also possible to carry out pond to the partial region of characteristic spectrum.

As shown in fig. 6,050 being characterized map in Fig. 6,060 is pond matrix, to the progress of characteristic spectrum 050 maximum value pond Change, obtains pond matrix and obtain 060, pond matrix 060 is the matrix of 3 rows 3 column, and pond matrix 060 is specially [8,7,8； 6,5,9；7,9,5].

As shown in fig. 7, in one embodiment, step S202 includes:

Step S2022 obtains the corresponding apex coordinate of prediction area-of-interest, and prediction area-of-interest is rectangular area, Prediction tilt angle is calculated according to trigonometric function relationship according to apex coordinate.Or

It is corresponding with prediction area-of-interest to obtain the corresponding geometric center point coordinate of prediction area-of-interest by step S2024 Apex coordinate, prediction tilt angle is calculated according to geometric center point coordinate and apex coordinate.

It should be understood that although each step in the flow chart of Fig. 7 is successively shown according to the instruction of arrow, this A little steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly state otherwise herein, these steps It executes there is no the limitation of stringent sequence, these steps can execute in other order.

Specifically, prediction area-of-interest area includes but is not limited to pass through prediction area-of-interest apex coordinate and/or several What center point coordinate and/or area and/or side length information and/or tilt angle etc. are described.As shown in figure 8, being used in Fig. 8 The information of description prediction area-of-interest includes but is not limited to apex coordinate (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), Geometric center point coordinate (Xc, Yc), side length information wide W and high H, tilt angle theta.

Prediction area-of-interest is described using above-mentioned apex coordinate, geometric center point coordinate, side length information and tilt angle Mode include it is a variety of, in order to simplify expression, and the mesh of the freedom degree of constrained forecast E area-of-interest can be using its middle part Point information is described, and remaining information can pass through geometrical relationship and determine.It can such as be retouched using (Xc, Yc), W, H and θ State prediction area-of-interest.By known (Xc, Yc), W, H and θ can be calculated (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4).Shown in specific calculating process such as formula (1):

Wherein W>=1, H>=1 ,-pi/2<θ≤+ pi/2.

In one embodiment, using (X1, Y1), (X2, Y2) and H description prediction area-of-interest.By known (X1, Y1), (X2, Y2) and H are calculated shown in the detailed process such as formula (2) of (X3, Y3), (X4, Y4), (Xc, Yc), W and θ:

Wherein X2 >=X1, H >=1.

In another embodiment, prediction area-of-interest is described using (X1, Y1), (X3, Y3) and W.

In yet another embodiment, using (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4) description prediction region of interest Domain.

As shown in figure 9, in one embodiment, step S208 includes:

Pond window is calculated according to the first side length of the first dimensional information and prediction area-of-interest in step S2082 The second side length of pond window is calculated according to the second side length of the second dimensional information and prediction area-of-interest for first side length.

Specifically, the dimensional information of object pool matrix includes the first dimensional information and the second dimensional information, the first dimension The corresponding elevation information of information is row information or width information is column information, predicts that the first side length of area-of-interest can be width Degree or elevation information.Using the first dimension as row information, predict that the first side length of area-of-interest is pond window for elevation information The first side length be elevation information, the height of pond window is equal to the elevation information for predicting area-of-interest divided by object pool matrix Row information.Second side length of pond window is width information, and the width information of pond window is equal to the width of prediction area-of-interest Information is divided by object pool matrix column information.

Step S2084, according to the first side length of pond window, the second side length of pond window and the corresponding inclination information of pond window Obtain sliding pond window.

Specifically, sliding pond window is that the elevation information of an elevation information and width information and pond window and width are believed Manner of breathing is same, inclination information closure frame identical with pond window inclination information.Referring to Fig. 3, the 030 inclined rectangular frame constituted in figure It is a sliding pond window.

Step S2086 is slided sliding pond window in prediction area-of-interest to obtain corresponding and sliding pond The area of window is identical multiple to pond region.

Specifically, sliding pond window being slided in prediction area-of-interest, sliding step can be customized, when When sliding step is identical as the sliding width information of pond window, multiple area equations are obtained, and nonoverlapping to pond region. The region division to prediction area-of-interest is easy to implement by sliding pond window and sliding step.

As shown in Figure 10, in one embodiment, step S210 includes:

Step S2102 obtains the pixel for obtaining from characteristic spectrum currently to pond region and currently occupying to pond region Point obtains currently to pond pixel collection.

Wherein, currently to pond region refer to one of them multiple in the region of pond will carry out pond to pond Region.The pixel currently occupied to pond region, the set being made of the pixel of each occupancy are obtained from characteristic spectrum For currently to pond pixel collection.It is currently big to pixel each in pool area and the overlapping region area currently to pool area Indicate that the pixel is currently occupied to pond region when 0, currently occupying pixel to pond region includes interior pixels point With edge pixel point, wherein the entire pixel region of interior pixels point expression is inside currently to pond region, edge pixel point Pixel partial region is indicated inside currently to pond region, pixel partial region is currently to pond region exterior.Ginseng According to Fig. 3, currently to pond region by inclined rectangular frame 030 mark, characteristic value be 10 pixel with currently to pond region There is no overlapping region, therefore the pixel that characteristic value is 10 is not the pixel currently occupied to pond region, characteristic value is 5 Pixel with currently to pond region, there are overlapping regions, and entirely pixel region inside currently to pond region, Gu Te The pixel that value indicative is 5 is interior pixels point, the pixel and currently there are department overlay regions to pond region that characteristic value is 2 The pixel partial region that domain, i.e. characteristic value are 2 is inside currently to pond region, and partial region is outside currently to pond region Portion, therefore the pixel that characteristic value is 2 is edge pixel point.

Step S2104, according to currently to each currently to pond pixel currently to pond in the pixel collection of pond Area acquisition in region is each currently to the corresponding characteristic value of pond pixel, obtains currently corresponding to pond to pond region Change characteristic value collection.

Specifically, to currently to pond region carry out pond before, need to currently to pond edges of regions pixel progress Determine, determines edge pixel point whether as currently to the corresponding pixel in pond region.Whether edge pixel point is used as currently Judgment mode to the corresponding pixel in pond region includes but is not limited to according to edge pixel point currently in the region of pond Area determined with currently to which whether the area ratio outside the region of pond is greater than preset threshold, when edge pixel point is corresponding Ratio be greater than preset threshold when, determine edge pixel point be currently to the corresponding pixel in pond region.It obtains currently to pond The characteristic value for changing the corresponding pixel in region, obtains currently corresponding to pond characteristic value collection to pond region.

Step S2106 treats pond characteristic value collection progress pond according to default pond algorithm and obtains currently to Chi Huaqu Domain is corresponding to work as forebay result.

Specifically, it is consistent with the default pond algorithm in step S210 to preset pond algorithm.According to default pond algorithm pair It is currently corresponding to pond characteristic value collection progress pond to pond region, it obtains currently corresponding when forebay to pond region As a result.

In one embodiment, obtain it is next to the conduct of pond region currently to pond region, into from characteristic spectrum The step of pixel that middle acquisition is currently occupied to pond region is obtained currently to pond pixel collection, is completed currently to pond The pond operation in region obtains currently corresponding when forebay is as a result, until multiple to each in the region of pond to pond region It is a to complete pond operation to pond region, it obtains with each to the corresponding pond result in pond region.

By to it is each whether belong to pond edges of regions pixel each carried out to the corresponding pixel in pond region Determine, obtains accurately the accuracy in pond being improved, to obtain more accurate characteristics of image to pond region.

As shown in figure 11, in one embodiment, after step S212, further includes:

Step S214 obtains the corresponding pixel collection of each pond result on characteristic spectrum.

Specifically, each pond result is corresponding with each pond region, obtains on characteristic pattern to each pond region pair The pixel answered obtains pixel collection corresponding with each pond region.

Step S216 obtains each Chi Huajie in object pool matrix in the back-propagation process of neural network model The corresponding initial passback gradient of fruit.

Specifically, the output error of pond layer in the backpropagation that gradient is neural network model is initially returned.With input Layer is in bottom, and output layer is for top, and the output error of current layer is all that a upper layer network for current layer passes in neural network Pass the error of current layer.Each layer of output error is determined according to the difference between true output and anticipated output, If the error of output layer is being determined according to the anticipated output result and the true difference exported between result of output layer.

Step S218 determines the corresponding target pixel points of each pixel collection according to default pond algorithm, according to initial Returning gradient is that the corresponding target pixel points of each pixel collection distribute target passback gradient.

Specifically, the pixel collection having an impact to each pond result is determined according to pond algorithm, it will be to each pond The pixel collection that change result has an impact is as target pixel points corresponding with each pond result.If pond algorithm is It is that maximum eigenvalue is corresponding in each pond region to the pixel that each pond result has an impact when Max pooling Pixel, the target passback gradient of the corresponding pixel of maximum eigenvalue is initial passback in each pond region on characteristic spectrum Gradient, the target passback of other pixels on characteristic spectrum in each pond region in addition to the corresponding pixel of maximum eigenvalue Gradient is 0.It is each pond to the pixel that each pond result has an impact if pond algorithm is Average Pooling Change the corresponding all pixels point in region, the corresponding initial passback of each pond result is terraced divided by the corresponding pixel of each pond result Point number obtains each target to each pixel in the region of pond and returns gradient.If being to the corresponding pixel in pond region [1,2,3,4；1, Isosorbide-5-Nitrae, 4；2,1, Isosorbide-5-Nitrae；Isosorbide-5-Nitrae, 2,4], the initial passback gradient of object pool matrix is [5,2；8,4], then to The target passback gradient of each pixel is [5/4,5/4,1/2,1/2 in the region of pond；5/4,5/4,1/2,1/2；2,2,1, 1；2,2,1,1].

In one embodiment, determine that current predictive is interested according to the corresponding pond matrix of each prediction area-of-interest Region is the probability of target output.If the target output of current predictive area-of-interest is character area, to prediction region of interest The corresponding pond matrix in domain is handled, and determines that the target output of current predictive area-of-interest is the probability of character area.Such as It include 4 prediction area-of-interests in figure, respectively marked as 082,084,086 and 088 shown in Figure 12.Wherein marked as 082 Prediction area-of-interest is that the probability of character area is 1, and the prediction area-of-interest marked as 084 is that the probability of character area is 0.99, the prediction area-of-interest marked as 086 is that the probability of character area is 0.96, the prediction region of interest marked as 088 Domain is that the probability of character area is 0.98.

As shown in figure 13, in one embodiment, above-mentioned image processing method, further includes:

Subject thread number is calculated according to the number of prediction area-of-interest and dimensional information in step S220.

Wherein, the number for predicting area-of-interest is the number counted to the areal of prediction area-of-interest According to prediction area-of-interest is equipped with prediction region of interest domain identifier, and wherein region of interest domain identifier is including but not limited to interested Zone number such as predicts that the number of area-of-interest is R, then region of interest Field Number includes but is not limited to r=0,1,2 ..., R-1.Subject thread number is calculated according to the number R of prediction area-of-interest and dimensional information PH, PW and channel C. Subject thread number total_thread_num=R × PH × PW × C.

Step S222 opens up parallel thread corresponding with subject thread number.

Specifically, the parallel thread number opened up is identical as subject thread number, wherein referring to multiple events or work parallel Dynamic to take place at the same instant, parallel thread refers to multiple program synchronizations in identical or different (Central Processing Unit, CPU) or graphics processor (Graphics Processing Unit, GPU) on be performed simultaneously, GPU It is a kind of microprocessor for specially doing image operation work, GPU can be characterized map and distribute thread by pixel granularity, i.e., one defeated A thread can be corresponded to out, as each pond result corresponds to a thread in object pool matrix.The many-core parallel computation of GPU Model can handle multiple images simultaneously, and carrying out processing to image by the many-core parallel computational model of GPU can be improved image Processing speed.

Step S210, comprising:

Step S2108 is assigned to multiple in parallel thread to pond region, is calculated by parallel thread according to default pondization Method carries out parallel pond to pond region to each, obtains with each to the corresponding pond result in pond region.

Specifically, one carries out pond operation according to default pond algorithm by a parallel thread to pond region, often The pond result that a thread obtains with it is each corresponding to pond region.Such as will currently it distribute to pond region into parallel thread One thread, by the thread to currently pond operation is carried out to pond region, obtain with currently to the corresponding pond in pond region Change result.GPU parallel thread is distributed to pond region by each, while carrying out pond operation to pond region to each, is added The processing speed of fast prediction area-of-interest.

As shown in figure 14, in one embodiment, step S2108 includes:

Step S2108a, according to prediction region of interest domain identifier, the matrix element position of dimensional information and object pool matrix It sets and determines thread number to be allocated.

Specifically, prediction region of interest domain identifier is for identifying the including but not limited to interested of prediction area-of-interest Zone number or label etc..Thread number is the number for identifying thread, according to prediction region of interest domain identifier, dimensional information and The matrix element position of object pool matrix determines thread number to be allocated.Such as according to r-th of RoI, object pool matrix PH, PW, The thread number to be allocated determined at (ph, pw, c) is thread_id=(r × PH × PW+ph × PW+pw) × C+c.

Step S2108b distributes thread number to be allocated to each parallel thread, and each parallel thread, which exists, to be corresponded Subject thread number.

Specifically, each parallel thread distributes a thread number, obtains thread number corresponding with each parallel thread, will Unique thread number corresponding with each parallel thread is as subject thread number.If the number of parallel thread is total_ Thread_num=R × PH × PW × C, the subject thread number of each parallel thread are thread_id=(r × PH × PW+ph ×PW+pw)×C+c。

Step S2108c determines that the corresponding target of each parallel thread waits for pond region according to subject thread number.

Specifically, pond corresponding with subject thread number is determined by subject thread number and the dimensional information of object pool matrix Region, using the pond region as object pool region.If subject thread number is thread_id=(r × PH × PW+ph × PW + pw) × C+c, then it is calculated according to the dimensional information of subject thread number and object pool matrix corresponding with subject thread number Prediction number is r=thread_id/ (PH × PW × C).According to determining number and the dimensional information of object pool matrix with to The corresponding relationship in pond region determines the corresponding object pool region of subject thread number.

Step S2108d, each parallel thread wait for that pond region carries out parallel to corresponding target according to default pond algorithm Chi Hua obtains pond corresponding with each thread number result.

Specifically, presetting pond algorithm is algorithm identical with the default pond algorithm in step S210.According to default pond Change algorithm and pond region, which carries out parallel pond, is waited for corresponding target by parallel thread, obtains pond corresponding with each thread number Change result.Each parallel thread handles an object pool region, obtains a pond result.

Step S2108e, according to the matrix element position of thread number and prediction region of interest domain identifier, object pool matrix Incidence relation, the corresponding pond result of each thread number is written to the matrix element of object pool matrix corresponding with thread number At position.

Specifically, thread number is matrix element position and the mesh according to prediction region of interest domain identifier and object pool matrix The dimensional information for marking pond matrix determines, therefore thread number pass corresponding with the matrix element position existence anduniquess of object pool matrix System is specifically write at the matrix element position of the output result write-in object pool matrix of the corresponding parallel thread of thread number Entering mode can be customized according to demand.

In one embodiment, object pool corresponding with thread number is being written into the corresponding pond result of each thread number After at the matrix element position of matrix, comprising: obtain the corresponding picture of the corresponding pond result of each thread number on special image Vegetarian refreshments set obtains corresponding with each thread number in object pool matrix in the back-propagation process of neural network model The corresponding initial passback gradient of each pond result, determines the corresponding target picture of each pixel collection according to default pond algorithm Vegetarian refreshments is that the corresponding target pixel points of each pixel collection distribute target passback gradient according to initial passback gradient.

As shown in figure 15, in a specific embodiment, image processing method, comprising:

Step S602 obtains the prediction area-of-interest of characteristic spectrum and characteristic spectrum, predicts the region of area-of-interest The inclination information of mark, prediction area-of-interest.

Specifically, in neural network model, convolution algorithm is carried out to image to be processed by convolutional layer and obtains characteristic pattern Spectrum obtains N to open height being H after such as carrying out convolution algorithm post-processing to image to be processed, and width W, port number is the feature of C Map is numbered each characteristic spectrum to obtain a figure ID (figure ID=0,1,2 ..., N-1), schemes ID unique identification feature Figure, if the area-of-interest total number on N characteristic spectrums is R, each area-of-interest existence anduniquess ROI is identified, wherein RoI Mark includes but is not limited to ROI number r, if RoI (r)=[figure ID, X1, Y1, X2, Y2, H].ROI (r) is for describing prediction sense The inclination information of area-of-interest can be calculated by the coordinate information of ROI in ROI (r), specifically calculated for interest region Shown in journey such as formula (3).

Step S604 obtains the dimensional information of object pool matrix.

Specifically, the dimensional information of object pool matrix includes elevation information, width information and the channel information of matrix, Middle object pool matrix is the target output that pond is carried out to prediction area-of-interest, the channel information of object pool matrix and spy The channel information for levying map is consistent.It is PH if object pool matrix is a height, width PW, port number is the matrix of C.

Step S606 feels prediction according to the inclination information of the dimensional information of object pool matrix and prediction area-of-interest Interest region progress region division obtains multiple to pond region.

Specifically, predict that the inclination information of area-of-interest determines the inclination information of pond window, according to object pool square The dimensional information of battle array and the inclination information of pond window obtain prediction area-of-interest progress region division multiple to Chi Huaqu Domain.The dimensional information of object pool matrix includes the first dimensional information and the second dimensional information, according to prediction area-of-interest First side length information and the first dimensional information of object pool matrix determine the first side length of pond window, interested according to predicting The second side length information in region and the second dimensional information of object pool matrix determine the second side length of pond window, according to first Side length, the second side length and the corresponding inclination information of pond window obtain sliding pond window, and sliding pond window is emerging in prediction sense Sliding pond window is slided to obtain corresponding multiple to pond region in interesting region.

Step S608 determines subject thread according to the dimensional information of the number of prediction area-of-interest and object pool matrix Number opens up parallel thread corresponding with subject thread number.The number for predicting area-of-interest is R, object pool matrix Dimensional information includes height PH, width PW and port number C, is determined according to the number of above-mentioned prediction area-of-interest and dimensional information Subject thread number, wherein subject thread data total_thread_num=R × PH × PW × C.

Step S610, according to the dimensional information and object pool matrix of prediction region of interest domain identifier, object pool matrix Matrix element position determine thread number to be allocated, thread number to be allocated is distributed to each parallel thread, each parallel thread There are one-to-one subject threads number.Such as predict that area-of-interest is identified as prediction ROI number, current predictive ROI number position R, the dimensional information of object pool matrix include height PH, width PW and port number C, the matrix element position of object pool matrix For (ph, pw, c), then thread number to be allocated is thread_id=(r × PH × PW+ph × PW+pw) × C+c, by line to be allocated Journey number distribution is to each parallel thread, the corresponding subject thread number of each parallel thread existence anduniquess.

Step S612 according to subject thread number, the dimensional information of object pool matrix, predicts that the association of area-of-interest is closed System determines that the corresponding target of each parallel thread waits for pond region.The corresponding subject thread number of each parallel thread existence anduniquess, The corresponding pond region to be processed of subject thread number is determined according to the dimensional information of subject thread number and object pool matrix.According to The corresponding ROI number r=thread_id/ (PH of single line thread to be processed is calculated in the thread number thread_id of current thread ×PW×C).According to the incidence relation determination of ROI number, the position coordinates of object pool matrix and subject thread number and score The corresponding target of journey number waits for pond region.

Step S614, according to the matrix element position of thread number and prediction region of interest domain identifier, object pool matrix The matrix element position of object pool matrix corresponding with thread number is written in the corresponding pond result of each thread number by incidence relation Set place.Such as according to prediction region of interest domain identifier r, is searched in RoI (r), obtain the figure ID of characteristic spectrum.According to current thread Number determine the position of the output result write-in object pool matrix of current thread for (ph, pw, c), wherein ph=(thread_id/ PW/C) %PH, pw=(thread_id/C) %PW, c=thread_id%C.According to the dimensional information of object pool matrix and The wide BW and high BH of the pond window of r-th of ROI is calculated in RoI (r), wherein BH=ROI (r) .H/PH, width BW=ROI (r).W/PW.According to the tilt angle of BW, BH, ph, pw and r-th of ROI, calculate current thread pond window to be processed relative to Shown in the offset coordinates of (X1, Y1) such as formula (3):

X1'=X1+pwBWcos θ+phBHcos θ, Y1'=Y1-pwBWcos θ+phBHcos θ (3)

Coordinate set of the corresponding pond window to be processed of current thread in characteristic spectrum is calculated according to offset coordinates Shown in S, coordinate set S such as formula (4):

During the forward calculation of neural network, characteristic point coordinate set in the corresponding pond window to be processed of current thread S, acquisition characteristic value collection D=value | and value=Map (n, y, x, c), (x, y) ∈ S }, example is turned to maximum pond, from feature Set D chooses maximum eigenvalue as pond calculated result, i.e. Pool (r, ph, pw, c)=MAX { D } records current thread pair The pixel position coordinates (XmaxYmax) of maximum eigenvalue in the pond window answered.

During the retrospectively calculate of neural network, the gradient that note passes back on pond layer output Pool isThen Current thread handles corresponding passback gradient(r, ph, pw, c), the corresponding pixel position coordinates of maximum eigenvalue Input return gradient Current thread processing The input passback gradient of other pixels is 0 in corresponding pond window.

As shown in figure 16, in one embodiment, a kind of image data processing system 200 is provided, comprising:

Data acquisition module 202, for obtaining the prediction area-of-interest of characteristic spectrum, predicting the inclination of area-of-interest The dimensional information of information and object pool matrix.

Data processing module 204, for determining the corresponding inclination of pond window according to the inclination information of prediction area-of-interest Information.

Region division module 206 is used for according to dimensional information and the corresponding inclination information of pond window to prediction region of interest Domain progress region division obtains corresponding multiple to pond region.

Pond module 208, for according to default pond algorithm to it is each to pond region progress pond, obtain with it is each to The corresponding pond in pond region as a result, according to dimensional information obtain it is each to the corresponding pond in pond region as a result, form target Pond matrix.

As shown in figure 17, in one embodiment, image data processing system 200 further includes characteristic spectrum generation module 210, characteristic spectrum generation module 210 includes:

Image to be processed is inputted neural network mould for obtaining image to be processed by image acquisition unit 2102 to be processed In type.

Image processing unit 2104 is mentioned for carrying out feature to image to be processed by the convolutional layer in neural network model It takes and is predicted with area-of-interest, obtain characteristic spectrum, characteristic spectrum includes prediction area-of-interest.

Image data processing system 200 further include:

Data input module 212, for characteristic spectrum to be inputted pond layer, into data acquisition module, pond layer and mind Through the convolutional layer connection in network model.

In one embodiment, data acquisition module 202 is for obtaining the corresponding apex coordinate of prediction area-of-interest, in advance Survey area-of-interest is rectangular area, and prediction tilt angle is calculated according to trigonometric function relationship according to apex coordinate.Or

The corresponding geometric center point coordinate of prediction area-of-interest apex coordinate corresponding with prediction area-of-interest is obtained, Prediction tilt angle is calculated according to geometric center point coordinate and apex coordinate.

As shown in figure 18, in one embodiment, region division module 206 includes:

Side length computing unit 2062, for being calculated according to the first side length of the first dimensional information and prediction area-of-interest To the first side length of pond window, pond window is calculated according to the second side length of the second dimensional information and prediction area-of-interest Second side length.

Sliding window determination unit 2064, for according to the first side length of pond window, the second side length of pond window and pond The corresponding inclination information of window obtains sliding pond window.

Area division unit 2066, for slide in the prediction area-of-interest sliding pond window To corresponding identical the multiple to pond region with the sliding area of pond window.

As shown in figure 19, in one embodiment, pond module 208 includes:

Pixel acquiring unit 2082 obtains currently to pond for obtaining currently to pond region from characteristic spectrum The pixel that region occupies obtains currently to pond pixel collection.

Characteristic value acquiring unit 2084, for according to currently to each currently to pond pixel in the pixel collection of pond It is each currently to the corresponding characteristic value of pond pixel to the area acquisition in the region of pond currently, it obtains currently to Chi Huaqu Domain is corresponding to pond characteristic value collection.

Pond unit 2086, for according to default pond algorithm treat pond characteristic value collection carry out pond obtain currently to Pond region is corresponding to work as forebay result.

As shown in figure 20, in one embodiment, image data processing system 200 further include:

Pixel collection obtains module 214, for obtaining the corresponding pixel point set of each pond result on characteristic spectrum It closes.

Initial passback gradient obtains module 216, for obtaining object pool in the back-propagation process of neural network model Change the corresponding initial passback gradient of each pond result in matrix.

Target returns gradient distribution module 218, for determining that each pixel collection is corresponding according to default pond algorithm Target pixel points are that the corresponding target pixel points of each pixel collection distribute target passback gradient according to initial passback gradient.

As shown in figure 21, in one embodiment, image data processing system 200 further include:

Number of threads computing module 220, for mesh to be calculated according to the number and dimensional information of prediction area-of-interest Mark number of threads.

Thread opens up module 222, for opening up parallel thread corresponding with the subject thread number.

Pond module 208 is also used to be assigned to multiple in parallel thread to pond region, by parallel thread according to pre- If pond algorithm carries out parallel pond to pond region to each, obtain with each to the corresponding pond result in pond region.

As shown in figure 22, in one embodiment, pond module 208 further include:

Thread number determination unit 2088, for according to prediction region of interest domain identifier, dimensional information and object pool matrix Matrix element position determine thread number to be allocated.

Thread number allocation unit 2090, for distributing thread number to be allocated to each parallel thread, each parallel thread There are one-to-one subject threads number.

Target waits for pond area determination unit 2092, for determining the corresponding mesh of each parallel thread according to subject thread number Mark is to pond region.

Thread pool unit 2094 waits for Chi Huaqu to corresponding target according to default pond algorithm for each parallel thread Domain carries out parallel pond, obtains pond corresponding with each thread number result.

Pond result writing unit 2096, for according to thread number and prediction region of interest domain identifier, object pool matrix Matrix element position incidence relation, object pool corresponding with thread number is written into the corresponding pond result of each thread number At the matrix element position of matrix.

Figure 23 shows the internal structure chart of computer equipment in one embodiment.The computer equipment specifically can be figure Terminal 110 (or server 120) in 1.As shown in figure 23, it includes passing through system which, which includes the computer equipment, Processor, memory, network interface, input unit and the display screen of bus connection.Wherein, memory includes non-volatile memories Medium and built-in storage.The non-volatile memory medium of the computer equipment is stored with operating system, can also be stored with computer Program when the computer program is executed by processor, may make processor to realize image processing method.In the built-in storage Computer program can also be stored, when which is executed by processor, processor may make to execute image real time transfer Method.The display screen of computer equipment can be liquid crystal display or electric ink display screen, the input dress of computer equipment It sets and can be the touch layer covered on display screen, be also possible to the key being arranged on computer equipment shell, trace ball or touch-control Plate can also be external keyboard, Trackpad or mouse etc..

It will be understood by those skilled in the art that structure shown in Figure 23, only part relevant to application scheme The block diagram of structure, does not constitute the restriction for the computer equipment being applied thereon to application scheme, and specific computer is set Standby may include perhaps combining certain components or with different component layouts than more or fewer components as shown in the figure.

In one embodiment, image data processing system provided by the present application can be implemented as a kind of computer program Form, computer program can be run in computer equipment as shown in figure 23.Group can be stored in the memory of computer equipment At each program module of the image data processing system, for example, data acquisition module 202, data processing mould shown in Figure 16 Block 204, region division module 206 and pond module 208.The computer program that each program module is constituted executes processor Step in the image processing method of each embodiment of the application described in this specification.

For example, computer equipment shown in Figure 23 can pass through the data in image data processing system as shown in figure 16 It obtains module 202 and executes the prediction area-of-interest for obtaining characteristic spectrum, the inclination information and object pool for predicting area-of-interest Change the dimensional information of matrix.Computer equipment can be implemented in inclining according to prediction area-of-interest by data processing module 204 Oblique information determines the corresponding inclination information of pond window.Computer equipment can be executed by region division module 206 to be believed according to dimension Breath inclination information corresponding with pond window obtains prediction area-of-interest progress region division corresponding multiple to pond region. Computer equipment can be executed by pond module 208 carries out pond to pond region to each according to default pond algorithm, obtains With it is each to the corresponding pond in pond region as a result, according to dimensional information obtain it is each to the corresponding pond in pond region as a result, Form object pool matrix.

In one embodiment, a kind of computer equipment, including memory and processor, the memory storage are provided There is computer program, when the computer program is executed by the processor, so that the processor executes following steps: obtaining The prediction area-of-interest of characteristic spectrum obtains the inclination information of the prediction area-of-interest, obtains object pool matrix Dimensional information, according to prediction area-of-interest inclination information determine the corresponding inclination information of pond window, according to dimensional information with The corresponding inclination information of pond window to prediction area-of-interest carry out region division obtain it is corresponding multiple to pond region, according to Default pond algorithm to each to pond region progress pond, obtain with it is each to the corresponding pond in pond region as a result, according to Dimensional information obtain it is each to the corresponding pond in pond region as a result, form object pool matrix.

In one embodiment, the computer program also makes the processor execute following steps: obtaining to be processed Image, by image to be processed input neural network model in, by the convolutional layer in neural network model to image to be processed into Row feature extraction and area-of-interest prediction obtain characteristic spectrum, and characteristic spectrum includes prediction area-of-interest, by characteristic spectrum Pond layer is inputted, volume the step of into the prediction area-of-interest for obtaining characteristic spectrum, in pond layer and neural network model Lamination connection.

In one embodiment, the inclination information of prediction area-of-interest is obtained, comprising: obtain prediction area-of-interest pair The apex coordinate answered, prediction area-of-interest are rectangular area, are calculated according to apex coordinate according to trigonometric function relationship pre- Tilt angle is surveyed, or obtains the corresponding geometric center point coordinate of prediction area-of-interest vertex corresponding with prediction area-of-interest Prediction tilt angle is calculated according to geometric center point coordinate and apex coordinate in coordinate.

In one embodiment, prediction area-of-interest is carried out according to dimensional information and pond window corresponding inclination information Region division obtains corresponding multiple to pond region, comprising: according to the first dimensional information and the first of prediction area-of-interest The first side length of pond window is calculated in side length, is calculated according to the second side length of the second dimensional information and prediction area-of-interest To the second side length of pond window, believed according to the first side length of pond window, the second side length of pond window and the corresponding inclination of pond window Breath obtains sliding pond window, and sliding pond window is slided to obtain corresponding and sliding pond in prediction area-of-interest The area of window is identical multiple to pond region.

In one embodiment, according to default pond algorithm to it is each to pond region carry out pond, obtain with it is each to The corresponding pond result in pond region, comprising: obtain currently to pond region, obtained from characteristic spectrum currently to pond region The pixel of occupancy obtains currently to pond pixel collection, according to currently to each currently to pond in the pixel collection of pond Pixel currently in the region of pond area obtain it is each currently to the corresponding characteristic value of pond pixel, obtain currently to Pond region is corresponding to pond characteristic value collection, treats pond characteristic value collection progress pond according to default pond algorithm and obtains It is currently corresponding when forebay result to pond region.

In one embodiment, according to dimensional information obtain it is each to the corresponding pond in pond region as a result, form target After the matrix of pond, the computer program also makes the processor execute following steps: obtaining on characteristic spectrum each Result corresponding pixel collection in pond obtains each in object pool matrix in the back-propagation process of neural network model The corresponding initial passback gradient of a pond result, determines the corresponding object pixel of each pixel collection according to default pond algorithm Point is that the corresponding target pixel points of each pixel collection distribute target passback gradient according to initial passback gradient.

In one embodiment, the computer program also makes the processor execute following steps: being felt according to prediction Subject thread number is calculated in the number and dimensional information in interest region, opens up parallel line corresponding with subject thread number Journey carries out pond to pond region to each according to default pond algorithm, obtains with each to the corresponding Chi Huajie in pond region Fruit, comprising: be assigned to multiple in parallel thread to pond region, by parallel thread according to default pond algorithm to it is each to Pond region carries out parallel pond, obtains with each to the corresponding pond result in pond region.

In one embodiment, it is assigned to multiple in parallel thread to pond region, by parallel thread according to default Pond algorithm carries out parallel pond to pond region to each, obtains with each to the corresponding pond result in pond region, comprising: Thread number to be allocated is determined according to the matrix element position of prediction region of interest domain identifier, dimensional information and object pool matrix, Thread number to be allocated is distributed to each parallel thread, there are one-to-one subject threads number for each parallel thread, according to mesh Mark thread number determines that the corresponding target of each parallel thread waits for pond region, and each parallel thread is according to default pond algorithm to right The target answered waits for that pond region carries out parallel pond, obtains pond corresponding with each thread number as a result, according to thread number and in advance The incidence relation for surveying the matrix element position of region of interest domain identifier, object pool matrix, by the corresponding pond of each thread number As a result it is written at the matrix element position of object pool matrix corresponding with thread number.

In one embodiment, a kind of computer readable storage medium is provided, computer program, computer program are stored with When being executed by processor, so that processor executes following steps: obtaining the prediction area-of-interest of characteristic spectrum, obtain described pre- The inclination information of area-of-interest is surveyed, the dimensional information of object pool matrix is obtained, is believed according to the inclination of prediction area-of-interest It ceases and determines the corresponding inclination information of pond window, according to dimensional information and the corresponding inclination information of pond window to prediction area-of-interest Carry out region division obtain it is corresponding multiple to pond region, according to default pond algorithm to each to pond region progress pond Change, obtain with it is each to the corresponding pond in pond region as a result, each to the corresponding pond in pond region according to dimensional information acquisition Change as a result, composition object pool matrix.

Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the program can be stored in a non-volatile computer and can be read In storage medium, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, provided herein Each embodiment used in any reference to memory, storage, database or other media, may each comprise non-volatile And/or volatile memory.Nonvolatile memory may include that read-only memory (ROM), programming ROM (PROM), electricity can be compiled Journey ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..

Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance Shield all should be considered as described in this specification.

The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously The limitation to the application the scope of the patents therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art For, without departing from the concept of this application, various modifications and improvements can be made, these belong to the guarantor of the application Protect range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.

Claims

1. a kind of image processing method, comprising:

Obtain the dimensional information of object pool matrix；

Region is carried out to the prediction area-of-interest according to the dimensional information and the corresponding inclination information of the pond window to draw Get corresponding multiple to pond region；

According to default pond algorithm to it is each it is described to pond region carry out pond, obtain with it is each described corresponding to pond region Pond result；

According to the dimensional information obtain it is each it is described to the corresponding pond in pond region as a result, forming the object pool square Battle array.

2. the method according to claim 1, wherein the generation step of the characteristic spectrum, comprising:

Image to be processed is obtained, it will be in the image input neural network model to be processed；

Feature extraction is carried out to the image to be processed by the convolutional layer in the neural network model and area-of-interest is pre- It surveys, obtains the characteristic spectrum, the characteristic spectrum includes the prediction area-of-interest；

The method also includes:

By the characteristic spectrum input pond layer, into it is described acquisition characteristic spectrum prediction area-of-interest the step of, it is described Pond layer is connect with the convolutional layer in the neural network model.

3. the method according to claim 1, wherein the inclination information includes tilt angle, the acquisition institute State the inclination information of prediction area-of-interest, comprising:

The corresponding apex coordinate of the prediction area-of-interest is obtained, the prediction area-of-interest is rectangular area, according to institute It states apex coordinate and the prediction tilt angle is calculated according to trigonometric function relationship；Or

Obtain the corresponding geometric center point coordinate of the prediction area-of-interest and the corresponding vertex of the prediction area-of-interest The prediction tilt angle is calculated according to the geometric center point coordinate and the apex coordinate in coordinate.

4. the method according to claim 1, wherein the dimensional information includes the first dimensional information and the second dimension Information is spent, it is described that the prediction area-of-interest is carried out according to the dimensional information and the pond window corresponding inclination information Region division obtains corresponding multiple to pond region, comprising:

The of the pond window is calculated according to the first side length of first dimensional information and the prediction area-of-interest One side length；

The of the pond window is calculated according to the second side length of second dimensional information and the prediction area-of-interest Two side lengths；

It is obtained according to the first side length of the pond window, the second side length of the pond window and the corresponding inclination information of the pond window To sliding pond window；

The sliding pond window is slided to obtain corresponding and sliding pond window in the prediction area-of-interest The area of mouth is identical the multiple to pond region.

5. the method according to claim 1, wherein it is described according to default pond algorithm to each described to pond Region carries out pond, obtain with it is each described to the corresponding pond result in pond region, comprising:

It obtains currently to pond region, the pixel currently occupied to pond region is obtained from the characteristic spectrum and is obtained Currently to pond pixel collection；

According to it is described currently in the pixel collection of pond it is each currently to pond pixel described currently in the region of pond Area obtain it is each currently to the corresponding characteristic value of pond pixel, obtain described currently corresponding to pond to pond region Characteristic value collection；

According to default pond algorithm to it is described to pond characteristic value collection carry out pond obtain it is described currently to pond region correspondence Work as forebay result.

6. the method according to claim 1, wherein described each described to pond according to dimensional information acquisition After changing the corresponding pond in region as a result, forming the object pool matrix, further includes:

The corresponding pixel collection of each pond result is obtained on the characteristic spectrum；

In the back-propagation process of the neural network model, it is corresponding to obtain each pond result in the object pool matrix Initial passback gradient；

The corresponding target pixel points of each pixel collection are determined according to the default pond algorithm, according to described initial time Passing gradient is that the corresponding target pixel points of each pixel collection distribute target passback gradient.

7. the method according to claim 1, wherein the method also includes:

Subject thread number is calculated according to the number of the prediction area-of-interest and the dimensional information；

Open up parallel thread corresponding with the subject thread number；

It is described according to default pond algorithm to it is each it is described to pond region carry out pond, obtain with it is each described to pond region Corresponding pond result, comprising:

It is assigned to the multiple in the parallel thread to pond region, by the parallel thread according to default pond algorithm To it is each it is described carry out parallel pond to pond region, obtain with it is each described to the corresponding pond result in pond region.

8. the method according to the description of claim 7 is characterized in that it is described by it is the multiple to pond region be assigned to it is described simultaneously In line journey, by the parallel thread according to default pond algorithm to it is each it is described carry out parallel pond to pond region, obtain To with it is each described to the corresponding pond result in pond region, comprising:

It is determined according to the matrix element position of prediction region of interest domain identifier, dimensional information and the object pool matrix to be allocated Thread number；

The thread number to be allocated is distributed to each parallel thread, there are one-to-one mesh for each parallel thread Mark thread number；

Determine that the corresponding target of each parallel thread waits for pond region according to the subject thread number；

Each parallel thread waits for that pond region carries out parallel pond to corresponding target according to default pond algorithm, obtain with The corresponding pond result of each thread number；

It is associated with according to thread number with the matrix element position of the prediction region of interest domain identifier, the object pool matrix System, by the matrix element of the corresponding pond result write-in of each thread number object pool matrix corresponding with the thread number At position.

9. a kind of image data processing system, which is characterized in that described device includes:

Data acquisition module, for obtaining the prediction area-of-interest of characteristic spectrum, the inclination letter of the prediction area-of-interest The dimensional information of breath and object pool matrix；

Data processing module, for determining that window corresponding inclination in pond is believed according to the inclination information of the prediction area-of-interest Breath；

Region division module, for feeling emerging to the prediction according to the dimensional information and the corresponding inclination information of the pond window Interesting region progress region division obtains corresponding multiple to pond region；

Pond module, for according to default pond algorithm to it is each it is described to pond region progress pond, obtain with it is each described To the corresponding pond in pond region as a result, being obtained according to the dimensional information each described to the corresponding Chi Huajie in pond region Fruit forms the object pool matrix.

10. device according to claim 9, which is characterized in that described device further includes characteristic spectrum generation module, described Characteristic spectrum generation module includes:

Image acquisition unit to be processed will be in the image input neural network model to be processed for obtaining image to be processed；

Image processing unit is mentioned for carrying out feature to the image to be processed by the convolutional layer in the neural network model It takes and is predicted with area-of-interest, obtain the characteristic spectrum, the characteristic spectrum includes the prediction area-of-interest；

Described device further include:

Data input module, for the characteristic spectrum to be inputted pond layer, into data acquisition module, the pond layer and institute State the convolutional layer connection in neural network model.

11. device according to claim 9, which is characterized in that the region division module includes:

Side length computing unit, for being calculated according to the first side length of first dimensional information and the prediction area-of-interest To the first side length of the pond window, calculated according to the second side length of second dimensional information and the prediction area-of-interest Obtain the second side length of the pond window；

Sliding window determination unit, for according to the first side length of the pond window, the second side length of the pond window and described Window corresponding inclination information in pond obtains sliding pond window；

Area division unit, it is corresponding for being slided to obtain the sliding pond window in the prediction area-of-interest It is identical the multiple to pond region with the sliding area of pond window.

12. device according to claim 9, which is characterized in that the pond module includes:

Pixel acquiring unit obtains described currently to pond for obtaining currently to pond region from the characteristic spectrum The pixel that region occupies obtains currently to pond pixel collection；

Characteristic value acquiring unit is used for according to described currently to each currently to pond pixel in institute in the pixel collection of pond State currently in the region of pond area obtain it is each currently to the corresponding characteristic value of pond pixel, obtain described currently to pond It is corresponding to pond characteristic value collection to change region；

Pond unit, for according to default pond algorithm to it is described to pond characteristic value collection carry out pond obtain it is described currently to Pond region is corresponding to work as forebay result.

13. device according to claim 9, which is characterized in that described device further include:

Number of threads computing module, for mesh to be calculated according to the number and the dimensional information of the prediction area-of-interest Mark number of threads；

Thread opens up module, for opening up parallel thread corresponding with the subject thread number；

The pond module is also used to be assigned to the multiple in the parallel thread to pond region, by described parallel Thread according to default pond algorithm to it is each it is described carry out parallel pond to pond region, obtain with it is each described to pond region Corresponding pond result.

14. a kind of computer readable storage medium is stored with computer program, when the computer program is executed by processor, So that the processor is executed such as the step of any one of claims 1 to 8 the method.

15. a kind of computer equipment, including memory and processor, the memory is stored with computer program, the calculating When machine program is executed by the processor, so that the processor executes the step such as any one of claims 1 to 8 the method Suddenly.