CN110310343A - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN110310343A CN110310343A CN201910451850.6A CN201910451850A CN110310343A CN 110310343 A CN110310343 A CN 110310343A CN 201910451850 A CN201910451850 A CN 201910451850A CN 110310343 A CN110310343 A CN 110310343A
- Authority
- CN
- China
- Prior art keywords
- image
- key area
- target image
- notable
- critical areas
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The disclosure provides a kind of image processing method and device, is related to technical field of image processing, is able to solve in prior art the problem of cannot be considered in terms of high-quality display and low occupied bandwidth to the processing of image.The specific technical proposal is: target image is inputted the key area and non-critical areas that preset conspicuousness detection neural network determines target image；To obtaining the corresponding image data of key area after the corresponding image coded treatment of key area, and passes through the preset generator for generating confrontation network and generate the corresponding image data of non-critical areas.The disclosure in image procossing for taking into account high-quality display and low occupied bandwidth.
Description
Technical field
This disclosure relates to technical field of image processing more particularly to image processing method and device.
Background technique
Image is the main channel that people obtain information, and with the development of technology, picture quality is higher and higher, correspondingly,
Image data amount is also increasing, the transmission for image, since transmission Time Bandwidth is limited, is typically necessary and presses image
It is transmitted again after contracting processing.
Conventional images compression can be divided into lossy compression and lossless compression two major classes.The Lossy Compression Algorithm of industry mainstream at present
Such as JPEG etc., design object are the compressed files as far as possible under the premise of not influencing the mankind's distinguishable picture quality
Size, it means that eliminate the raw information of a part of picture；And lossless compression algorithm such as PNG-24 uses direct color
Dot chart, the data volume after lossless compression is five times of lossy compression or so, and still, the promotion of display effect is relatively small.
Have the following problems in current existing image procossing scheme: lossy compression image display effect is poor, lossless compression
Occupied bandwidth is high, in the limited situation of transmission bandwidth, cannot be considered in terms of high-quality display and low occupied bandwidth to the processing of image.
Summary of the invention
The embodiment of the present disclosure provides a kind of image processing method and device, is able to solve in prior art to image
Processing cannot be considered in terms of the problem of high-quality display and low occupied bandwidth.The technical solution is as follows:
According to the first aspect of the embodiments of the present disclosure, a kind of image processing method is provided, this method comprises:
By target image input preset conspicuousness detection neural network determine target image key area and non-pass
Key range；
To obtaining the corresponding image data of key area after the corresponding image coded treatment of key area, and by preset
The generator for generating confrontation network generates the corresponding image data of non-critical areas.
The key area and non-critical areas that neural network determines target image are detected by conspicuousness, for non-pass
The image of key range uses generation confrontation network generation display effect more preferable and the smaller image of data volume, for key area
Image does not do the compression processing of high compression ratio, to can take into account height to the processing of image in the limited situation of transmission bandwidth
Quality is shown and low occupied bandwidth.
In one embodiment, target image is inputted into the pass that preset conspicuousness detection neural network determines target image
Key range and non-critical areas, comprising:
Target image input conspicuousness detection neural network is obtained into the corresponding notable figure of target image；
The key area and non-critical areas of target image are determined according to notable figure.
In one embodiment, to obtaining the corresponding picture number of key area after the corresponding image coded treatment of key area
According to, comprising:
The corresponding image data of key area is obtained after being handled using lossless compression algorithm the corresponding image of key area.
The parts of images that user pays close attention to can be effectively ensured using lossless compression algorithm processing for the image of key area
The picture quality of content improves user experience.
In one embodiment, the key area and non-critical areas of target image are determined according to notable figure, comprising:
Notable figure is handled according to preset threshold；Wherein, pixel of the gray value greater than preset threshold in notable figure
It is assigned a value of 1, the pixel that gray value is less than or equal to preset threshold in notable figure is assigned a value of 0；
Matrix point multiplication operation is carried out according to treated notable figure and target image and determines key area, and according to key area
Domain and target image determine non-critical areas.
In one embodiment, conspicuousness detection neural network and generation fight the convolutional neural networks for including in network
Middle data type is 16 fixed points；The sparse matrix that conspicuousness detects neural network and generates in confrontation network is according to following
What mode stored: being converted into binary matrix according to nonzero value is denoted as 1 after the nonzero value in preset order storage sparse matrix
And it stores.
By simplifying the data storage of data type and sparse matrix in convolutional neural networks, it can effectively save and deposit
Store up space and can be with speed up processing.
In one embodiment, conspicuousness detection neural network is built based on VGG convolutional neural networks.
According to the second aspect of an embodiment of the present disclosure, a kind of image processing apparatus is provided, which includes:
Determining module, for target image to be inputted the key that preset conspicuousness detection neural network determines target image
Region and non-critical areas；
Processing module, for obtaining the corresponding picture number of key area after the corresponding image coded treatment of key area
According to, and the corresponding image data of non-critical areas is generated by the preset generator for generating confrontation network.
The key area and non-critical areas that neural network determines target image are detected by conspicuousness, for non-pass
The image of key range uses generation confrontation network generation display effect more preferable and the smaller image of data volume, for key area
Image does not do the compression processing of high compression ratio, to can take into account height to the processing of image in the limited situation of transmission bandwidth
Quality is shown and low occupied bandwidth.
In one embodiment, determining module is specifically used for:
Target image input conspicuousness detection neural network is obtained into the corresponding notable figure of target image；
The key area and non-critical areas of target image are determined according to notable figure.
In one embodiment, processing module is specifically used for:
The corresponding image data of key area is obtained after being handled using lossless compression algorithm the corresponding image of key area.
The parts of images that user pays close attention to can be effectively ensured using lossless compression algorithm processing for the image of key area
The picture quality of content improves user experience.
In one embodiment, determining module is specifically for including:
Notable figure is handled according to preset threshold；Wherein, pixel of the gray value greater than preset threshold in notable figure
It is assigned a value of 1, the pixel that gray value is less than or equal to preset threshold in notable figure is assigned a value of 0；
Matrix point multiplication operation is carried out according to treated notable figure and target image and determines key area, and according to key area
Domain and target image determine non-critical areas.
In one embodiment, conspicuousness detection neural network and generation fight the convolutional neural networks for including in network
Middle data type is 16 fixed points；The sparse matrix that conspicuousness detects neural network and generates in confrontation network is according to following
What mode stored: being converted into binary matrix according to nonzero value is denoted as 1 after the nonzero value in preset order storage sparse matrix
And it stores.
By simplifying the data storage of data type and sparse matrix in convolutional neural networks, it can effectively save and deposit
Store up space and can be with speed up processing.
In one embodiment, conspicuousness detection neural network is built based on VGG convolutional neural networks.
The image processing method and device that the embodiment of the present disclosure provides detect neural network by conspicuousness and determine target
The key area and non-critical areas of image generate display effect using confrontation network is generated for the image of non-critical areas
The more preferable and smaller image of data volume, does not do the image of key area the compression processing of high compression ratio, thus in transmission belt
In the limited situation of width, high-quality display and low occupied bandwidth can be taken into account to the processing of image.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.
Fig. 1 is a kind of flow diagram for image processing method that the embodiment of the present disclosure provides；
Fig. 2 is in the embodiment of the present disclosure for describing a schematic diagram of conspicuousness；
Fig. 3 is the schematic diagram for the notable figure that the embodiment of the present disclosure provides；
Fig. 4 is the flow diagram for generating confrontation network in the embodiment of the present disclosure and generating image；
Fig. 5 is a kind of implementation diagram for image processing method that the embodiment of the present disclosure provides；
Fig. 6 is a kind of effect diagram for image processing method that the embodiment of the present disclosure provides；
Fig. 7 is in the embodiment of the present disclosure for describing a schematic diagram of sparse matrix storage；
Fig. 8 is a kind of structural schematic diagram for image processing apparatus that the embodiment of the present disclosure provides.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the disclosure.
The embodiment of the present disclosure provides a kind of image processing method, be applied to image transmission apparatus, such as can be server,
The equipment such as mobile terminal, as shown in Figure 1, the image processing method the following steps are included:
101, by target image input preset conspicuousness detection neural network determine target image key area and
Non-critical areas.
When vision significance (Visual Attention Mechanism, VA) is referred in face of a scene, the mankind are automatic
Ground handles area-of-interest and selectively ignores region of loseing interest in, these people's area-of-interests are referred to as showing
Write region.Such as the jellyfish in the ocean Fig. 2 is more significant with respect to Yu Haiyang.
Vision significance detection calculate refer to using convolutional neural networks (Convolutional Neural Networks,
CNN the vision noticing mechanism for) simulating people, calculates the significance level of information in visual field.
In one embodiment of the present disclosure, conspicuousness detection neural network can be built based on VGG convolutional neural networks.
The advantages of VGG convolutional neural networks, is a simplified neural network structure, builds conspicuousness detection mind based on VGG convolutional neural networks
Training effectiveness can be improved in the training stage through network, and obtain good effect in practical applications.Specifically, the disclosure
The VGG16 that embodiment can be used in VGG convolutional neural networks builds conspicuousness detection neural network and carries out conspicuousness to image
Detection, VGG16 are Oxford University's computer vision group and a kind of depth convolutional neural networks that DeepMind company researches and develops jointly,
It is usually utilized to extract characteristics of image, the large size using ImageNet as training set for the research of visual object identification software is visual
Change database, VGG16 can recognize 1000 kinds of objects.
It should be noted that needing after the conspicuousness detection neural network in the embodiment of the present disclosure is put up using a large amount of
Image is trained achieve the desired results after carry out again using.Specific training method and process are referred to correlation technique data
Understand, is not detailed in the embodiment of the present disclosure.
In one embodiment, step 101 can specifically include:
1011, target image input conspicuousness detection neural network is obtained into the corresponding notable figure of target image.
The significant angle value of marking area is higher than the significant angle value in non-significant region, and significant angle value is significance measure parameter
Value, significant angle value is by the calculated value of presetting method, according to the available notable figure of significant angle value.
Specifically, by after target image input conspicuousness detection neural network, for each pixel in target image,
By the sigmoid function in VGG convolutional neural networks in the full articulamentum of the last layerExport each pixel
The corresponding significant angle value of point is the value in [0,1] section, and the gray value interval for being quantized to [0,255] is visual in a manner of grayscale image
Notable figure is turned to, significant angle value is higher, then and it is brighter in grayscale image, such as the notable figure in Fig. 3.Grayscale image be it is a kind of have from
The black monochrome image to white 256 grades of gray scales or grade, each pixel in the image is indicated with 8 data, therefore pixel
Point value is between 256 kinds of gray scales between black and white.
1012, the key area and non-critical areas of target image are determined according to notable figure.
In one embodiment, step 1012, it can specifically include:
1012a, notable figure is handled according to preset threshold；Wherein, gray value is greater than preset threshold in notable figure
Pixel is assigned a value of 1, and the pixel that gray value is less than or equal to preset threshold in notable figure is assigned a value of 0.
Wherein, notable figure is that significant angle value of the target image by significance detection neural network output quantifies to 0-255
An obtained grayscale image, stores the significance information of each pixel of target image, a substantially corresponding numerical value value
Matrix in [0,255] section.
Specifically, handling according to preset threshold notable figure, the corresponding picture of gray value of the preset threshold will be above
Vegetarian refreshments is assigned a value of 1；Pixel corresponding lower than the gray value of the preset threshold is assigned a value of 0.
It is noted that the size that the difference of preset threshold will lead to key area changes.Preset threshold is higher,
The judge of key area is stringenter, and critical area is smaller.With the decline of preset threshold, have around significant content in image
More pixels are defined as crucial pixel, and critical area increases.In addition, with the increase of critical area,
It can decode to obtain the higher picture of quality in decoder end, but lossless reservation or the data of better quality compression is needed to get over
More, the storage of image can also become larger.
1012b, key area is determined according to treated notable figure and target image progress matrix point multiplication operation, and according to
Key area and target image determine non-critical areas.
Specifically, notable figure, after the processing of step 1012a, the value of element is 0 or 1 in corresponding matrix, utilize
The corresponding matrix of notable figure that treated matrix corresponding with target image does the matrix of the available key area of point multiplication operation
So that it is determined that key area, the corresponding matrix of target image, which subtracts the corresponding matrix of key area, can be obtained by non-critical areas
Matrix so that it is determined that non-critical areas.It is noted that notable figure is corresponding when preset threshold is 255 in step 1012a
Matrix in element value be 0 all 0, then by after point multiplication operation determination target image whole region be non-key area
Domain.In addition, can also determine non-critical areas above by what point multiplication operation obtained, key area in the embodiment of the present disclosure/non-
Key area determines that the high region of significance can be key area and be also possible to non-critical areas according to actual needs.
It should be noted that for target image multistage key can also be formed by carrying out grade classification to significant angle value
Region.Such as by the significant angle value of range 0 to 255 be divided into five sections [0,50), [51,100), [101,150), [151,
200),[201,255].The setting of gray value threshold value can also be determined according to the division in significance section, such as ash can be set
Angle value threshold value is 0,50,100,150 or 200.Aforementioned 5 threshold values can be used for step 1012a respectively to carry out notable figure
Processing, it is then corresponding with target image using the corresponding matrix of 5 notable figures obtained after aforementioned processing respectively in step 1012b
Matrix do and determine 5 grades of key areas and corresponding 5 grades of non-critical areas after point multiplication operation.
102, to obtaining the corresponding image data of key area after the corresponding image coded treatment of key area, and by pre-
If the generator of generation confrontation network generate the corresponding image data of non-critical areas.
Confrontation network (Generative Adversarial Networks, GAN) is generated in 2014 by Ian Good
Fellow proposes that thought originates from game theory, by a generator (Generator) and an arbiter in his paper
(Discriminator) it forms.Confrontation network is generated, to solve the problems, such as being how to learn new samples out from training sample,
Training sample is that picture just generates new picture, and training sample is that article just exports new article etc..
Specifically, the high region in the region namely significance that can pay close attention to user in target image determines key area
Lossless compression/low compression ratio coded treatment is carried out to its corresponding image section behind domain, retains higher quality and more
Details；It is distribution based on view data since GAN generates image, human eye is more accepted, at present in some picture qualities
It can be more than the Standard of image compression such as JPEG, JPEG2000, WebP even PNG in evaluation index and subjective naked eyes likability, it is right
In the corresponding image of non-critical areas, under the premise of same picture size, compared to using high compression ratio coding processing mode,
GAN has the better potential advantages of subjective quality, and the image human eye likability generated by GAN more preferably, therefore can be to non-pass
Key range generates corresponding image data by GAN.Preferably, generally when conditions permit, to key in step 102
The corresponding image in region is handled using lossless compression algorithm.
The training objective of GAN is that generator will cheat arbiter as much as possible.Generator and arbiter are confronted with each other, constantly
Adjusting parameter, final purpose are to make: 1) arbiter can not judge whether the output result for generating network is true；2) confrontation net is generated
Network can generate the picture mixed the spurious with the genuine.In order to make it easy to understand, generator can be interpreted as one " thief of forge money ",
Arbiter is interpreted as " police of identification counterfeit money ", and thief makes counterfeit money, and it is counterfeit money that police, which determines this, and then thief constantly progresses greatly
Recognition capability is also continuously improved in fraud technology, police, and until thief produces the counterfeit money mixed the spurious with the genuine, police can not judge very
It is pseudo-.
GAN needs to be trained using great amount of samples image, and training process is as follows: sample image passes through the embodiment of the present disclosure
In above-mentioned conspicuousness detection neural network to determine the key area and non-critical areas of sample image, by encoder and amount
Change device and characteristic pattern is generated to sample image；In conjunction with foregoing teachings, generation generator generates at random according to the characteristic pattern of key area
Some very poor images, are then fed into generation arbiter, and arbiter can be quasi- according to original image as two classifiers
Really classify, can be set as the image output 0 to differ greatly with original image to generation, to differing with original image for generation
Small image output 1.For the purpose of the decline of objective function, the parameter of training generator makes two generation generators generate preferably figure
Picture, input generation arbiter are identified as true i.e. output 1, then train the parameter of arbiter, two generation arbiters is made to determine for two generations
It is false that generator, which generates picture,.And so on third time, the 4th time ..., with the iteration of frequency of training, the weight of generator
Parameter changes, and so that the pixel distribution of the image generated is intended to the pixel distribution of original image, until arbiter can not differentiate life
At image and the fitting of original image, that is, network, be both at this time 0.5 through arbiter output valve, epicycle training terminates.Meanwhile
The picture that also needing to be added in the training process makes the restrictive condition for generating image size to generate is less than original picture size,
And then achieve the effect that picture compression.
The process for generating image using trained GAN is as follows: as shown in figure 4, passing through encoder and amount to original image
Change and obtain corresponding characteristic pattern after device is handled, storage characteristic pattern is compressed actual size, and characteristic pattern is sent into and is generated
Device decoding (decoder is generator, the process that the decoded process of decoder namely generator generate in Fig. 4) restores image.Its
In, the characteristic pattern obtained after image coding (convolution) quantization stores the information of image, (warp after decoding using decoder
Product) it obtains and image similar in original image.
In addition, the storage that store the feature vector after the quantization in the pilot process of image information is that compression of images is big
It is small, the compressed images storage size S of GAN Web compression is all used for whole regionGANCalculation formula is as follows:
Referring to Fig. 4, in above-mentioned formula, W is the width of picture to be compressed, and H is the height of picture to be compressed, and N is that drop is adopted times
Number represents width/high minification of characteristic pattern after coded quantization, and C is characterized the port number of figure, and L is quantization digit (as do not quantified
The data memory format of preceding each weight parameter is floating-point 32, i.e. digit is 32, and digit becomes after being quantized into the weight that series is 8
For log2 8=3), the storage size for generating image is controlled by adjusting N, C, L.Arithmetic coding etc. is used further for characteristic pattern
Entropy coding mode can further decrease memory space, obtain higher compression ratio.
For additional fraction significance region, original image image position can be recorded using binary coding, be overlapped.
Wherein the size in significance region needs additional addition, the storage size maximum value S in significance regionkeyAre as follows:
Skey=Sum (key pixel) * log2DkeyUnit: bit
In above-mentioned formula；Key pixel is the pixel number of key area (with higher significance), DkeyIt is each
The corresponding pixel depth of pixel.Because Partial key picture can also use lossless compression or other lossy compression modes thus
It is compressed, so storage size will be generally less than Skey, therefore the image size S under embodiment of the present disclosure technical solution meets
Following inequality relation:
S≤SGAN+Skey
It is noted that after GAN completes training in actual use, it, can since generator has been learned to generate image
Only retain generator to remove arbiter, can continue to reserve judgement device, the embodiment of the present disclosure does not limit this.In addition,
When to target image treated data volume size has different require when, when if any realizing higher compression ratios demand, can reduce
Key area chooses lesser key area, as in Fig. 5 multistage significance region to personage's lossless compression at boat center/
Low compression ratio compression, rest part are generated with GAN；When if any the demand for reducing compression ratio, key area can be increased and chosen
Biggish key area, if lossless compression equal to personage and boat/low compression ratio is compressed in Fig. 5 multistage significance region,
Remaining part point is generated with GAN.Clearly as the latter's constricted zone is smaller, superimposed picture storage is bigger, and compression ratio is smaller.It is real
In the application of border, as shown in figure 5, different key areas can be partitioned into for selecting key area face according to compression ratio demand
Long-pending size.
It is also pointed out that GAN network has background the generation of better quality, but there is the small mesh of a large amount of details
Mark can not effectively retain, or even cannot recognize that the information of object.As shown in fig. 6, Fig. 6 a is for original image and accordingly
Marking area schematic diagram (non-significant figure, this schematic diagram are to more intuitively show to vision significance content)；Figure
6b is the picture directly generated with GAN, can find out from detail view and be lost largely carefully in the high building of road end distal end
Section, the stop signpost " P " by road can not generate well, but the objects such as these building, direction boards often have compared with
High information content；Fig. 6 c, which show the technical solution provided according to the embodiment of the present disclosure, to be had firstly in image compared with multi information
Marking area identified, retained with better quality or low compression ratio compression, rest part utilized with high compression ratio
The image that GAN is obtained after being generated.Compared to the picture directly generated with GAN in Fig. 6 b, by the region high to significance
After carrying out high quality reservation, details abundant can be remained to find out the high building of road distal end in the detail view of Fig. 6 c, by road
Stop signpost " P " can also identify.
In one embodiment, conspicuousness detection neural network and generation fight the convolutional neural networks for including in network
Middle data type is 16 fixed points；The sparse matrix that conspicuousness detects neural network and generates in confrontation network is according to following
What mode stored: being converted into binary matrix according to nonzero value is denoted as 1 after the nonzero value in preset order storage sparse matrix
And it stores.
Specifically, the weight for detecting neural network for conspicuousness and generating CNN used in confrontation network is joined
Quantification, such as 16 fixed points are converted by 32 floating-points of data type, can have under the requirement for guaranteeing propaedeutics effect
Effect saves memory space.Neural network is detected for conspicuousness and generates the sparse matrix occurred in confrontation network, using new
Storage scheme, as shown in fig. 7, in 4 × 4 sparse matrixes it is each value be 16 integer datas, altogether occupy 16 × 4 × 4=256
Position.Binary matrix occupies 4 × 4=16 storages according to regulation journal nonzero value position；It is every according to certain journal
A nonzero value, each nonzero value are 16 integer datas, occupy 16 × 5=80 storages altogether, therefore store square using new departure
Battle array saves 256- (16+80)=160 storage, and matrix storage reduces 62.5%.
It is noted that being conducive to be transplanted on the portable terminals such as mobile phone for being compressed with for neural network model, and mention
The arithmetic speed of high network.The transplanting compressed and accelerate that model can be improved is carried out for compression of images model neural network based
Property with compression speed (number/second of compressed picture).
In one embodiment, the method can also include；
103, the non-critical areas figure of treated key area image data and generation is sent to image receiving apparatus
As data.
Specifically, by the corresponding matrix of key area image data square corresponding with the non-critical areas image data of generation
Battle array is added, and image data of the matrix of complete image data to be completed can be obtained, and image transmission apparatus can incite somebody to action
After aforementioned two parts data merge the image data completed, decoded after being sent to image receiving apparatus by image receiving apparatus
Obtain the high quality graphic of corresponding target image content；Alternatively, image transmission apparatus can also directly send two parts data
Merge after image receiving apparatus, after being received by image receiving apparatus and is decoded after the image data completed to obtain again pair
Answer the high quality graphic of target image content.
Above-mentioned image receiving apparatus can be the equipment such as mobile terminal, server, and the image in the embodiment of the present disclosure is sent
Equipment, image receiving apparatus are a kind of functionally division modes to equipment, and in practical application, some equipment may be at one
It is image transmission apparatus under scene, may is image receiving apparatus under another scene.
The image processing method that the embodiment of the present disclosure provides detects neural network by conspicuousness and determines target image
Key area and non-critical areas, for non-critical areas image using generate confrontation network generate display effect it is more preferable and
The smaller image of data volume, does not do the image of key area the compression processing of high compression ratio, thus limited in transmission bandwidth
In the case where, high-quality display and low occupied bandwidth can be taken into account to the processing of image.
Based on image processing method described in above-mentioned corresponding embodiment, following is embodiment of the present disclosure, can
For executing above-mentioned embodiments of the present disclosure.
The embodiment of the present disclosure provides a kind of image processing apparatus, as shown in figure 8, the image processing apparatus 80 includes:
Determining module 801 determines target image for target image to be inputted preset conspicuousness detection neural network
Key area and non-critical areas；
Processing module 802, for obtaining the corresponding image of key area after the corresponding image coded treatment of key area
Data, and the corresponding image data of non-critical areas is generated by the preset generator for generating confrontation network.
The key area and non-critical areas that neural network determines target image are detected by conspicuousness, for non-pass
The image of key range uses generation confrontation network generation display effect more preferable and the smaller image of data volume, for key area
Image does not do the compression processing of high compression ratio, to can take into account height to the processing of image in the limited situation of transmission bandwidth
Quality is shown and low occupied bandwidth.
In one embodiment, determining module 801 is specifically used for:
Target image input conspicuousness detection neural network is obtained into the corresponding notable figure of target image；
The key area and non-critical areas of target image are determined according to notable figure.
In one embodiment, processing module 802 is specifically used for:
The corresponding image data of key area is obtained after being handled using lossless compression algorithm the corresponding image of key area.
The parts of images that user pays close attention to can be effectively ensured using lossless compression algorithm processing for the image of key area
The picture quality of content improves user experience.
In one embodiment, determining module 801 is specifically for including:
Notable figure is handled according to preset threshold；Wherein, pixel of the gray value greater than preset threshold in notable figure
It is assigned a value of 1, the pixel that gray value is less than or equal to preset threshold in notable figure is assigned a value of 0；
Matrix point multiplication operation is carried out according to treated notable figure and target image and determines key area, and according to key area
Domain and target image determine non-critical areas.
In one embodiment, conspicuousness detection neural network and generation fight the convolutional neural networks for including in network
Middle data type is 16 fixed points；The sparse matrix that conspicuousness detects neural network and generates in confrontation network is according to following
What mode stored: being converted into binary matrix according to nonzero value is denoted as 1 after the nonzero value in preset order storage sparse matrix
And it stores.
By simplifying the data storage of data type and sparse matrix in convolutional neural networks, it can effectively save and deposit
Store up space and can be with speed up processing.
In one embodiment, conspicuousness detection neural network is built based on VGG convolutional neural networks.
The advantages of VGG convolutional neural networks, is a simplified neural network structure, is built significantly based on VGG convolutional neural networks
Property detection neural network can improve training effectiveness in the training stage, and obtain good effect in practical applications.
The image processing apparatus that the embodiment of the present disclosure provides detects neural network by conspicuousness and determines target image
Key area and non-critical areas, for non-critical areas image using generate confrontation network generate display effect it is more preferable and
The smaller image of data volume, does not do the image of key area the compression processing of high compression ratio, thus limited in transmission bandwidth
In the case where, high-quality display and low occupied bandwidth can be taken into account to the processing of image.
Based on image processing method described in the corresponding embodiment of above-mentioned Fig. 1, the embodiment of the present disclosure also provides one kind
Computer readable storage medium, for example, non-transitorycomputer readable storage medium can be read-only memory (English: Read
Only Memory, ROM), it is random access memory (English: Random Access Memory, RAM), CD-ROM, tape, soft
Disk and optical data storage devices etc..It is stored with computer instruction on the storage medium, for executing the corresponding embodiment of above-mentioned Fig. 1
Described in image processing method, details are not described herein again.
Those skilled in the art will readily occur to its of the disclosure after considering specification and practicing disclosure disclosed herein
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Person's adaptive change follows the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure
Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are wanted by right
It asks and points out.
Claims (10)
1. a kind of image processing method, which is characterized in that the described method includes:
By target image input preset conspicuousness detection neural network determine the target image key area and non-pass
Key range；
To obtaining the corresponding image data of the key area after the corresponding image coded treatment of the key area, and by pre-
If the generator of generation confrontation network generate the corresponding image data of the non-critical areas.
2. the method according to claim 1, wherein described input preset conspicuousness detection mind for target image
The key area and non-critical areas of the target image are determined through network, comprising:
The target image is inputted into the conspicuousness detection neural network and obtains the corresponding notable figure of the target image；
The key area and non-critical areas of the target image are determined according to the notable figure.
3. the method according to claim 1, wherein described to the corresponding image coded treatment of the key area
After obtain the corresponding image data of the key area, comprising:
The corresponding image of the key area is obtained after being handled using lossless compression algorithm the corresponding image of the key area
Data.
4. according to the method described in claim 2, it is characterized in that, described determine the target image according to the notable figure
Key area and non-critical areas, comprising:
The notable figure is handled according to preset threshold；Wherein, gray value is greater than the preset threshold in the notable figure
Pixel be assigned a value of 1, in the notable figure gray value be less than or equal to the preset threshold pixel be assigned a value of 0；
Matrix point multiplication operation, which is carried out, according to treated the notable figure and the target image determines the key area, and root
The non-critical areas is determined according to the key area and the target image.
5. the method according to claim 1, wherein conspicuousness detection neural network and the generation pair
Data type is 16 fixed points in the convolutional neural networks for including in anti-network；Conspicuousness detection neural network and described
The sparse matrix generated in confrontation network stores in the following way: storing in the sparse matrix according to preset order
The nonzero value 1 is denoted as after nonzero value to be converted into binary matrix and store.
6. a kind of image processing apparatus, which is characterized in that described device includes:
Determining module, for target image to be inputted the key that preset conspicuousness detection neural network determines the target image
Region and non-critical areas；
Processing module, for obtaining the corresponding image of the key area after the corresponding image coded treatment of the key area
Data, and the corresponding image data of the non-critical areas is generated by the preset generator for generating confrontation network.
7. device according to claim 6, which is characterized in that the determining module is specifically used for:
The target image is inputted into the conspicuousness detection neural network and obtains the corresponding notable figure of the target image；
The key area and non-critical areas of the target image are determined according to the notable figure.
8. device according to claim 6, which is characterized in that the processing module is specifically used for:
The corresponding image of the key area is obtained after being handled using lossless compression algorithm the corresponding image of the key area
Data.
9. device according to claim 7, which is characterized in that the determining module is specifically for including:
The notable figure is handled according to preset threshold；Wherein, gray value is greater than the preset threshold in the notable figure
Pixel be assigned a value of 1, in the notable figure gray value be less than or equal to the preset threshold pixel be assigned a value of 0；
Matrix point multiplication operation, which is carried out, according to treated the notable figure and the target image determines the key area, and root
The non-critical areas is determined according to the key area and the target image.
10. device according to claim 6, which is characterized in that the conspicuousness detection neural network and the generation
Data type is 16 fixed points in the convolutional neural networks for including in confrontation network；The conspicuousness detection neural network and institute
It states the sparse matrix generated in confrontation network to store in the following way: be stored in the sparse matrix according to preset order
Nonzero value after the nonzero value is denoted as 1 is converted into binary matrix and stores.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910451850.6A CN110310343B (en) | 2019-05-28 | 2019-05-28 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910451850.6A CN110310343B (en) | 2019-05-28 | 2019-05-28 | Image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310343A true CN110310343A (en) | 2019-10-08 |
| CN110310343B CN110310343B (en) | 2023-10-03 |
Family
ID=68075231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910451850.6A Active CN110310343B (en) | 2019-05-28 | 2019-05-28 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310343B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027576A (en) * | 2019-12-26 | 2020-04-17 | 郑州轻工业大学 | Cooperative significance detection method based on cooperative significance generation type countermeasure network |
| CN111193932A (en) * | 2019-12-13 | 2020-05-22 | 西安万像电子科技有限公司 | Image processing method and device |
| CN111563517A (en) * | 2020-04-20 | 2020-08-21 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN112651940A (en) * | 2020-12-25 | 2021-04-13 | 郑州轻工业大学 | Collaborative visual saliency detection method based on dual-encoder generation type countermeasure network |
| CN113362315A (en) * | 2021-06-22 | 2021-09-07 | 中国科学技术大学 | Image quality evaluation method and evaluation model based on multi-algorithm fusion |
| CN113450313A (en) * | 2021-06-04 | 2021-09-28 | 电子科技大学 | Image significance visualization method based on regional contrast learning |
| CN114445437A (en) * | 2021-12-29 | 2022-05-06 | 福建慧政通信息科技有限公司 | Image compression clipping method of license picture and storage medium |
| WO2022127865A1 (en) * | 2020-12-18 | 2022-06-23 | 中兴通讯股份有限公司 | Video processing method, apparatus, electronic device, and storage medium |
| CN114937019A (en) * | 2022-05-30 | 2022-08-23 | 杭州健培科技有限公司 | Key point detection method and device based on self-adaptive local gray scale balance and application |
| CN115396670A (en) * | 2022-07-28 | 2022-11-25 | 西安空间无线电技术研究所 | Image data compression method for local area processing |
| CN115841486A (en) * | 2023-02-20 | 2023-03-24 | 深圳市特安电子有限公司 | Gas sensing infrared image processing method |
| CN117915096A (en) * | 2023-12-14 | 2024-04-19 | 北京大兴经济开发区开发经营有限公司 | Target identification high-precision high-resolution video coding method and system for AI large model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016409A (en) * | 2017-03-20 | 2017-08-04 | 华中科技大学 | A kind of image classification method and system based on salient region of image |
| US20180336471A1 (en) * | 2017-05-19 | 2018-11-22 | Mehdi Rezagholizadeh | Semi-supervised regression with generative adversarial networks |
| CN109300128A (en) * | 2018-09-29 | 2019-02-01 | 聚时科技(上海)有限公司 | The transfer learning image processing method of structure is implied based on convolutional Neural net |
-
2019
- 2019-05-28 CN CN201910451850.6A patent/CN110310343B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016409A (en) * | 2017-03-20 | 2017-08-04 | 华中科技大学 | A kind of image classification method and system based on salient region of image |
| US20180336471A1 (en) * | 2017-05-19 | 2018-11-22 | Mehdi Rezagholizadeh | Semi-supervised regression with generative adversarial networks |
| CN109300128A (en) * | 2018-09-29 | 2019-02-01 | 聚时科技(上海)有限公司 | The transfer learning image processing method of structure is implied based on convolutional Neural net |
Non-Patent Citations (2)
| Title |
|---|
| 徐海月等: "基于编解码网络的多姿态人脸图像正面化方法", 《中国科学:信息科学》 * |
| 徐海月等: "基于编解码网络的多姿态人脸图像正面化方法", 《中国科学:信息科学》, no. 04, 30 April 2019 (2019-04-30) * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193932A (en) * | 2019-12-13 | 2020-05-22 | 西安万像电子科技有限公司 | Image processing method and device |
| CN111027576A (en) * | 2019-12-26 | 2020-04-17 | 郑州轻工业大学 | Cooperative significance detection method based on cooperative significance generation type countermeasure network |
| CN111563517A (en) * | 2020-04-20 | 2020-08-21 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111563517B (en) * | 2020-04-20 | 2023-07-04 | 腾讯科技（深圳）有限公司 | Image processing method, device, electronic equipment and storage medium |
| WO2022127865A1 (en) * | 2020-12-18 | 2022-06-23 | 中兴通讯股份有限公司 | Video processing method, apparatus, electronic device, and storage medium |
| CN112651940A (en) * | 2020-12-25 | 2021-04-13 | 郑州轻工业大学 | Collaborative visual saliency detection method based on dual-encoder generation type countermeasure network |
| CN113450313A (en) * | 2021-06-04 | 2021-09-28 | 电子科技大学 | Image significance visualization method based on regional contrast learning |
| CN113450313B (en) * | 2021-06-04 | 2022-03-15 | 电子科技大学 | Image significance visualization method based on regional contrast learning |
| CN113362315A (en) * | 2021-06-22 | 2021-09-07 | 中国科学技术大学 | Image quality evaluation method and evaluation model based on multi-algorithm fusion |
| CN114445437A (en) * | 2021-12-29 | 2022-05-06 | 福建慧政通信息科技有限公司 | Image compression clipping method of license picture and storage medium |
| CN114937019A (en) * | 2022-05-30 | 2022-08-23 | 杭州健培科技有限公司 | Key point detection method and device based on self-adaptive local gray scale balance and application |
| CN114937019B (en) * | 2022-05-30 | 2022-12-23 | 杭州健培科技有限公司 | Key point detection method and device based on self-adaptive local gray scale balance and application |
| CN115396670A (en) * | 2022-07-28 | 2022-11-25 | 西安空间无线电技术研究所 | Image data compression method for local area processing |
| CN115396670B (en) * | 2022-07-28 | 2024-05-03 | 西安空间无线电技术研究所 | Image data compression method for local processing |
| CN115841486A (en) * | 2023-02-20 | 2023-03-24 | 深圳市特安电子有限公司 | Gas sensing infrared image processing method |
| CN115841486B (en) * | 2023-02-20 | 2023-04-25 | 深圳市特安电子有限公司 | Gas perception infrared image processing method |
| CN117915096A (en) * | 2023-12-14 | 2024-04-19 | 北京大兴经济开发区开发经营有限公司 | Target identification high-precision high-resolution video coding method and system for AI large model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310343B (en) | 2023-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310343A (en) | Image processing method and device | |
| CN110032926B (en) | Video classification method and device based on deep learning | |
| CN111598762B (en) | Generating type robust image steganography method | |
| CN110689599B (en) | 3D visual saliency prediction method based on non-local enhancement generation countermeasure network | |
| CN108846323A (en) | A kind of convolutional neural networks optimization method towards Underwater Targets Recognition | |
| Po et al. | A novel patch variance biased convolutional neural network for no-reference image quality assessment | |
| CN107680077A (en) | A kind of non-reference picture quality appraisement method based on multistage Gradient Features | |
| CN109635791B (en) | Video evidence obtaining method based on deep learning | |
| CN109657715B (en) | Semantic segmentation method, device, equipment and medium | |
| CN111832650A (en) | Image classification method based on generation of confrontation network local aggregation coding semi-supervision | |
| CN113781377A (en) | Infrared and visible light image fusion method based on antagonism semantic guidance and perception | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN108229682A (en) | A kind of image detection countercheck based on backpropagation attack | |
| He et al. | A visual residual perception optimized network for blind image quality assessment | |
| Mancas et al. | A rarity-based visual attention map-application to texture description | |
| CN115170430A (en) | Two-stage condition-based method for generating anti-network near-infrared image coloring | |
| Deng et al. | Frequency-tuned universal adversarial attacks on texture recognition | |
| Huang et al. | A method for identifying origin of digital images using a convolutional neural network | |
| CN104079934B (en) | Method for extracting regions of interest in real-time video communication | |
| CN110062125A (en) | Image transfer method and device | |
| CN117495718A (en) | Multi-scale self-adaptive remote sensing image defogging method | |
| Duan et al. | Image information hiding method based on image compression and deep neural network | |
| CN116452469A (en) | Image defogging processing method and device based on deep learning | |
| CN111080754A (en) | Character animation production method and device for connecting characteristic points of head and limbs | |
| Agrawal et al. | Exploring convolutional neural networks for automatic image colorization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |