CN101795344B - Digital hologram compression method and system, decoding method and system, and transmission method and system - Google Patents
Digital hologram compression method and system, decoding method and system, and transmission method and system Download PDFInfo
- Publication number
- CN101795344B CN101795344B CN 201010116678 CN201010116678A CN101795344B CN 101795344 B CN101795344 B CN 101795344B CN 201010116678 CN201010116678 CN 201010116678 CN 201010116678 A CN201010116678 A CN 201010116678A CN 101795344 B CN101795344 B CN 101795344B
- Authority
- CN
- China
- Prior art keywords
- hologram
- layer
- output
- input
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
- Holo Graphy (AREA)
Abstract
The invention discloses a digital hologram compression method, a digital hologram compression system, a digital hologram decoding method, a digital hologram decoding system, a digital hologram transmission method and a digital hologram transmission system. The transmission method comprises the following steps: synthesizing a hologram from an acquired digital image by using a computer; compressing the hologram by using a ratio of neuron points in an input layer to neuron points in a hidden layer in a neural network to acquire a compressed and coded synthetic hologram; decoding the compressed and coded synthetic hologram by using the ratio of the neuron points in the hidden layer to neuron points in an output layer in the neural network to acquire a decoded hologram; and calculating field distribution on an imaging surface by using Fresnel conversion and transmitting the decoded hologram. The digital hologram is compressed by using a BP neural network, so the nonlinearity of information distribution of the digital hologram, information distribution similar random noise and compression disadvantages caused by a big dynamic range are effectively avoided and the compression efficiency of the digital hologram is effectively enhanced.
Description
Technical field
The present invention relates to the Image Compression field, relate in particular to a kind of digital hologram compression, coding/decoding method and system, transmission method and system.
Background technology
1948, physicist Denis. lid primary (Dennis Gabor 1900-1979) invented holography, i.e. wavefront reconstruction technology when the resolution of research microscope.This is a kind of two step imaging techniques, the steps include: with object of coherent source irradiation, the diffracted wave that object produces and another bundle coherent reference wave interference stack produce the spatial fringe that distributes according to certain rules, with optical film these spatial fringe distribution records are got off, then pass through a series of physical chemistry treatment step, just formed the hologram that includes object full detail (amplitude information and phase information).When using when shining this hologram with a branch of reference light as playback light, the amplitude of object and phase place will be at Space Reconstruction out.
Along with the development of digital technology and computing technique, there is the scholar to propose directly to utilize the computer manufacture hologram.Calculation holographic is compared optical holographic, and it does not need real object to exist, as long as provide in advance the concrete mathematical description of object wave, just can utilize digital computer and plotter comprehensively to go out computed hologram, and can be by this object wave of hologram reconstruction.The computer manufacture hologram has reduced in the optical holographic various errors in the restriction of experiment condition and the processing procedure the impact of hologram quality on the other hand.The calculation holographic technology is widely used in the optical information processing now, and comprehensive and optics computing generates the detection that special reference corrugated is used for optical element such as spatial filter, or realizes various optical transforms as special wavefront transformation element.
One width of cloth hologram record full detail of object (comprising amplitude information and phase information), the light amplitude that every bit records on the hologram all is that the each point diffracted wave is with the result of reference light coherent superposition on the object, so the every bit on the hologram all includes the global information of object.If any one sub-block on the intercepting hologram can be reproduced the complete picture of original objects, just the definition of reproduced image can reduce along with the area of hologram sub-block and descend.This information that has proved absolutely that hologram comprises has larger redundancy.
The object of physical record contains abundant high-frequency information, in order to guarantee that reproduced image does not produce serious aliasing, requirement wants enough high to the sample rate of object wave, simultaneously in order to guarantee the definition of reproduced image, need hologram to have with great visual angle, the data volume that such width of cloth hologram comprises is very large, has brought a lot of inconvenience for storage and the transmission of hologram.Compression processing method and technology that therefore a lot of scholar's research holographic informations are arranged.
But, because the nonlinear transformations characteristic distributions of computer-generated hologram and high dynamic range, so that traditional image coding technique is difficult to process such problem.The computer-generated hologram of labyrinth object is that a kind of information of similar random noise distributes, larger dynamic range is arranged, the probability that each image pixel intensities occurs is close to normal distribution, because the structure of interference fringe is very meticulous, local pixel excursion is larger, comprise a lot of high-frequency informations, information is distributed on the whole spatial frequency domain average, and this is so that traditional lossless coding technique efficient on the compression hologram information is not high.
Summary of the invention
The object of the present invention is to provide a kind of digital hologram compression, coding/decoding method and system, transmission method and system.Based on the present invention, can well overcome non-linear, the large drawback of bringing of dynamic range of computer digit hologram image information distribution, well improve the compression efficiency of digital hologram.
The invention provides a kind of digital hologram compression method, comprise the steps: that the digital picture that will obtain adopts computer-generated hologram; Utilize the ratio that neuron is counted in the input layer and hidden layer in the neural net, described hologram is compressed, obtain the resultant hologram of compressed encoding.
Above-mentioned digital hologram compression method, preferred described computer hologram making step comprises: the mathematic(al) representation of selecting the object wave of needs description; Calculate the fresnel diffraction field distribution of object wave on holographic facet; The hologram synthesis step is encoded into optical field distribution the transmittance function of hologram, finishes the synthetic of hologram.
Above-mentioned digital hologram compression method, between preferred described computer hologram making step and the described neural network image compression step, also comprise the neural network learning training step, described neural metwork training step comprises: the neural network parameter initialization step, parameter to described neural net is carried out initialization, and each neuronic connection weights and bias are set to random number to avoid the BP neural net to be absorbed in local minimum; Input step is with a training sample (X among the training sample set S
i, Y
i) be input to described BP neural net, X
iSend into input layer as input vector, Y
iBe sent to output layer as teacher's vector; Hidden layer output calculation procedure is utilized formula
Calculate the output h of each neuron j of hidden layer
j, realize that described training sample is through described BP neural net propagated forward; Wherein, x
iBe the input of input layer, h
jBe the output of hidden layer, w
JiBe the connection weights of input layer to hidden layer, bj is biasing; Output layer output calculation procedure is utilized formula
Calculate the output y of each neuron i of output layer
jWherein, y
iBe the output of output layer, w
Ij' be the connection weights that hidden layer arrives output layer, b
i' for setovering; Output layer error amount calculation procedure, the error value E of calculating output layer, described error E obtains E=∑ (Y by the difference of calculating actual output vector and teacher's vector
i-y
i)
2The error back propagation computing unit be used for to calculate output layer and is connected the adjustment of connection weights with hidden layer, and the computing formula of weights adjustment is: Δ w
Ij=α δ
jo
i, wherein, α is the learning rate of network, o
j=f (s
j) be according to j neuronic input value s
jThe output valve that calculates, δ
jBe the error E of output layer to the partial derivative of j neuron input, the δ value of one deck can be according to the δ value calculating of later layer before the network, formula is
δ wherein
j M-1Be m-1 layer j neuronic δ value, s
j M-1Be m-1 layer j neuronic input value, f ' is the first derivative of neuron activation functions, and the weights adjustment ginseng value of described hidden layer is calculated from the anti-pass error of output layer, until the connection weights adjustment of each node in the described hidden layer is calculated; Revise step, utilize output layer to be connected connection weights modified computing formulae with hidden layer, revise hidden layer to the connection weight w of output layer
Ij', input layer is to the connection weight w of hidden layer
Ji
Above-mentioned digital hologram compression method, in the preferred described input step, described training sample set obtains as follows: segmentation procedure with the hologram of N * N pixel, is divided into first size and is the square data block of m * m; Training sample obtains subelement, converts the square data block of each described m * m to a m
2The vector of * 1 dimension, m
2Corresponding to input node number in the described BP neural net, then the image of described N * N pixel is divided into (N/m)
2Individual training vector is as the described training sample of described BP neural net, and N is the pixel size of image to be compressed, and m requires N to be divided exactly by m for pixel being carried out the pixel size of piecemeal;
Above-mentioned digital hologram compression method, in the preferred described cutting unit, the value of m is 8.
The invention also discloses a kind of digital hologram coding/decoding method, comprise the steps: the neural network image decoding step, utilize the ratio that neuron is counted in the hidden layer and output layer in the described neural net, the resultant hologram of described compressed encoding is decoded, obtain decoded hologram; Fresnel reproduces step, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, more described decoded hologram.
The invention also discloses a kind of digital hologram transmission method, comprise outside the digital hologram compression method based on artificial synthetic neural net, also comprise: the neural network image decoding step, utilize the ratio that neuron is counted in the hidden layer and output layer in the described neural net, resultant hologram to described compressed encoding is decoded, and obtains decoded hologram; Fresnel reproduces step, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, more described decoded hologram.
On the other hand, the invention also discloses a kind of digital hologram compression system, comprising: computer hologram is made module, and the digital picture that is used for obtaining adopts computer-generated hologram; The neural network image compression module is used for utilizing the ratio that neuron is counted in neural net input layer and the hidden layer, and described hologram is compressed, and obtains the resultant hologram of compressed encoding.
Above-mentioned digital hologram compression system, preferred described computer hologram is made module and comprised: the mathematic(al) representation selected cell of object wave is used for selecting needing the mathematic(al) representation of the object wave described; Fresnel diffraction linear accelerator unit is used for calculating the fresnel diffraction field distribution of object wave on holographic facet; The hologram synthesis unit for the transmittance function that optical field distribution is encoded into hologram, is finished the synthetic of hologram.
Above-mentioned digital hologram compression system, preferred described computer hologram is made between module and the described neural network image compression module, also be connected with the neural network learning training module, specifically comprise: the neural network parameter initialization unit, be used for the parameter of described neural net is carried out initialization, each neuronic connection weights and bias are set to random number to avoid the BP neural net to be absorbed in local minimum; Input unit is used for a training sample (X with training sample set S
i, Y
i) be input to described BP neural net, X
iSend into input layer as input vector, Y
iBe sent to output layer as teacher's vector; Hidden layer output computing unit is used for utilizing formula
Calculate the output h of each neuron j of hidden layer
j, realize that described training sample is through described BP neural net propagated forward; Wherein, x
iBe the input of input layer, h
jBe the output of hidden layer, w
JiBe the connection weights of input layer to hidden layer, b
jBe biasing; Output layer output computing unit is used for utilizing formula
Calculate the output y of each neuron i of output layer
jWherein, y
iBe the output of output layer, w
Ij' be the connection weights that hidden layer arrives output layer, b
i' for setovering; Output layer error amount computing unit, for the error value E of calculating output layer, described error E obtains E=∑ (Y by the difference of calculating actual output vector and teacher's vector
i-y
i)
2The error back propagation computing unit be used for to calculate output layer and is connected the adjustment of connection weights with hidden layer, and the computing formula of weights adjustment is: Δ w
Ij=α δ
jo
i, wherein, α is the learning rate of network, o
j=f (s
j) be according to j neuronic input value s
jThe output valve that calculates, δ
jBe the error E of output layer to the partial derivative of j neuron input, the δ value of one deck can be according to the δ value calculating of later layer before the network, formula is
δ wherein
j M-1Be m-1 layer j neuronic δ value, s
j M-1Be m-1 layer j neuronic input value, f ' is the first derivative of neuron activation functions, and the weights adjustment ginseng value of described hidden layer is calculated from the anti-pass error of output layer, until the connection weights adjustment of each node in the described hidden layer is calculated; Amending unit is used for utilizing output layer to be connected connection weights modified computing formulae with hidden layer, revises hidden layer to the connection weight w of output layer
Ij', input layer is to the connection weight w of hidden layer
Ji
Above-mentioned digital hologram compression system, preferred described input unit comprises: cut apart subelement, with the image of N * N pixel, be divided into first size and be the square data block of m * m; Training sample obtains subelement, converts the square data block of each described m * m to a m
2The vector of * 1 dimension, m
2Corresponding to input node number in the described BP neural net, then the image of described N * N pixel is divided into (N/m)
2Individual training vector is as the described training sample of described BP neural net, and N is the pixel size of image to be compressed, and m requires N to be divided exactly by m for pixel being carried out the pixel size of piecemeal.
Above-mentioned digital hologram compression system, in the preferred described cutting unit, the value of m is 8.
On the other hand, the present invention also provides a kind of digital hologram decode system, comprising:
The neural network image decoder module utilizes the ratio that neuron is counted in the hidden layer and output layer in the described neural net, and the resultant hologram of described compressed encoding is decoded, and obtains decoded hologram; Fresnel reproduces module, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, more described decoded hologram.
On the other hand, the present invention also provides a kind of digital hologram image transmission system, comprise digital hologram compression system as described, also comprise: the neural network image decoder module, be used for utilizing the ratio that neuron is counted in neural net hidden layer and the output layer, resultant hologram to described compressed encoding is decoded, and obtains decoded hologram; Fresnel reproduces module, is used for the employing Fresnel and changes the field distribution that is calculated to be on the image planes, more described decoded hologram.
The present invention adopts the BP neural net that digital hologram is compressed, and effectively overcome the problems such as the nonlinear degree that digital hologram information distributes is high, dynamic range is large, effectively raises the compression efficiency of digital hologram.And, have intelligent and adaptivity in the nonlinear holographic information distribution problem of processing, when compression ratio changed, the reproduced image quality that compression obtains had preferably robustness.
Description of drawings
Fig. 1 is the flow chart of steps according to the digital hologram processing method embodiment that the present invention is based on artificial synthetic neural net;
Fig. 2 is the flow chart of steps of computer-generated hologram;
Fig. 3 is the recording beam path schematic diagram of Fresnel hologram;
Fig. 4 a is three layers of BP neural network structure schematic diagram;
Fig. 4 b is neuronic mathematical description schematic diagram;
Fig. 5 is BP neural network learning and training step flow chart;
Fig. 6 is the schematic diagram of cutting apart of train samples collection;
Fig. 7 a is Lena source images schematic diagram;
The Fresnel computer-generated hologram of Fig. 7 b for obtaining according to the calculation holographic principle;
Fig. 7 c-Fig. 7 j is respectively at 64:32 (R=50%), 64:16 (R=25%), 64:4 (6.25%), in the situation of 64:1 (R=1.5625%), the reproduction figure of the hologram of the decompression after the process neural net compression that obtains and the hologram of decompression;
Fig. 8 a is the PSNR curve synoptic diagram of BP neural net compression algorithm;
Fig. 8 b is the MSE curve synoptic diagram of BP neural net compression algorithm;
Fig. 9 is DCT, DWT, the PSNR curve comparison schematic diagram of three kinds of compression algorithms of BP neural net;
Figure 10 is DCT, DWT, the MSE curve comparison schematic diagram of three kinds of compression algorithms of BP neural net;
Figure 11 a is the structural representation based on the digital hologram treatment system embodiment of artificial synthetic neural net;
Figure 11 b is the structural representation based on the digital hologram treatment system embodiment of artificial synthetic neural net from the angle extraction of transmitting terminal and receiving terminal.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
Nonlinear characteristic for the distribution of digital hologram information, and the information distributional class is like random noise and the characteristics that have than great dynamic range, aspect the processing nonlinear problem adaptivity and intelligent advantage are being arranged in conjunction with the BP neural net, that has proposed a kind of Information Compression processing method of the computer-generated hologram that conversion is reproduced based on BP neural net and Fresnel the present invention: with computer one width of cloth digital picture is made into computer-generated hologram, then according to the structure of neural net hologram is decomposed and convert training sample to, after reaching convergence by the sample training neural net, neural net with this convergence is carried out compressed encoding and decoding to hologram, adopts at last Fresnel conversion to reproduce decoded hologram.
With reference to Fig. 1, Fig. 1 is the flow chart of steps according to the digital hologram processing method embodiment that the present invention is based on artificial synthetic neural net, comprising: computer hologram making step 110, adopt computer-generated hologram with the digital picture of obtaining; Neural network image compression step 120 utilizes the ratio that neuron is counted in the input layer and hidden layer in the neural net, and described hologram is compressed, and obtains the resultant hologram of compressed encoding; Neural network image decoding step 130 is utilized the ratio that neuron is counted in the hidden layer and output layer in the neural net, and the resultant hologram of described compressed encoding is decoded, and obtains decoded hologram; Fresnel reproduces step 140, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, more described decoded hologram.
Wherein, with reference to Fig. 2, computer hologram making step 110 further comprises following three steps: the mathematic(al) representation of object wave is selected step 210, and selection needs the mathematic(al) representation of the object wave of description; Fresnel diffraction linear accelerator step 220 is calculated the fresnel diffraction field distribution of object wave on holographic facet; Hologram synthesis step 230 is encoded into optical field distribution the transmittance function of hologram, finishes the synthetic of hologram.Above-mentioned steps is namely made the computer-generated hologram of digital picture in conjunction with the calculation holographic principle.According to principle of holography, object wave can be expressed as
Parallel reference light can be expressed as Rexp[j2 π (α x+ β y)], the transmittance function h (x, y) that resulting distribution of light intensity distribution is hologram on the holographic recording face so can be expressed as:
In computer-generated hologram is made, first object wave function and reference wave function are carried out discretization sampling and quantification, calculate the computer-generated hologram that just can obtain object according to (1) formula.Because such hologram function is the nonnegative function of a reality, can be easily with the computer processing of encoding.With reference to Fig. 3 a, wherein, 3a1 is point-source of light, and 3a2 is object, and 3a3 is hologram plane.
Behind the hologram that step among Fig. 2 is obtained, after the compression of process neural net, the decompression, carry out the reproduction of hologram image according to Fresnel transform, when being execution in step 140, realize according to following principle: when Fresnel hologram reproduces, with the playback light irradiation hologram identical with reference light, the transmitted field amplitude on the holographic facet is so:
T(x,y)=R(x,y)·h(x,y)=R(x,y)·[|R(x,y)+O(x,y)|
2]
=R·(R·R*+O·O*+R*·O+R·O*) (2)
=R|R|
2+R|O|
2+RR*O+RRO*
(2) in the formula, first and second is noise item, field component that is proportional to original object wave of the 3rd expression, to form in reproduction light source one side of hologram a virtual image of object, the 4th is a field component that is proportional to original object wave conjugation, to form in observer's one side of hologram a real image of object, shown in Fig. 3 b.Wherein, light source is reproduced in the 3b1 representative, and 3b2 represents the virtual image, and 3b3 represents hologram, and 3b4 represents real image.
The below is described in detail neural network image compression step 120, neural network image decoding step 130.
At first, neural net is introduced.Neural net is a kind of large-scale parallel distributed processors by the simple process cell formation.Neural net has learning ability, can produce rational output to different training sample data collection.This information processing capability is so that neural net can solve the challenge that some current techniques can't be processed.Computational process in the neural net is exactly the information transmission processing process in the neural net.From another point of view, the hologram that comprises object 3D information can be regarded as the set of a series of optics pixels, and these pixels are comprising amplitude information and phase information simultaneously.These optics pixels are followed identical optical propagation law when spatial, so very high self-similarity is arranged between the pixel of hologram.This specific character is very similar with the neuronic contact in the neural net, so neural net can be used for processing the nonlinear transformations distribution problem of hologram.
Neural net is a kind of intellectuality and the higher algorithm of fitness, and one self is nonlinear by the interconnected neural net that forms of non-linear neuron, and this specific character is more effective for processing nonlinear image information.The expressed relation of neural net is a kind of mapping relations that are input to output simultaneously, from a training set, choose at random a sample to network, network is just adjusted its connection weights, makes by input signal and approaches Expected Response to reach network convergence by the real response that network calculations produces with least-mean-square-error criterion.When processing hologram information with neural net method, because the self-similarity between each pixel of hologram, just in time corresponding to the annexation between each neuron in the neural net, so neural net method is used to process nonlinear holographic information and distributes more suitable.
In actual applications, the neuron number that neural net comprises and the training sample of neural net and training time are subject to certain restrictions, therefore neural net is a kind of algorithm that diminishes when image is processed in compression, and the object 3D information that comprises of computer-generated hologram can be lost in the neural net compression process in generation like this.Along with reducing of compression ratio, losing of information can be along with increase, and this reproduced image quality on hologram can produce certain impact.
Because the information redundancy of hologram itself uses different compression methods, the degree of losing of the amount of information of hologram is different when compression ratio descends.Neural net itself has the characteristics of adaptability and fault-tolerance, and it can adjust the weights of self with the variation of adaptation training sample set, is unlikely to badly influence the convergence capabilities of whole network when some or several neuronic abnormal behaviours.So neural net has certain robustness when processing hologram information.
With reference to Fig. 4 a, Fig. 4 b.Wherein, in Fig. 4 a, X represents input layer, and H represents hidden layer, the Y output layer.Typical three layers of BP (the backward transmission of Back Propagation) neural network structure is with reference to Fig. 4 a, Fig. 4 b.In Fig. 4 a, X represents input layer, and H represents hidden layer, the Y output layer.Comprising the input layer as Data Input Interface, hidden layer and the output layer of exporting as data.Basic processing unit in the neural net is neuron, and the relation of neuronic input and output can be expressed as:
Neuron is accepted a series of input x
iAnd weight w
iWeighted sum, θ is biasing, f is neuronic activation primitive, y is neuronic response output.Therefore in neural net, the neuronic output of each of hidden layer and output layer can be expressed as:
In formula (4), f (x) is the activation primitive of every one deck, w
JiThe K * N dimension connection weight value matrix from the input layer to the hidden layer, w
Ij' be the N * K dimension connection weight value matrix from the hidden layer to the output layer, b
jAnd b
i' be respectively each neuronic biasing.
In the reality, in described neural network image compression step 120 and the described neural network image decoding 130, described neural net is the BP neural net of convergence, obtains by following training and learning procedure:
Step 3, training sample namely utilize formula through BP neural net propagated forward
Calculate the output h of each neuron j of hidden layer
j
Step 4, utilize formula
Calculate the output y of each neuron i of output layer
j
The error amount of step 5, calculating output layer, this error obtains E=‖ Y-y ‖=∑ (Y by the difference of calculating actual output vector and teacher's vector
i-y
i)
2
The error amount of step 6, calculating hidden layer
This error is calculated from the anti-pass error of output layer, until the error of each node in the hidden layer is calculated.
Step 7, utilize output layer to be connected the connection weights of each layer of connection weights modified computing formulae correction with hidden layer.
Step 8 is returned step 2, is next one input learning sample repeating step 2 to 7.
Said process can be understood in conjunction with Fig. 5.
Image transmitting needed image is done preliminary treatment before the BP neural net is carried out encoding and decoding.According to the input neuron number of BP neural net, segment the image into little sub-block as the training sample set of neural net.Be the image of N * N pixel for size, be divided into first size and be the square data block of m * m.Then the square data block of each m * m converts a m to
2The vector of * 1 dimension, m
2Corresponding to the input node number of BP network, this sampled images just is divided into (N/m)
2Individual training vector is as the learning sample set of network, as shown in Figure 6.
Typical three layers of BP neural net that are used for image compression, the structure of general input layer and output layer is symmetrical, identical neuron node number is arranged, the input layer that the vector that the training sample that obtains after the image segmentation preliminary treatment is concentrated is fed to the BP network also is fed to output layer as teacher signal simultaneously as input signal.After reaching convergence by BP algorithm network training, preserve all weights and the codec of other network parameters as the compression hologram information of this moment.
The BP network realizes that the principle of data compression is to realize by the neuron node number of adjusting hidden layer and input layer.If the nodal point number of hidden layer is less than the nodal point number of input layer in the network, will reduce when image data transmission is to hidden layer so, if the view data that hidden layer is preserved adds that the data total amount of neural network weight and other network parameter is less than from the image data amount of input layer input and just realized data compression.Can be according to the neural network weight that receives and the original BP neural net of network parameter reconstruct at receiving terminal, view data is transferred to the recovery that output layer is realized view data from the hidden layer of the neural net of this reconstruct.
In BP neural net resultant hologram compression processing system, at first preliminary treatment obtains a training sample set to resultant hologram through image, when cutting apart hologram, the value of sub-block size parameter m should be chosen different values according to practical application, and what the m value was got is excessive or too small all improper, if the m value is excessive, then sample size is less, and the input number of network nodes of neural net increases, the complicated network structure, if the m value is too small, then sample size is more, makes net training time lengthen the constringency performance variation, by experiment, find that the m value is 8 proper, whole system has lower complexity, and shorter training convergence time.
After the convergence of BP neural net reached and stablizes, hologram data to be compressed was first by image preliminary treatment, the then compressed encoding that is transmitted hologram information from the input layer to the hidden layer.The ratio of the neuron nodal point number by adjusting hidden layer and input layer can obtain different data compression rates.The hidden layer nodal point number is fewer, and compression ratio is lower, otherwise the hidden layer nodal point number is more, and compression ratio is higher.The decoding that is transmitted hologram information from the hidden layer to the output layer, decoded image are through post processing of image, and namely the pretreated inverse process of image reduces and obtains complete hologram.
Because hologram comprises the 3D information of object, the information that this and general pattern comprise has than big difference.In the quality evaluation for the processing of hologram image, do not have unified standard.We can be by human eye vision observation and comparison original hologram and the hologram after processing, but such subjective assessment can not characterize the hologram picture quality after processed objectively.Therefore, this paper adopts traditional quality of image processing evaluation criterion, namely uses compression ratio (R), mean square error (MSE), and Y-PSNR (PSNR) is estimated compression efficiency and distortion measurement.R, MSE, PSNR is defined as:
In formula (5), S
cThe computer-generated hologram size of data after the compression, S
oIt is original computer-generated hologram size of data.Q is the neuron number of hidden layer, and P is the neuron number of input layer.In formula (6), M * N is the pixel size (such as M * N=256 * 256) of image, and f (x, y) is the reproduced image function of original hologram,
It is the reproduced image function of the hologram after overcompression is processed.In formula (7), x
pIt is the peak-to-peak value of view data.
In the various compression processing method experiments of computer-generated hologram, the present invention adopts the hologram reconstruction picture quality after above three evaluation functions are estimated processing.
Fig. 7 (a) is Lena source images (256 * 256 pixel sizes, 8bpp/256,65kb), the Fresnel computer-generated hologram of Fig. 7 (b) for obtaining according to the calculation holographic principle.Fig. 7 (c)-Fig. 7 (j) is respectively at 64:32 (R=50%), 64:16 (R=25%), 64:4 (6.25%), in the situation of 64:1 (R=1.5625%), the reproduction figure of the hologram of the decompression after the process neural net compression that obtains and the hologram of decompression.
The MSE of experimental result and the reproduced image under various compression ratios and PSNR experimental data are respectively shown in table 1 and Fig. 8 a, Fig. 8 b.
R, PSNR and the MSE experimental data of table 1BP neural net compression algorithm
Analyze from table 1, can find that the PSNR of reproduced image and compression ratio are proportional.Because the BP neural net is not a model that accuracy is very high, in the compressed encoding of neural net, the neuron number of hidden layer is fewer, and the compression ratio that obtains is lower, and the ability to express of network is just poorer, so information in the transmittance process from the input layer to the hidden layer, lose just more.The degree direct influence of information loss reproduced image quality and PSNR value.Fig. 8 a, 8b are PSNR curve and the MSE curve of BP neural net compression algorithm.Abscissa is compression ratio.
Image function is very complicated, and is difficult to accurately describe with mathematical expression, and neural net can be approached any mathematical function approx, and the approximation capability of neural net is directly related to its network configuration and the neuronic number that comprises.In general, the training sample set of larger amt and multilayer neural network structure can be than the training sample set of limited quantity and simple three-layer neural network structure token image functions better.In the experiment of this paper, the BP neural net that is used for doing compression is a simple three-layer neural network structure, comprises 64 input neurons and 64 output neurons, and training sample set is a width of cloth computer-generated hologram picture.These have all limited the ability of neural net, so neural net is accurate not to the expression that nonlinear holographic information distributes under higher compression ratios.But from Fig. 7 (i), Fig. 7 (j) in the situation that compression ratio reaches 1: 64 (R=1.5625%), although the hologram blocking artifact after decompressing clearly, still can access identifiable reproduced image.
Below, advantage of the present invention is described in further detail.
For better relatively discrete cosine (DCT) compression, discrete wavelet (DWT) compression and BP neural net are compressed, and the PSNR of three kinds of compression methods-R curve and MSE-R curve can be drawn in the same coordinate system, and be such as Fig. 9, shown in Figure 10.
From Fig. 9, Figure 10 analysis can find, in the PSNR of three kinds of compression methods curve and MSE curve, the PSNR curve of neural net compression is more steady than other two kinds of methods.Along with the decline of compression ratio R, the PSNR value of DCT compression and DWT compression descends very fast, and the PSNR of BP neural net compression then descends slowly relatively.This explanation DCT compression and DWT compression are higher than the compression of BP neural net to the influence degree of information loss and reproduced image quality.This that is to say, compresses with respect to DCT compression and DWT, and neural net has intelligent and adaptivity in the nonlinear holographic information distribution problem of processing, and different compression ratios is had robustness.In addition, under very low compression ratio, neural net compression ratio DCT compression and DWT are compressed with better reproduced image quality and less reconstruction error.Reach 4: 64 at compression ratio when (R=6.25%), the PSNR value that the PSNR value of the reproduced image that the neural net compression obtains and DCT compression obtain is approaching, and when compression ratio further descends, neural net compression ratio DCT compresses more effective.Reach 1: 64 in compression ratio when (R=1.5625%), the neural net compression also than the DWT compression effectively.As seen, in the situation that require very little compressible, removing to process hologram with neural net is a kind of more effective method.
On the other hand, the present invention also provides a kind of digital hologram treatment system based on artificial synthetic neural net, and with reference to Figure 11 a, this system comprises:
Computer hologram is made module 1101, and the digital picture that is used for obtaining adopts computer-generated hologram; Neural network image compression module 1102 is used for utilizing the ratio that neuron is counted in neural net input layer and the hidden layer, and described hologram is compressed, and obtains the resultant hologram of compressed encoding; Neural network image decoder module 1103 is used for utilizing the ratio that neuron is counted in neural net hidden layer and the output layer, and the resultant hologram of described compressed encoding is decoded, and obtains decoded hologram; Fresnel reproduces module 1104, is used for the employing Fresnel and changes the field distribution that is calculated to be on the image planes, more described decoded hologram.
Identical in the operation principle of above-mentioned modules and the embodiment of the method, do not repeat them here, relevant part is mutually with reference to getting final product.
With reference to Figure 11 b, Figure 11 b is the structural representation based on the digital hologram treatment system embodiment of artificial synthetic neural net from the angle extraction of transmitting terminal (image compression system) and receiving terminal (image decoding system).Wherein, 11a represents transmitting terminal, and 11b represents receiving terminal.Be that transmitting terminal 11a comprises computer hologram making module 1101, neural network image compression module 1102; Receiving terminal 11b comprises that neural network image decoder module 1103, Fresnel reproduce module 1104.
Neural network image compression module 1102 among the transmitting terminal 11a utilizes the ratio that neuron is counted in the input layer and hidden layer in the neural net, described hologram is compressed, after obtaining the resultant hologram of compressed encoding, synthetic hologram is sent to receiving terminal 11b; Simultaneously, also need the relevant parameter of hidden layer, output layer in the neural net also is sent to receiving terminal 11b, like this, neural network image decoder module 1103 could be decoded by the parameter information that the receives hologram after to compressed encoding.
Identical in the operation principle of above-mentioned modules and the embodiment of the method, do not repeat them here, relevant part is mutually with reference to getting final product.
In sum, the present invention is in order to construct a kind of general holographic information compressibility, has intelligent and adaptivity processing nonlinear holographic information distribution problem, and when namely compression ratio changed, the reproduced image quality that compression obtains had preferably robustness.Main advantage is:
The first, utilize the learning ability of neural net, by the sample training collection of hologram, make neural net have the ability of expressing the hologram nonlinear Distribution.
The second, utilize the intelligent of neural net and self adaptation characteristics, go to process complicated hologram information distribution problem, large such as dynamic range, nonlinear degree is high.
Three, utilize that relevance between the hologram pixel and the neuronic relevance in the neural network structure are very similar to be processed nonlinear holographic information and distribute.
Above a kind of digital hologram compression provided by the present invention, coding/decoding method and system, transmission method and system are introduced in detail, used specific embodiment herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications.In sum, this description should not be construed as limitation of the present invention.
Claims (6)
1. a digital hologram compression method is characterized in that, comprises the steps:
The computer hologram making step adopts computer that the digital picture of obtaining is made into computer-generated hologram;
The neural network image compression step utilizes the ratio that neuron is counted in the input layer and hidden layer in the BP neural net, and described computer-generated hologram is compressed, and obtains the computer-generated hologram of compressed encoding; Wherein,
Between described computer hologram making step and the described neural network image compression step, also comprise the neural network learning training step, described neural metwork training step comprises:
The neural network parameter initialization step carries out initialization to the parameter of described BP neural net, and each neuronic connection weights and bias are set to random number to avoid the BP neural net to be absorbed in local minimum;
Input step is with a training sample (X among the training sample set S
i, Y
i) be input to described BP neural net, X
iSend into input layer as input vector, Y
iBe sent to output layer as teacher's vector;
Hidden layer output calculation procedure is utilized formula
Calculate the output h of each neuron j of hidden layer
j, realize that described training sample is through described BP neural net propagated forward; Wherein, x
iBe the input of input layer, h
jBe the output of hidden layer, w
JiBe the connection weights of input layer to hidden layer, b
jBe biasing;
Output layer output calculation procedure is utilized formula
Calculate the output y of each neuron i of output layer
jWherein, y
iBe the output of output layer, w '
IjBe the connection weights of hidden layer to output layer, b
i' for setovering;
Output layer error amount calculation procedure, the error value E of calculating output layer, described error E obtains E=∑ (Y by the difference of calculating actual output vector and teacher's vector
i-y
i)
2
The error back propagation calculation procedure be used for to be calculated output layer and is connected the adjustment of connection weights with hidden layer, and the computing formula of weights adjustment is: Δ w
Ij=α δ
jo
i, wherein, α is the learning rate of network, o
j=f (s
j) be according to j neuronic input value s
jThe output valve that calculates, δ
jBe the error E of output layer to the partial derivative of j neuron input, the δ value of one deck can be according to the δ value calculating of later layer before the network, formula is
Wherein
Be m-1 layer j neuronic δ value,
Be m-1 layer j neuronic input value, f ' is the first derivative of neuron activation functions, and the weights adjustment ginseng value of described hidden layer is calculated from the anti-pass error of output layer, until the connection weights adjustment of each node in the described hidden layer is calculated;
Revise step, utilize output layer to be connected connection weights modified computing formulae with hidden layer, revise hidden layer to the connection weight w of output layer '
Ij, input layer is to the connection weight w of hidden layer
Ji
Described computer hologram making step comprises:
The mathematic(al) representation of object wave is selected step, and selection needs the mathematic(al) representation of the object wave of description;
Fresnel diffraction linear accelerator step is calculated the fresnel diffraction field distribution of object wave on holographic facet;
The hologram synthesis step is encoded into optical field distribution the transmittance function of hologram, finishes the synthetic of computer-generated hologram; In the described input step, described training sample set obtains as follows:
Segmentation procedure with the computer-generated hologram of N * N pixel, is divided into first size and is the square data block of m * m;
Training sample obtains substep, converts the square data block of each described m * m to a m
2The vector of * 1 dimension, m
2Corresponding to input node number in the described BP neural net, then the image of described N * N pixel is divided into (N/m)
2Individual training vector is as the described training sample of described BP neural net, and N is the pixel size of image to be compressed, and m requires N to be divided exactly by m for pixel being carried out the pixel size of piecemeal;
In the described segmentation procedure, the value of m is 8.
2. a digital hologram coding/decoding method corresponding with digital hologram compression method claimed in claim 1 is characterized in that, comprises the steps:
The neural network image decoding step is utilized the ratio that neuron is counted in the hidden layer and output layer in the described BP neural net, and the computer-generated hologram of described compressed encoding is decoded, and obtains decoded computer-generated hologram;
Fresnel reproduces step, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, reproduces described decoded computer-generated hologram.
3. a digital hologram transmission method is characterized in that, comprises digital hologram compression method as claimed in claim 1 and digital hologram coding/decoding method claimed in claim 2.
4. a digital hologram compression system is characterized in that, comprising:
Computer hologram is made module, and the digital picture that is used for obtaining adopts computer to be made into computer-generated hologram;
The neural network image compression module is used for utilizing the ratio that neuron is counted in BP neural net input layer and the hidden layer, and described computer-generated hologram is compressed, and obtains the computer-generated hologram of compressed encoding; Wherein
Described computer hologram is made between module and the described BP neural network image compression module, also is connected with the neural network learning training module, specifically comprises:
The neural network parameter initialization unit is used for the parameter of described BP neural net is carried out initialization, and each neuronic connection weights and bias are set to random number to avoid the BP neural net to be absorbed in local minimum;
Input unit is used for a training sample (X with training sample set S
i, Y
i) be input to described BP neural net, X
iSend into input layer as input vector, Y
iBe sent to output layer as teacher's vector;
Hidden layer output computing unit is used for utilizing formula
Calculate the output h of each neuron j of hidden layer
j, realize that described training sample is through described BP neural net propagated forward; Wherein, x
iBe the input of input layer, h
jBe the output of hidden layer, w
JiBe the connection weights of input layer to hidden layer, b
jBe biasing;
Output layer output computing unit is used for utilizing formula
Calculate the output y of each neuron i of output layer
jWherein, y
iBe the output of output layer, w '
IjBe the connection weights of hidden layer to output layer, b
i' for setovering;
Output layer error amount computing unit, for the error value E of calculating output layer, described error E obtains E=∑ (Y by the difference of calculating actual output vector and teacher's vector
i-y
i)
2
The error back propagation computing unit be used for to calculate output layer and is connected the adjustment of connection weights with hidden layer, and the computing formula of weights adjustment is: Δ w
Ij=α δ
jo
i, wherein, α is the learning rate of network, o
j=f (s
j) be according to j neuronic input value s
jThe output valve that calculates, δ
jBe the error E of output layer to the partial derivative of j neuron input, the δ value of one deck can be according to the δ value calculating of later layer before the network, formula is
Wherein
Be m-1 layer j neuronic δ value,
Be m-1 layer j neuronic input value, f ' is the first derivative of neuron activation functions, and the weights adjustment ginseng value of described hidden layer is calculated from the anti-pass error of output layer, until the connection weights adjustment of each node in the described hidden layer is calculated;
Amending unit is used for utilizing output layer to be connected connection weights modified computing formulae with hidden layer, revise hidden layer to the connection weight w of output layer '
Ij, input layer is to the connection weight w of hidden layer
Ji
Described computer hologram is made module and is comprised:
The mathematic(al) representation selected cell of object wave is used for the mathematic(al) representation that selection needs the object wave of description;
Fresnel diffraction linear accelerator unit is used for calculating the fresnel diffraction field distribution of object wave on holographic facet;
The hologram synthesis unit for the transmittance function that optical field distribution is encoded into hologram, is finished the synthetic of computer-generated hologram;
Described input unit comprises:
Cut apart subelement, with the computer-generated hologram of N * N pixel, be divided into first size and be the square data block of m * m;
Training sample obtains subelement, converts the square data block of each described m * m to a m
2The vector of * 1 dimension, m
2Corresponding to input node number in the described BP neural net, then the image of described N * N pixel is divided into (N/m)
2Individual training vector is as the described training sample of described BP neural net, and N is the pixel size of image to be compressed, and m requires N to be divided exactly by m for pixel being carried out the pixel size of piecemeal;
In the described cutting unit, the value of m is 8.
5. a digital hologram decode system corresponding with digital hologram compression claimed in claim 4 system is characterized in that, comprising:
The neural network image decoder module utilizes the ratio that neuron is counted in the hidden layer and output layer in the described BP neural net, and the computer-generated hologram of described compressed encoding is decoded, and obtains decoded computer-generated hologram;
Fresnel reproduces module, and the conversion of employing Fresnel is calculated to be the field distribution on the image planes, reproduces described decoded computer-generated hologram.
6. a digital hologram image transmission system is characterized in that, comprises digital hologram compression system as claimed in claim 4 and digital hologram decode system claimed in claim 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201010116678 CN101795344B (en) | 2010-03-02 | 2010-03-02 | Digital hologram compression method and system, decoding method and system, and transmission method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201010116678 CN101795344B (en) | 2010-03-02 | 2010-03-02 | Digital hologram compression method and system, decoding method and system, and transmission method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101795344A CN101795344A (en) | 2010-08-04 |
CN101795344B true CN101795344B (en) | 2013-03-27 |
Family
ID=42587759
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 201010116678 Active CN101795344B (en) | 2010-03-02 | 2010-03-02 | Digital hologram compression method and system, decoding method and system, and transmission method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101795344B (en) |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102122513B (en) * | 2011-03-25 | 2012-06-13 | 上海大学 | Coaxial recording device for digital hologram of transparent substance based on beam-splitting property of Fresnel biprism |
CN103049083B (en) * | 2012-12-17 | 2016-08-17 | 深圳市世纪天河科技有限公司 | A kind of virtual hologram input and output control method and system |
US9712871B2 (en) * | 2014-05-01 | 2017-07-18 | Qualcomm Incorporated | Determination bitstream decoding capability in video coding |
CN107111782B (en) * | 2014-11-26 | 2021-05-14 | 柯纳睿资本有限责任公司 | Neural network structure and method thereof |
US10223635B2 (en) * | 2015-01-22 | 2019-03-05 | Qualcomm Incorporated | Model compression and fine-tuning |
CN104765263B (en) * | 2015-04-13 | 2017-09-19 | 北京大学 | Electronic holographic three-dimensional information compressed encoding transmission method |
CN106447035B (en) * | 2015-10-08 | 2019-02-26 | 上海兆芯集成电路有限公司 | Processor with variable rate execution unit |
CN116468815A (en) * | 2016-01-25 | 2023-07-21 | 渊慧科技有限公司 | Generating images using neural networks |
WO2017136083A1 (en) | 2016-02-05 | 2017-08-10 | Google Inc. | Compressing images using neural networks |
CN105976408A (en) * | 2016-04-28 | 2016-09-28 | 北京大学 | Digital holographic compression transmission method of quantum backward propagation nerve network |
CN109564636B (en) * | 2016-05-31 | 2023-05-02 | 微软技术许可有限责任公司 | Training one neural network using another neural network |
CN106254879B (en) * | 2016-08-31 | 2019-11-08 | 广东精点数据科技股份有限公司 | A kind of application encodes the Image Lossy Compression method of neural network certainly |
CN106650918B (en) * | 2016-11-25 | 2019-08-30 | 东软集团股份有限公司 | The method and apparatus for constructing system model |
WO2018107414A1 (en) * | 2016-12-15 | 2018-06-21 | 上海寒武纪信息科技有限公司 | Apparatus, equipment and method for compressing/decompressing neural network model |
CN107463932B (en) * | 2017-07-13 | 2020-07-10 | 央视国际网络无锡有限公司 | Method for extracting picture features by using binary bottleneck neural network |
CN107547773B (en) * | 2017-07-26 | 2020-01-03 | 新华三技术有限公司 | Image processing method, device and equipment |
CN108881660B (en) * | 2018-05-02 | 2021-03-02 | 北京大学 | Method for compressing and calculating hologram by adopting quantum neural network for optimizing initial weight |
CN110929837B (en) * | 2018-09-19 | 2024-05-10 | 北京搜狗科技发展有限公司 | Associated word prediction method and device |
CN111353598B (en) * | 2018-12-20 | 2024-09-24 | 中科寒武纪科技股份有限公司 | Neural network compression method, electronic equipment and computer readable medium |
CN109933676A (en) * | 2019-02-14 | 2019-06-25 | 深兰科技(上海)有限公司 | A kind of information base construction method and device |
WO2020232613A1 (en) * | 2019-05-20 | 2020-11-26 | 深圳先进技术研究院 | Video processing method and system, mobile terminal, server and storage medium |
CN114782565A (en) * | 2022-06-22 | 2022-07-22 | 武汉搜优数字科技有限公司 | Digital archive image compression, storage and recovery method based on neural network |
CN114998457B (en) * | 2022-08-01 | 2022-11-22 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Image compression method, image decompression method, related device and readable storage medium |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1097070A (en) * | 1993-06-30 | 1995-01-04 | 惠州市新世纪发展总公司 | Make the method for the synthetic rainbow holography master hologram of mold pressing with calculating holographic technique |
-
2010
- 2010-03-02 CN CN 201010116678 patent/CN101795344B/en active Active
Non-Patent Citations (5)
Title |
---|
Alision E,etc..Compression of optically encrypted digital holograms using artificial neural networks.《JOURNAL OF DISPLAY TECHNOLOGY》.2006,第2卷(第4期),401-410. * |
张丽峰等.双曲正切非线性函数和加均值量化算法对BP神经网络图像压缩处理的影响.《工程图学学报》.2006,(第1期),第110-115页. * |
胡金滨等.人工神经网络的BP算法及其应用.《信息技术》.2004,第28卷(第4期),第1-4页. * |
苏蓉等.应用菲涅尔计算全息制作二维光子晶体的研究.《江苏、山东、河南、江西、黑龙江五省光学(激光)联合学术05年会论文集》.2005,摘要. * |
马义德等.一种基于分类的改进BP神经网络图像压缩方法.《兰州大学学报》.2005,第41卷(第4期),第70-72页. * |
Also Published As
Publication number | Publication date |
---|---|
CN101795344A (en) | 2010-08-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101795344B (en) | Digital hologram compression method and system, decoding method and system, and transmission method and system | |
CN105451024B (en) | Digital hologram coding transmission method adopting compressed sensing | |
CN104765263B (en) | Electronic holographic three-dimensional information compressed encoding transmission method | |
CN108419083B (en) | Image multilevel wavelet full subband compressed sensing coding method | |
CN115063492B (en) | Method for generating countermeasure sample for resisting JPEG compression | |
CN116797461A (en) | Binocular image super-resolution reconstruction method based on multistage attention-strengthening mechanism | |
CN107942377A (en) | A kind of seismic data compression and reconstructing method | |
CN114387395A (en) | Phase-double resolution ratio network-based quick hologram generation method | |
CN114067015A (en) | Pure phase hologram generation method and system combining DNN | |
CN115100300A (en) | Computer hologram compression method based on attention quantum heuristic neural network | |
CN116824525B (en) | Image information extraction method based on traffic road image | |
CN114022356A (en) | River course flow water level remote sensing image super-resolution method and system based on wavelet domain | |
CN117037290A (en) | Human face fake detection method based on global context structure difference | |
Ren et al. | Information content compression and zero-order elimination of computer-generated hologram based on discrete cosine transform | |
CN110691229A (en) | Hologram compression method, encoder and reproduced image output system | |
CN115065817A (en) | Hologram compression method, encoder and holographic reconstruction module | |
CN115797181A (en) | Image super-resolution reconstruction method for mine fuzzy environment | |
CN113658330B (en) | Holographic encoding method based on neural network | |
Shi et al. | Multi-head global attention and spatial spectral information fusion for remote sensing image compression | |
CN109445262A (en) | The storage method and device of holographic data | |
Babu et al. | Image compression and reconstruction using improved Stockwell transform for quality enhancement. | |
Kumari et al. | Image quality estimation by entropy and redundancy calculation for various wavelet families | |
CN113538236A (en) | Image super-resolution reconstruction method based on generation countermeasure network | |
CN104934038A (en) | Spatial audio encoding-decoding method based on sparse expression | |
CN117493786B (en) | Remote sensing data reconstruction method combining countermeasure generation network and graph neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant |