CN104376547A - Motion blurred image restoration method - Google Patents

Abstract

The invention belongs to the technical field of digital image processing, and particularly relates to a motion blurred image restoration method. The method is used for using video sequences obtained by two or more cameras in the same scene for restoring a motion blurred image through a super-resolution reconstruction method. The method comprises the steps of camera type selection, scene sequence image acquisition, sequence image space-time registration, high-definition image sequence reconstruction and the like. Compared with the prior art, the motion blurred image restoration method has the advantages that the image sequences obtained by the multiple cameras in the same scene and sequential control of a sequential control circuit are fully utilized, so that detailed information of the obtained scene image is added; a layering search strategy is fully utilized, so that the image matching speed and precision are improved; sub pixel level complementary information between different video images and a super-resolution reconstruction algorithm are fully utilized, so that the motion blurred image is restored, and the video image resolution ratio is improved.

Description

Motion blur image restoration method

Technical field

The invention belongs to digital image processing techniques field, be specifically related to a kind of motion blur image restoration method, the video sequence that the method is used for the same scene obtained by two or more cameras restores motion blur image by the method for super-resolution reconstruction.

Background technology

Along with the fast development of CCD and cmos image sensor, digital imaging apparatus and digital processing technology are widely deployed Digital Television, netcast, video monitoring, medical diagnosis, traffic administration etc. and people and live closely bound up daily field, and also become that aerospace, remote sensing, guidance, early warning and astrosurveillance etc. are military, indispensable equipment and technology in scientific research field.In recent years. the requirement of people to video quality improves constantly, and wishes that shooting function captures more multidate information and more detailed information, namely wishes the video obtaining more high spatial resolution and temporal resolution.

But, any imaging device all has certain room and time resolution, spatial resolution depends primarily on spacial distribution density and the spatial point spread function of optoelectronic sensor, and temporal resolution depends primarily on sampling frame per second and the time shutter of senser element.Image is in imaging process, and can produce motion blur when object and camera lens exist relative motion, its essence is in imaging time, and in direction of motion, each pixel superposition causes.Because motion has instantaneity, the motion blur in reality is usually spatial variations, and simultaneously due to factor impacts such as image-generating unit sizes, often can only obtain multiframe low-resolution image, wherein the HFS of image, namely details can be lost.

Summary of the invention

(1) technical matters that will solve

The technical problem to be solved in the present invention is: how to make full use of super-resolution image reconstruction thought to provide a kind of motion blur image restoration method.

(2) technical scheme

For solving the problems of the technologies described above, the invention provides a kind of motion blur image restoration method, the method is implemented based on image restoration module, and this image restoration module comprises: sequential control circuit, data buffer storage unit, time-space relation unit, data processing unit;

The method comprises the steps:

Step S1: camera type selecting, the occasion applied according to system and cost, select camera and the camera quantity of suitable parameters; Camera is selected to consider the relative motion size that object and camera lens exist, the camera that the select time resolution parameter that relative motion is large is high, the camera that namely time shutter is short, image taking speed is fast;

Step S2: control the sequence image of each camera imaging acquisition with scene;

When receiving external pulse triggering, sequential control circuit controls each camera imaging successively, and namely each camera imaging triggers and there is small time deviation, asynchronous triggering imaging; Camera imager when receiving trigger pip will start photoelectric signal transformation, light signal is converted to electric signal and is export data buffer storage unit to after digital picture through analog to digital conversion, waits for subsequent treatment;

Step S3: use hierarchical search strategy to carry out time-space relation to the same scene different images sequence collected;

Namely space-time registration unit first carries out layer preprocessing to same scene different images sequence, search coupling from the layering of lowest resolution again, determine the position of thick coupling, then the template matches position that low resolution obtains is delivered on the slightly high-resolution image of last layer, and tentatively determines the searching position that this one deck may exist; Hierarchical search goes down so successively, until ultimate resolution layer determines matched position;

Wherein, image registration is according to similarity measurement, and its value larger expression template is more similar to Matching sub-image picture, and similarity adopted here is:

$S(x,y)=\frac{{(x-\stackrel{\‾}{x})}^T(y-\stackrel{\‾}{y})}{\left|\right|x-\stackrel{\‾}{x}\left|\right|\·\left|\right|y-\stackrel{\‾}{y}\left|\right|}---\left(1\right)$

In formula, x and y is the coordinate figure vector of certain pixel on image, with be respectively the average of vector x and y; S (x, y) represents the Similar measure between template and Matching sub-image picture;

Step S4: use the method for super-resolution rebuilding to rebuild high frame rate and high-resolution image sequence, restoring movement blurred picture according to the image sequence after registration;

After data processing unit obtains the digital picture of each collected by camera, according to the Time And Space Parameters between the video that image registration estimates, super resolution ratio reconstruction method is used to reconstruct high-definition picture, restoring movement blurred picture sequence; This process comprises:

By the k width low resolution figure g of input _kthe first width g ₀image, as with reference to image, by image registration, estimates low-resolution image g _kwith reference picture g ₀sub-pixel displacement amount, obtain the geometric transformation operator T that kth opens image _k;

First, the estimated value of high-definition picture is obtained as initial value f according to method of interpolation to reference picture ⁽⁰⁾, according to this initial value f ⁽⁰⁾obtain one group and observation image corresponding low-resolution image

$g_k^{\left(n\right)}=(T_k\left(f^{\left(n\right)}\right)*h)\↓s---\left(2\right)$

Wherein, h is the point spread function of system, and ↓ s is down-sampled operator;

Then, interpolation image in each value, utilize the f that core p back projection of back projection arrives ⁽⁰⁾corresponding part, correct original hypothesis value further, obtain a width preferably high-definition picture f thus ⁽¹⁾, repeatedly repeat this process, until meet the condition of formula (3), then obtain correct result;

${\left|\right|\mathrm{\&delta;}-h*p\left|\right|}_2<\frac{1}{\frac{1}{K}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\left|\right|T_k\left|\right|}_2}---\left(3\right)$

Wherein, δ is the unit impulse function at (0,0) place;

Wherein, repeated process is iterative process, and it is expressed as formula (4):

$f^{(n+1)}=f^{\left(n\right)}+\frac{1}{k}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{T}_k^{-1}(((g_k-g_k^{\left(n\right)})\&UpArrow;s)*p)---\left(4\right)$

Wherein, ↑ s is up-sampling operator, and k is the image quantity participating in rebuilding;

If back projection core p meets formula (3), then formula (4) is exponentially restrained.

(3) beneficial effect

Technical solution of the present invention comprises: camera type selecting; Sequence of scenes Image Acquisition; Time-space relation between sequence image; The steps such as high-definition picture sequence reconstruct.

Compared with prior art, the present invention possesses following beneficial effect:

(1) the present invention takes full advantage of polyphaser and obtains sequential control with scene image sequence and sequential control circuit, adds the detailed information of obtained scene image;

(2) the present invention takes full advantage of hierarchical search strategy, improves speed and the precision of images match;

(3) the present invention takes full advantage of sub-pixel complementary information and super-resolution rebuilding algorithm between different video image, has restored motion blur image, has improve the resolution of video image.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of technical solution of the present invention.

Fig. 2 is image registration layering schematic diagram in technical solution of the present invention.

Fig. 3 (a) and Fig. 3 (b) is restoring movement blurred picture, and wherein Fig. 3 (a) is low resolution motion blur image, and Fig. 3 (b) is the high resolution restoration motion blur image of reconstruct.

Embodiment

For making object of the present invention, content and advantage clearly, below in conjunction with drawings and Examples, the specific embodiment of the present invention is described in further detail.

Motion blur image restoration method provided by the present invention, is mainly applied in Digital Image Processing.If in imaging process, there is relative motion in object and camera lens, and so formed image will produce motion blur, and the detail of the high frequency of image will be lost, and the resolution of image also declines naturally thereupon, has a strong impact on normal process and the use of image.Image restoration generally first need estimate the point spread function (Point Spread Function, PSF) of fuzzy model, then according to the inverse process of image degradation, under PSF instructs, adopts corresponding Image Restoration Algorithm to restore.Fuzzy model PSF estimates that inaccurate meeting causes restored image to there is the phenomenon such as ringing effect, artifact, has a strong impact on the quality of restored image.Therefore, accurately draw the PSF of fuzzy model, and effectively select Image Restoration Algorithm to become the key moving into the image restoration of picture mixed fuzzy.Degradation model single video sequence accurately being estimated to image is more difficult, and the image utilizing two or more cameras to obtain carries out the frame frequency that super-resolution reconstruction not only can improve image, restore blurred picture, the resolution of image can also be improved, increase the detail of the high frequency of image.

For solving the problem of prior art, the invention provides a kind of motion blur image restoration method, the method is implemented based on image restoration module, and this image restoration module comprises: sequential control circuit, data buffer storage unit, time-space relation unit, data processing unit;

The method comprises the steps:

Step S1: camera type selecting, the occasion applied according to system and cost, select camera and the camera quantity of suitable parameters; Camera is selected to consider the relative motion size that object and camera lens exist, the camera that the select time resolution parameter that relative motion is large is high, the camera that namely time shutter is short, image taking speed is fast;

Step S2: control the sequence image of each camera imaging acquisition with scene;

When receiving external pulse triggering, sequential control circuit controls each camera imaging successively, and namely each camera imaging triggers and there is small time deviation, asynchronous triggering imaging; Camera imager when receiving trigger pip will start photoelectric signal transformation, light signal is converted to electric signal and is export data buffer storage unit to after digital picture through analog to digital conversion, waits for subsequent treatment;

Step S3: use hierarchical search strategy to carry out time-space relation to the same scene different images sequence collected;

Namely space-time registration unit first carries out layer preprocessing to same scene different images sequence, search coupling from the layering of lowest resolution again, determine the position of thick coupling, then the template matches position that low resolution obtains is delivered on the slightly high-resolution image of last layer, and tentatively determines the searching position that this one deck may exist; Hierarchical search goes down so successively, until ultimate resolution layer determines matched position;

Wherein, image registration is according to similarity measurement, and its value larger expression template is more similar to Matching sub-image picture, and similarity adopted here is:

$S(x,y)=\frac{{(x-\stackrel{\&OverBar;}{x})}^T(y-\stackrel{\&OverBar;}{y})}{\left|\right|x-\stackrel{\&OverBar;}{x}\left|\right|\&CenterDot;\left|\right|y-\stackrel{\&OverBar;}{y}\left|\right|}---\left(1\right)$

In formula, x and y is the coordinate figure vector of certain pixel on image, with be respectively the average of vector x and y; S (x, y) represents the Similar measure between template and Matching sub-image picture;

Step S4: use the method for super-resolution rebuilding to rebuild high frame rate and high-resolution image sequence, restoring movement blurred picture according to the image sequence after registration;

After data processing unit obtains the digital picture of each collected by camera, according to the Time And Space Parameters between the video that image registration estimates, super resolution ratio reconstruction method is used to reconstruct high-definition picture, restoring movement blurred picture sequence; This process comprises:

By the k width low resolution figure g of input _kthe first width g ₀image, as with reference to image, by image registration, estimates low-resolution image g _kwith reference picture g ₀sub-pixel displacement amount, obtain the geometric transformation operator T that kth opens image _k;

First, the estimated value of high-definition picture is obtained as initial value f according to method of interpolation to reference picture ⁽⁰⁾, according to this initial value f ⁽⁰⁾obtain one group and observation image corresponding low-resolution image

$g_k^{\left(n\right)}=(T_k\left(f^{\left(n\right)}\right)*h)\&DownArrow;s---\left(2\right)$

Wherein, h is the point spread function of system, and ↓ s is down-sampled operator;

Then, interpolation image in each value, utilize the f that core p back projection of back projection arrives ⁽⁰⁾corresponding part, correct original hypothesis value further, obtain a width preferably high-definition picture f thus ⁽¹⁾, repeatedly repeat this process, until meet the condition of formula (3), then obtain correct result;

${\left|\right|\mathrm{\&delta;}-h*p\left|\right|}_2<\frac{1}{\frac{1}{K}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\left|\right|T_k\left|\right|}_2}---\left(3\right)$

Wherein, δ is the unit impulse function at (0,0) place;

Wherein, repeated process is iterative process, and it is expressed as formula (4):

$f^{(n+1)}=f^{\left(n\right)}+\frac{1}{k}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{T}_k^{-1}(((g_k-g_k^{\left(n\right)})\&UpArrow;s)*p)---\left(4\right)$

Wherein, ↑ s is up-sampling operator, and k is the image quantity participating in rebuilding;

If back projection core p meets formula (3), then formula (4) is exponentially restrained.

Wherein, in described step S1, camera type selecting takes full advantage of the parameter of camera design.Any imaging device all has regular hour resolution, and temporal resolution depends primarily on sampling frame per second and the time shutter of senser element.Image exist fuzzy mainly in imaging process object and camera lens there is relative motion, within the exposure image time, in direction of motion, each pixel there occurs superposition.Therefore, application scenario and cost should be considered according to each camera of temporal resolution Selecting parameter.

Wherein, in described step S2, sequence of scenes Image Acquisition takes full advantage of circuit sequence and controls.The sequence of low resolution pictures that high-definition picture reconstruct mainly utilizes Same Scene to there is sub-pixed mapping displacement is rebuild.When object and camera lens exist relative motion, suitably controlled the imaging of each camera by sequential control circuit, realize each camera obtain image sequence between sub-pixed mapping displacement.

Wherein, in described step S3, between sequence image, time-space relation takes full advantage of hierarchical search strategy.Image registration is mainly in order to follow-up super-resolution image reconstruction, and compared with other image registration, it needs amount of images to be processed many, and operand is large.A lot of image registration algorithm has quite high precision at present, but in order to find the corresponding point on reference picture and image subject to registration, all deals with the every bit in region of search, wastes very large resource.Therefore, the reconstruction for super-resolution image just more needs an effective search strategy.Hierarchical search first image is carried out layer preprocessing, search coupling from the layering of lowest resolution again, determine the position of thick coupling, then the template matches position that low resolution obtains is delivered on the slightly high-resolution image of last layer, and tentatively determines the searching position that this one deck may exist.Hierarchical search goes down successively like this, until ultimate resolution layer determines matched position.

Wherein, in described step S4, high-definition picture sequence reconstructs the sub-pixed mapping displacement taken full advantage of between same sequence of scenes image.High-definition picture reconfiguration technique refers to utilize a width or sequence low-resolution image, obtains a panel height image in different resolution by corresponding algorithm.These sequence of low resolution pictures can be that the image imager of multiple mutual sub-pixed mapping displacement is to the imaging simultaneously of a scene; Also can be an image imager within a period of time to Same Scene repeatedly imaging, during this period of time, image imager has the displacement of sub-pixed mapping size.Compared to other method, sequence image reconstructing method overcomes the problem improving image imager manufacture craft difficulty, inherently improves image resolution ratio, also solves the large problem of pure optical means system bulk simultaneously.

Describe in detail below in conjunction with specific embodiment.

Embodiment

Process flow diagram as shown in Figure 1, first the method is camera type selecting, the occasion applied according to system and cost, selects camera and the camera quantity of suitable parameters; Again according to the sequential control circuit of application scenarios and camera parameter performance design application system, and gather the sequence image with scene; Hierarchical search strategy is used to carry out precise spatio-temporal registration to the same scene different images sequence collected; Finally the method for super-resolution rebuilding is used to rebuild high frame rate and high-resolution image sequence, restoring movement blurred picture according to the image sequence after registration.

Specifically comprise the steps:

Step one: camera type selecting

The senser element of any imaging device all has certain sampling frame per second and time shutter.And image blurring main be exactly in imaging process object and camera lens there is relative motion, namely within the exposure image time, in direction of motion, each pixel there occurs superposition.Therefore, select camera should consider the relative motion size that object and camera lens exist, what relative motion was large answers the camera that select time resolution parameter is high, and namely the time shutter is short, the camera that image taking speed is fast.But the camera price that temporal resolution is high is high, and manufacture craft is relative complex also, the camera price that sometimes temporal resolution is high is several times of low temporal resolution camera.In the present embodiment, the sampling frame per second of each camera is 25f/s.

Step 2: sequence of scenes Image Acquisition

Application system is when receiving external pulse and triggering, and sequential control circuit controls to start each camera imaging successively, and that is each camera imaging triggers and there is small time deviation, asynchronous triggering imaging.Camera imager when receiving trigger pip will start photoelectric signal transformation, light signal is converted to electric signal and by charge transfer circuit and and the process such as pre-process circuit output to data buffer storage unit, wait for subsequent treatment.

Step 3: time-space relation between sequence image

Image registration is mainly according to similarity measurement, and its value larger expression template is more similar to Matching sub-image picture.The present embodiment employing similarity is:

$S(x,y)=\frac{{(x-\stackrel{\&OverBar;}{x})}^T(y-\stackrel{\&OverBar;}{y})}{\left|\right|x-\stackrel{\&OverBar;}{x}\left|\right|\&CenterDot;\left|\right|y-\stackrel{\&OverBar;}{y}\left|\right|}---\left(1\right)$

In formula, x and y is the coordinate figure vector of certain pixel on image, with be respectively the average of vector x and y; S (x, y) represents the Similar measure between template and Matching sub-image picture.

The present embodiment have employed hierarchical search strategy in image registration, namely space-time registration unit first carries out layer preprocessing image, search coupling from the layering of lowest resolution again, determine the position of thick coupling, then the template matches position that low resolution obtains is delivered on the slightly high-resolution image of last layer, and tentatively determines the searching position that this one deck may exist.Hierarchical search goes down so successively, until ultimate resolution layer determines matched position, as shown in Figure 2.Hierarchical search strategy effectively reduces calculated amount, improves matching efficiency.

Step 4: high-definition picture sequence reconstructs

Data processing unit, after the digital picture obtaining each collected by camera, according to the Time And Space Parameters between the video that images match estimates, uses super resolution ratio reconstruction method to reconstruct high-definition picture, restoring movement blurred picture sequence.

By the k width low resolution figure g of input _kthe first width g ₀image, as with reference to image, by registration, estimates low-resolution image g _kwith reference picture g ₀sub-pixel displacement amount, obtain the geometric transformation operator T that kth opens image _k.First the estimated value of high-definition picture is obtained as initial value f by method of interpolation to reference picture ⁽⁰⁾one group and observation image is obtained with it corresponding low-resolution image

$g_k^{\left(n\right)}=(T_k\left(f^{\left(n\right)}\right)*h)\&DownArrow;s---\left(2\right)$

Wherein, h is the point spread function of system, and ↓ s is down-sampled operator.

Then interpolation image in each value, utilize the f that core back projection of back projection arrives ⁽⁰⁾corresponding part, be used for correcting original hypothesis value further.So just can obtain a width preferably high-definition picture f ⁽¹⁾.Repeatedly repeat this process, until error function meets certain condition:

$f^{(n+1)}=f^{\left(n\right)}+\frac{1}{k}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{T}_k^{-1}(((g_k-g_k^{\left(n\right)})\&UpArrow;s)*p)---\left(3\right)$

Wherein, ↑ s is up-sampling operator, and k is the image quantity participating in rebuilding.

If back projection core p meets (4) formula, then (3) formula will exponentially restrain.The complexity of this algorithm visible is lower, and therefore speed of convergence is very fast, but the condition that back projection's core will meet is harsher.

${\left|\right|\mathrm{\&delta;}-h*p\left|\right|}_2<\frac{1}{\frac{1}{K}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\left|\right|T_k\left|\right|}_2}---\left(4\right)$

Wherein, δ is the unit impulse function at (0,0) place.

When reconstructed results is not unique, the characteristic information according to residing environmental information and image/video uniquely changes process, generates final reconstruction result.This is because iterative backprojection method cannot retrain by known prior imformation, and the reconstructed results of iterative backprojection may not uniquely, therefore also need uniquely to change process according to the information of image/video to this.

In addition, the ringing in video time super-resolution rebuilding on meeting generation time, be also called " ghost phenomenon ", this can have influence on the effect of reconstruction video usually.Therefore, the constraint conditions such as regularization can also be introduced to suppress the generation of ghost, improve the reconstruction quality of image.

The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the prerequisite not departing from the technology of the present invention principle; can also make some improvement and distortion, these improve and distortion also should be considered as protection scope of the present invention.

Claims (1)

1. a motion blur image restoration method, is characterized in that, the method is implemented based on image restoration module, and this image restoration module comprises: sequential control circuit, data buffer storage unit, time-space relation unit, data processing unit;

The method comprises the steps:

Step S1: camera type selecting, the occasion applied according to system and cost, select camera and the camera quantity of suitable parameters; Camera is selected to consider the relative motion size that object and camera lens exist, the camera that the select time resolution parameter that relative motion is large is high, the camera that namely time shutter is short, image taking speed is fast;

Step S2: control the sequence image of each camera imaging acquisition with scene;

When receiving external pulse triggering, sequential control circuit controls each camera imaging successively, and namely each camera imaging triggers and there is small time deviation, asynchronous triggering imaging; Camera imager when receiving trigger pip will start photoelectric signal transformation, light signal is converted to electric signal and is export data buffer storage unit to after digital picture through analog to digital conversion, waits for subsequent treatment;

Step S3: use hierarchical search strategy to carry out time-space relation to the same scene different images sequence collected;

Namely space-time registration unit first carries out layer preprocessing to same scene different images sequence, search coupling from the layering of lowest resolution again, determine the position of thick coupling, then the template matches position that low resolution obtains is delivered on the slightly high-resolution image of last layer, and tentatively determines the searching position that this one deck may exist; Hierarchical search goes down so successively, until ultimate resolution layer determines matched position;

Wherein, image registration is according to similarity measurement, and its value larger expression template is more similar to Matching sub-image picture, and similarity adopted here is:

$S(x,y)=\frac{{(x-\stackrel{\&OverBar;}{x})}^T(y-\stackrel{\&OverBar;}{y})}{\left|\right|x-\stackrel{\&OverBar;}{x}\left|\right|\&CenterDot;\left|\right|y-\stackrel{\&OverBar;}{y}\left|\right|}---\left(1\right)$

In formula, x and y is the coordinate figure vector of certain pixel on image, with be respectively the average of vector x and y; S (x, y) represents the Similar measure between template and Matching sub-image picture;

Step S4: use the method for super-resolution rebuilding to rebuild high frame rate and high-resolution image sequence, restoring movement blurred picture according to the image sequence after registration;

After data processing unit obtains the digital picture of each collected by camera, according to the Time And Space Parameters between the video that image registration estimates, super resolution ratio reconstruction method is used to reconstruct high-definition picture, restoring movement blurred picture sequence; This process comprises:

By the k width low resolution figure g of input _kthe first width g ₀image, as with reference to image, by image registration, estimates low-resolution image g _kwith reference picture g ₀sub-pixel displacement amount, obtain the geometric transformation operator T that kth opens image _k;

First, the estimated value of high-definition picture is obtained as initial value f according to method of interpolation to reference picture ⁽⁰⁾, according to this initial value f ⁽⁰⁾obtain one group and observation image corresponding low-resolution image

$g_k^{\left(n\right)}=(T_k\left(f^{\left(n\right)}\right)*h)\&DownArrow;s---\left(2\right)$

Wherein, h is the point spread function of system, and ↓ s is down-sampled operator;

Then, interpolation image in each value, utilize the f that core p back projection of back projection arrives ⁽⁰⁾corresponding part, correct original hypothesis value further, obtain a width preferably high-definition picture f thus ⁽¹⁾, repeatedly repeat this process, until meet the condition of formula (3), then obtain correct result;

${\left|\right|\mathrm{\&delta;}-h*p\left|\right|}_2<\frac{1}{\frac{1}{K}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\left|\right|T_k\left|\right|}_2}---\left(3\right)$

Wherein, δ is the unit impulse function at (0,0) place;

Wherein, repeated process is iterative process, and it is expressed as formula (4):

$f^{(n+1)}=f^{\left(n\right)}+\frac{1}{k}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{T}_k^{-1}(((g_k-g_k^{\left(n\right)})\&UpArrow;s)*p)---\left(4\right)$

Wherein, ↑ s is up-sampling operator, and k is the image quantity participating in rebuilding;

If back projection core p meets formula (3), then formula (4) is exponentially restrained.