CN102227127B - Automatic multimedia material defect detection and quality analysis method - Google Patents

Abstract

The invention provides an automatic multimedia material defect detection and quality analysis method. The method comprises the following steps: establishment of media material defect model set; detection of media material defect; analysis of media material quality; detection of media material defect and preservation of quality analysis result. Software which is realized based on the method in the invention can reach an effect of real-time processing in a present mainstream computer, and is used for digitlization recovery of mass simulation media material accumulated in history and pretreatment of a storage system. The method in the invention provides real-time automatic defect detection (comprising defect types, frame number, time stamp/fragment information, defect position and the like) for simulation material recovery and necessary low layer quality information (comprising video audio signal to noise ratio, video audio bandwidth, video audio mean value and the like) which media compression storage needs. Based on the information, tasks of media material recovery repairing, video audio compression and digitlization storage and the like can be carried out manually or automatically.

Description

Full-automatic multimedia material defects detection and mass analysis method

Technical field

The present invention relates to multimedia technology, relate in particular to simulation ARTBEATS DESKTOP TECHNOLOGY NTSC defects detection and recovery and digitlization memory technology.

Background technology

Motion picture technique resulted from for 19 end of the centurys, and maturation also developed so far in 20th century; Broadcast television technique occurs a little later, and greatly about resulting from the World War II later stage, and development so far.Medium such as early stage film, TV, disc all are to be stored in such as on the analog storage mediums such as cinefilm, video tape, simulation disc record.As time goes on the media materials that is stored on these simulation mediums reaches increasing of access times, aging, damaged gradually, thereby causes quality to descend.Its outstanding characteristics show as various media materials generation of defects, for example since the video tape damage or stain the complete or various partial frame contents disappearances of the frame of video that causes, the aging content frame quality that causes of video tape descend (as cut, frame roll, be off color normal) etc.

Since the eighties in 20th century, computer and digital media technology have got into the high speed development stage.Because digital storage technique possesses characteristics such as memory capacity is big, easy to use, information is difficult for losing, the storage of a lot of media materials carries out the transition to the employing digital storage media by simulation gradually.To today nearly all medium all be to be stored on the digital device.But, a large amount of early stage primary analog media materials, As time goes on, its historical value is more and more important, is faced with again simultaneously and As time goes on reaches the unfavorable of safeguard measure, causes quality to descend even the risk of irrecoverable regeneration is arranged.According to the authoritative department research report, Europe has millions of hours analog media material to face this kind crisis, and China faces such problem equally as a big country.A large amount of at present early stage films and television programs, documentary film, as part of history, precious.In time rescuing these data, utilize the storage of advanced computer and digital technology as early as possible, recover these data, is the important topic that radio, film and television industry faces now.

On the other hand, because historical accumulation, this early simulation media materials quantity and huge (in 1,000,000 hours) thereof that await rescuing preservation utilize the method for manual detection and recovery consuming time costly.Therefore, full-automatic multimedia material defects detection, recovery and a digitlization storage system are demanded urgently occurring.

Summary of the invention

The present invention seeks to retain a large amount of, precious analog media materials in history in order in time to rescue protection; Utilize the high-tech digital means automatically its quality to be carried out preanalysis; Detect wherein all kinds of video defectives and video (comprising signal to noise ratio, bandwidth, active zone, mean value etc.), audio frequency low layer quality information (comprising signal to noise ratio, bandwidth, mean value etc.) are provided; Be the necessary information of reprocessing work such as further recovery, reconstruction, compressed encoding preparation; So, a kind of full-automatic multimedia material defects detection and mass analysis method are proposed.

First technical problem that the present invention will solve is to analyze existing various standard analog media materials after longer-term storage; Owing to reasons such as environmental change, storage medium wear out, breakages; The defective of all types of video contents that cause, and set up the perfect video defect model of a cover.

Second technical problem that the present invention will solve is according to setting up good analog video defect model, proposing the effective detection method of a cover and detect in real time.

The 3rd technical problem that the present invention will solve provides a cover full-automatic video audio frequency gross information analysis algorithm.

For this reason, full-automatic multimedia material defects detection provided by the invention and mass analysis method comprise:

1st, the foundation of media materials defect model collection;

2nd, the foundation of media materials quality analysis index set;

3rd, the detection of media materials defective;

4th, media materials quality analysis;

5th, preserve the detection and the quality analysis result of media materials defective.

Wherein, the 1st the step described media materials defect model collection comprise:

11st, inefficacy frame defect model, the video frame content of this model lacks fully, is substituted by some rambling signals;

1.2nd, the capable inefficacy frame defect of I type model, the video defective of this model is a unit with " OK ", impaired capable information lacks fully, non-impaired capable information keeps fully, and the row of damage with do not have special relevancy in the ranks;

1.3rd, the capable inefficacy frame defect of II type model, the video defective of this model is a unit with " OK ", impaired capable information lacks fully, and in the affected area in the ranks information much at one, the line-to-line correlation degree is extremely strong;

1.4th, the capable inefficacy frame defect of III type model; The video defective of this model is a unit with " OK "; Impaired capable information lacks fully, and the capable internal information regularity of affected area squints to the right, forms the black triangle of a no signal thus on the affected area left side;

1.5th, the capable inefficacy frame defect of IV type model; The video defective characteristics of this model move to left for whole frame; Stay a no signal black region on whole frame the right, defective is a unit with " OK " simultaneously, impaired capable message part disappearance; And affected area is whole relatively, and frame is parallel moves to right, and impaired capable left part also has a no signal black region simultaneously;

1.6th, cut type defect model, the video defective characteristics of this model are not lost for whole frame information fully, but increase the horizontal cut shape content of fixed-site or variation on the video image;

1.7th, interframe roll type defect model, the characteristics of the video defective of this model do, shows the parts of images of adjacent two frames in the frame simultaneously, and the motion up or down of rolling in time of two interframe partitioning portions, time remaining at 0.5 second between the several seconds;

1.8th, cross-color type defect model, the notable feature of the video defective of this model be a certain be that the field color of unit is unusual with " OK ", its reason is this standard signal part phase information disappearance.

The 2nd step, described media materials quality analysis index set comprised:

2.1st, video signal frame mean value and effective coverage are detected;

2.2nd, the bandwidth of video signal;

2.3rd, vision signal signal to noise ratio;

2.4th, audio signal frame mean value;

2.5th, audio signal bandwidth;

2.6th, audio signal signal to noise ratio.

The 3rd step detection algorithm of described media materials defective is:

3.1st, inefficacy frame defect model detection algorithm utilizes inter-frame correlation information, detects failpoint set in the frame, then based on the distribution and the density of failpoint, judges whether this frame is the inefficacy frame;

3.2nd, the capable inefficacy frame defect of I type model detection algorithm is 1 dimensional signal with frame greyscale image transitions to be detected, and wherein line number is an independent variable, and every capable grey scale pixel value accumulated value is a dependent variable; On this basis of signals, investigate interframe, line-to-line correlation information, and combine motion compensation in the ranks then, the final inefficacy row of judging such defective that detects;

3.3rd, the capable inefficacy frame defect of II type model detection algorithm is 1 dimensional signal with frame greyscale image transitions to be detected, and wherein line number is an independent variable, and every capable grey scale pixel value accumulated value is a dependent variable; Then on this basis of signals, according to correlation information in the ranks in the frame,, detect defective candidate row in the frame in conjunction with frame-to-frame correlation information, from detecting the row beginning of losing efficacy, search up and down adopts the method for region growing to detect all defect zone in this frame at last;

3.4th, the capable inefficacy frame defect of III type model detection algorithm utilizes line-to-line correlation property and frame-to-frame correlation information, detects candidate row; Because this type of defective possesses displacement in the ranks, when calculating line-to-line correlation property, candidate row is searched in adjacent ± row, and the correlation maximum between calculating two row is as line-to-line correlation property; This candidate row possesses the characteristics of interior high correlation of row and interframe low correlation; According to area distribution, the degree of correlation of candidate row, confirm the existence zone of such defective then;

3.5th, the capable inefficacy frame defect of IV type model detection algorithm through detecting left side black characteristic area in the frame, goes out the defective candidate regions with image segmentation, and promptly the left side detects the set of the row of black region; Utilize the interframe histogram comparison of candidate regions then, confirm the existence zone of such defective;

3.6th, cut type defect model detection algorithm in frame, detect the linear object of width less than 4 pixels, and binaryzation is as template; On this template basis, utilize the Hough conversion then, detect the x wire shaped object, filter out the linear object of all the other directions; At last, utilize morphology open-close operator to carry out motion compensation, x wire shaped object concentration filter falls because the line that object of which movement causes in the frame stays because the line that defective produces from detecting; Utilize median filter method to filter out isolated pixel at last, stay the position that scratch defects exists;

3.7th, interframe roll type defect model detection algorithm utilizes in the frame correlation in the ranks, the part that two frames are connected before and after detecting; In continuous frame group, follow the tracks of the position of detecting this part then, till this part disappears, thereby obtain the position of frame roll and the frame fragment of rolling thereof;

3.8th, cross-color type defect model detection algorithm is 1 dimensional signal with 2 dimension image transitions, and wherein line number is an independent variable; Every capable grey scale pixel value accumulated value is a dependent variable, then on this basis, is unit with " OK "; According to frame-to-frame correlation, detect the zone of color exception.

The 4th step, said media materials mass analysis method was:

4.1st, video signal frame mean value and effective coverage detection method; Frame interior pixel gray value histogram; With its 5% pixel value is decision threshold, investigate laterally and the longitudinal boundary district, if so this row or row pixel value all less than this threshold value; Then be judged as the marginal zone of no signal, remaining area is an active zone; Calculating pixel average gray in active zone is vision signal mean value;

4.2nd, bandwidth of video signal evaluation method at first utilizes FFT by row rated output spectrum, then each row power spectrum in the frame is asked average; Obtain the bandwidth of this two field picture according to the frame inner average power spectrum: the low frequency of bandwidth is made as 0, and high frequency points is defined as the lower limit that makes average power spectra reach 95% frequency;

4.3rd, vision signal signal-to-noise ration estimation method at first utilizes the Daubechies small echo to carry out multiple dimensioned decomposition picture frame; Adopt two kinds of approach to carry out Noise Estimation according to the wavelet coefficient after decomposing is comprehensive: a kind of method that adopts the noise singular point, promptly investigate the correlation of the different scale coefficient of wavelet decomposition of same pixel point and confirm singular point; Another kind utilizes the method for threshold decision, and the noise variance of promptly estimating according to previous frame is as threshold parameter, investigates the coefficient after the wavelet decomposition; In conjunction with the result of two kinds of Noise Estimation, adopt its combined information, estimated noise energy calculates the signal to noise ratio of vision signal at last;

4.4th, audio signal frame mean value calculation method, audio signal mean value are defined as the mean value of amplitude of the audio fragment of corresponding certain frame of video;

4.5th, audio signal bandwidth evaluation method, audio signal bandwidth calculate based on the FFT conversion, the frequency between the power spectrum 5%～95% after the audio fragment FFT conversion are defined as the bandwidth of this audio fragment;

4.6th, audio signal signal-to-noise ration estimation method supposes that at first (HFS) is noise bandwidth beyond the audio bandwidth, utilizes audio bandwidth information then, and the audio power spectrum calculates noise power spectrum, thereby calculates the audio frequency signal to noise ratio.Advantage of the present invention and good effect:

The present invention adopts advanced computer video analytical technology, in real time, automatically detects all kinds of video defectives, the video quality analytical information is provided simultaneously, for rear end video recovery, compression, storage are submitted necessary information.Because the historical analog media material that accumulates out magnanimity, it is low that the use manual type is carried out detection efficiency, wastes time and energy, and adopts solution of the present invention, can practice thrift great amount of cost and social resources.

Description of drawings

Fig. 1 illustrates system architecture diagram of the present invention;

Fig. 2 illustrates Media Analysis subsystem architecture figure of the present invention

Fig. 3 illustrates the present invention about video defect detecting system framework

Fig. 4 illustrates the present invention about inefficacy frame defect model, testing result and detection method

Fig. 5 illustrates model, testing result and the detection method of the present invention about row defective I type

Fig. 6 illustrates model, testing result and the detection method of the present invention about row defective II type

Fig. 7 illustrates model, testing result and the detection method of the present invention about row defective III type

Fig. 8 illustrates model, testing result and the detection method of the present invention about row defective IV type

Fig. 9 illustrates the present invention about cut type defect model, testing result and detection method

Figure 10 illustrates the present invention about interframe roll type defect model, testing result and detection method

Figure 11 illustrates the present invention about cross-color type defect model, testing result and detection method

Figure 12 illustrates the present invention about the video signal-to-noise ratio computational methods

Figure 13 illustrates the present invention about the audio frequency signal-noise ratio computation method

Embodiment

One, full-automatic multimedia material defects detection provided by the invention and mass analysis method comprise:

1st, the foundation of media materials defect model collection;

2nd, the foundation of media materials quality analysis index set;

3rd, the detection of media materials defective;

4th, media materials quality analysis;

5th, preserve the detection and the quality analysis result of media materials defective.

Implement in the software in the present invention, being converted into resolution after the analog video collection is the 352x288 video frame images, and all algorithms are directed to this resolution video and handle.

1, media materials defect model collection

Comprise:

● inefficacy frame model.The type defect model and testing result are illustrated by Fig. 4.Visible by figure, such defective full frame holds disappearance fully, is substituted by some rambling signals.

● all kinds of capable inefficacy frame models comprise,

The capable inefficacy frame of ■ I type model.The type defect model and testing result are illustrated by Fig. 5.Visible by figure, the type defective is a unit with " OK ", and impaired capable information lacks fully, and non-impaired capable information keeps fully, and does not have correlation between the damage every trade.

The capable inefficacy frame of ■ II type model.The type defect model, testing result are illustrated by Fig. 6.Visible by figure, the type defective is a unit with " OK ", and impaired capable information lacks fully, and affected area in the ranks information much at one, the line-to-line correlation degree is extremely strong.

The capable inefficacy frame of ■ III type model.The type defect model and testing result are illustrated by Fig. 7.Visible by figure, the type defective is a unit with " OK ", and impaired capable information lacks fully, and affected area row internal information regularity squints to the right, forms the black triangle of a no signal thus on the affected area left side.

■ IV type inefficacy frame model.The type defect model and testing result are illustrated by Fig. 8.Visible by figure, such defective characteristics are that whole frame moves to left; The message one no signal black region on frame the right, defective is a unit with " OK " simultaneously, impaired capable message part disappearance; And affected area is parallel to move to right, and impaired capable left part also has a no signal black region simultaneously.

● cut type defect model and detection thereof.Such defect model and testing result are illustrated by Fig. 9.Visible by figure, such defective characteristics are that whole frame information is not lost, but increases fixing or dynamic laterally cut shape content on the video image.

● interframe roll type defect model and detection thereof.Such defect model and testing result are illustrated by Figure 10.Visible by figure; The characteristics of such defective do and since certain the time base information errors or lose, cause the parts of images that shows adjacent two frames in the frame simultaneously; And (or the downwards) motion of rolling in time upwards of two interframe partitioning portions, time remaining have 0.5 second and do not wait to the several seconds.

● cross-color type defect model and detection thereof.Such defect model and testing result are illustrated by Figure 11.Such defective generally appears in the video of SCEMA standard, and its reason is the disappearance of this standard signal type phase place.Visible by figure, it is characterized by a certain is the field color unusual (being generally purple) of unit with " OK ".

2, media materials quality analysis index set comprises:

● video signal frame mean value and effective coverage are detected;

● the bandwidth of video signal;

● the vision signal signal to noise ratio;

● audio signal frame mean value;

● audio signal bandwidth;

● the audio signal signal to noise ratio.

3, the detection of media materials defective comprises:

● inefficacy frame defect detection algorithm.The type defect inspection method block diagram is illustrated by Fig. 4.Detection method is:

■ at first utilizes inter-frame correlation information, detects failpoint set in the frame.For this reason, definition failpoint template does,

$M_{i,j}^t=\left\{\begin{array}{cc}1& \mathrm{if}\min \left(\right|x_{i,j}^t-x_{i,j}^{t-1}|,|x_{i,j}^t-x_{i,j}^{t+1}\left|\right)>150\\ 0& \mathrm{else}\end{array}\right.$

Here, t is a temporal information, and (i j) is coordinate, and x is a frame interior pixel gray value

■ judges then based on the distribution and the density of failpoint whether the inefficacy frame exists.

Entire frame is divided into 22x12 sub regions (each subregion size is 12x16).Surpass 80% of this row subregion number if contain the subregion of failpoint in the same lines, it is capable to define this subregion behavior inefficacy subregion; If such inefficacy subregion line number surpasses all subregion line numbers 50%, judge then whether this frame is the inefficacy frame

● all kinds of capable inefficacy frame defect detection algorithms, comprise,

The capable inefficacy frame defect of ■ I type detects.The type defect inspection method block diagram is illustrated by Fig. 5.Detection method is:

◆ be 1 dimensional signal (every capable grey scale pixel value adds up) at first with frame greyscale image transitions to be detected

$y\left(i\right)=\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}x(i,j)$

Here, i is a row-coordinate, and j is the row coordinate, and x is a frame interior pixel gray value, and N is the row number of pixels

◆ between definition frame-the line-to-line correlation function:

$H(i,k)=\left\{\begin{array}{cc}1& \mathrm{if}\min \left(\right|y^t\left(i\right)-y^{t-1}(i+k)|,|y^t\left(i\right)-y^{t+1}(i+k)\left|\right)>\min \left(\right|y^{t-1}(i+k)-y^{t+1}(i+k)|,20)\\ 0& \mathrm{else}\end{array}\right.$

Here, t is a temporal information, and k is shift factor in the ranks, and i is a row-coordinate.

◆ at last according to interframe-line-to-line correlation function,, carry out motion compensation simultaneously, confirm the existence zone of the type defective in conjunction with the information after frame 1 dimensionization of front and back.

Frame difference degree before the definition: $D_0(i,k)=\left\{\begin{array}{cc}\frac{1}{N}\underset{j=-4}{\overset{4}{\mathrm{\Σ}}}\left(\right|y^t(i+j)-y^{t-1}(i+j+k)|*H(i+j,k))& \mathrm{If\; N}\≠0\\ 0& \mathrm{Else}\end{array}\right.$

Definition back frame difference degree: $D_1(i,k)=\left\{\begin{array}{cc}\frac{1}{N}\underset{j=-4}{\overset{4}{\mathrm{\Σ}}}\left(\right|y^t(i+j)-y^{t-1}(i+j+k)|*H(i+j,k))& \mathrm{If\; N}\≠0\\ 0& \mathrm{Else}\end{array}\right.$

Here, $N=\underset{j=-4}{\overset{4}{\mathrm{\Σ}}}H(i+j,k)$

Frame difference degree motion compensation before the definition: $Z_0\left(i\right)=\underset{k=-10}{\overset{10}{\mathrm{Min}}}\left(D_0(i,k)\right)$

The frame difference degree motion compensation of definition back: $Z_1\left(i\right)=\underset{k=-10}{\overset{10}{\mathrm{Min}}}\left(D_1(i,k)\right)$

Decision rule function: min (Z ₀(i), Z ₁(i))＞min (| y ^T+1(i)-y ^T-1(i) |+15,30), promptly satisfy the capable defective row that is of i like this formula

The capable inefficacy frame defect of ■ II type detects.The type defect inspection method block diagram is illustrated by Fig. 6.Detection method is:

◆ be 1 dimensional signal (every capable grey scale pixel value adds up) at first with frame greyscale image transitions to be detected

$y\left(i\right)=\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\Σ}}}x(i,j)$

Here, i is a row-coordinate, and j is the row coordinate, and x is a frame interior pixel gray value, and N is the row number of pixels

◆ then according to correlation information in the ranks in the frame, in conjunction with frame-to-frame correlation information, detect defective candidate row in the frame, this candidate row satisfies the characteristics of high correlation and interframe low correlation in the ranks;

In the ranks high correlation requires to satisfy simultaneously:

$T_1\left(i\right)=\underset{k=-5}{\overset{5}{\max }}\left(\right|y\left(i\right)-y(i+k+1)|<3$

$T_2\left(i\right)=\underset{j=0}{\overset{N}{\max }}\left(\frac{1}{11}\underset{k=-5}{\overset{5}{\mathrm{\&Sigma;}}}\right|x(i,j)-x(i+k,j)|<20$

Here, k is the line displacement factor

The interframe low correlation requires to satisfy:

min(d ₀，d ₁)＞T ₂*(T ₁+0.5)

Here,

${\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="110621081156.png"\; state="\{\{state\}\}">d</figure-callout>}}_0=\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}|x^t(i,j)-x^{t-1}(i,j)|$

$d_1=\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}|x^t(i,j)-x^{t-1}(i,j)|,$

Wherein t is a temporal information

◆ begin to search for up and down from candidate row at last, all and defect area are adjacent, and have the row of high correlation to bring in the defect area with candidate row

The capable inefficacy frame defect of ■ III type detects.The type defect inspection method block diagram is illustrated by Fig. 7.Detection method is:

◆ at first utilize line-to-line correlation property and frame-to-frame correlation information, detect candidate row.Because this type of defective possesses displacement in the ranks, when calculating line-to-line correlation property, parallel moves wherein delegation among a small circle, and the correlation maximum after calculating is moved between two row is as line-to-line correlation property.This candidate row possesses the characteristics of interior high correlation of row and interframe low correlation;

The displacement degree of correlation between definition line:

$k\left(i\right)=\min \left(\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\right|x(i,j)-x(i+1,j)|,\frac{1}{N-1}\underset{j=0}{\overset{N-2}{\mathrm{\&Sigma;}}}|x(i,j)-x(i+1,j+1)|$

Here, i is a row-coordinate, and j is the row coordinate, and x is a frame interior pixel gray value, and N satisfies for the capable requirement of row number of pixels candidate defect simultaneously:

min(k(i)-k(i-1)，k(i)-k(i+1))＞5

$\underset{j=1}{\overset{10}{\max }}\left(k(i-j)\right)-\underset{j=1}{\overset{10}{\min }}\left(k(i-j)\right)<3$

◆ begin to search for up and down from candidate row at last, all and defect area are adjacent, and have the row of high correlation to bring in the defect area with candidate row

■ IV type inefficacy frame defect detects.The type defect inspection method block diagram is illustrated by Fig. 8.Detection method is:

◆ at first, image segmentation is gone out defective candidate regions (being the set that the left side detects the row of black region) through detecting left side black characteristic area in the frame;

Definition black region detection threshold is:

$T=\min (\underset{i}{\max }(\frac{1}{N}\underset{k=0}{\overset{i}{\mathrm{\&Sigma;}}}H\left(k\right)<0.05),50)$

Here H (k) is a histogram, and N is a frame interior pixel number.Utilize this thresholding, obtain the black candidate region

$M(i,j)=\left\{\begin{array}{cc}1& \mathrm{if\; x}(i,j)<\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="110621081156.png"\; state="\{\{state\}\}">T</figure-callout>}\\ 0& \mathrm{else}\end{array}\right.$

◆ (i j) and line-to-line correlation property, obtains defective candidate regions (with behavior unit) to utilize following formula M then.All row of this zone require to satisfy simultaneously:

1. the low in the ranks degree of correlation

2. (M (i, j)=1) is no less than 5 to detect black picture element on the left of the row

◆ last, utilize the interframe histogram comparison of defective candidate regions, confirm the existence zone of such defective.

● cut type defects detection algorithm.Such defect inspection method block diagram is illustrated by Fig. 9.Detection method is:

■ at first detects the linear object of width less than 4 pixels in frame, and binaryzation is as template; For this reason, have to give a definition:

$\left|\right|\stackrel{\mathrm{\&rho;}}{x}-\stackrel{\mathrm{\&rho;}}{y}\left|\right|={(x_r-y_r)}^2+{(x_g-y_g)}^2+{(x_b-y_b)}^2$

$d_0(i,j)=\left|\right|\stackrel{\mathrm{\&rho;}}{x}(i,j)-\stackrel{\mathrm{\&rho;}}{x}(i-2,j)\left|\right|$

$d_1(i,j)=\left|\right|\stackrel{\mathrm{\&rho;}}{x}(i,j)-\stackrel{\mathrm{\&rho;}}{x}(i+2,j)\left|\right|$

$d_2(i,j)=\left|\right|\stackrel{\mathrm{\&rho;}}{x}(i+2,j)-\stackrel{\mathrm{\&rho;}}{x}(i-2,j)\left|\right|$

$M_1(i,j)=\left\{\begin{array}{cc}1& \mathrm{if}\min (d_0(i,j),d_1(i,j))>1.5*(d_2(i,j)+10)\\ 0& \mathrm{else}\end{array}\right.$

$M_2(i,j)=\left\{\begin{array}{cc}1& \mathrm{if}\min \left(\right|x^t(i,j)-x^{t-1}(i,j)|,|x^t(i,j)-x^{t+1}(i,j)\left|\right)>40\\ 0& \mathrm{else}\end{array}\right.$

M＝M ₁⌒M ₂

Here, i is a row-coordinate, and j is the row coordinate, x _r, x _g, x _b, being pixel R, G, B component, t is a temporal information

■ on this template basis, utilizes the Hough conversion then, detects the x wire shaped object, filters out the linear object of all the other directions;

■ utilizes morphology open-close operator to carry out motion compensation then, and x wire shaped object concentration filter falls because the line that object of which movement causes in the frame stays because the line that defective produces from detecting;

■ utilizes median filter method to filter out isolated pixel at last, stays the position that scratch defects exists.

● interframe roll type defects detection algorithm.Such defect inspection method block diagram is illustrated by Figure 10.Detection method is:

■ at first utilizes in the frame correlation in the ranks, the part that two frames are connected before and after detecting (if existence); For this reason, just like giving a definition,

$y\left(i\right)=\frac{1}{N}\underset{j=0}{\overset{N}{\mathrm{\&Sigma;}}}|x(i,j)-x(i+1,j)|$

$T=\max \left(\frac{1}{N}\underset{j=0}{\overset{N}{\mathrm{\&Sigma;}}}\right|x(1,j)-x(M-1,j)|,50)$

$y\left(k\right)=\underset{i}{\max }\left(y\left(i\right)\right),y\left(i\right)>T$

Here, i is a row-coordinate, and j is the row coordinate, and M is the number of row, and N is the number of every capable pixel, and k connects row (if y (k)＞T, this row exists, otherwise does not exist) for rolling interframe

■ follows the tracks of the position (k (t)) of detecting this part then in continuous frame group, and calculates the transformation range of its rolling, till this part disappears, thereby obtains the position of frame roll and the frame fragment of rolling thereof.

● cross-color type defects detection algorithm.Such defect inspection method block diagram is illustrated by Figure 11.Detection method is:

■ at first with 2 the dimension image transitions be 1 dimensional signal (line number is a dimension, be worth for this row pixel with)

$y\left(i\right)=\frac{1}{N}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}x(i,j)$

Here, i is a row-coordinate, and j is the row coordinate, and x is a frame interior pixel gray value, and N is the row number of pixels

■ then on this basis, according to frame-to-frame correlation, the zone (is unit with " OK ") of detecting color exception defines for this reason:

T(i)＝max(|y ^t(i)-y ^t-1(i)|，|y ^t(i)-y ^t+1(i)|)

Defective row judgment criterion is:

$\min (y_r^t\left(i\right)-y_r^{t-1}\left(i\right),y_r^t\left(i\right)-y_r^{t+1}\left(i\right))>T\left(i\right)$

$\min (y_g^{t-1}\left(i\right)-y_g^t\left(i\right),y_g^{t+1}\left(i\right)-y_g^t\left(i\right))>T\left(i\right)$

$\min (y_b^{t-1}\left(i\right)-y_b^t\left(i\right),y_b^{t+1}\left(i\right)-y_b^t\left(i\right))>T\left(i\right)$

Here, y _r, y _g, y _b, be 1 dimensional signal R, G, B component, i is a line number, T (i) is a decision threshold.

4, media materials quality analysis; Comprise:

● video signal frame mean value and effective coverage are detected.

■ is in detecting the video active zone; The higher limit (utilizing histogram distribution to obtain) of utilizing 5% minimum pixel value set in the frame is as threshold value; Investigate and laterally reach the longitudinal boundary district; If this row (row) is so pixel value all less than this threshold value, then is judged as the marginal zone of no signal, remaining area is an active zone.

Definition: $\underset{k}{\mathrm{Min}}(\underset{i=0}{\overset{k}{\mathrm{\&Sigma;}}}H\left(i\right)>\frac{1}{20}\underset{i=0}{\overset{255}{\mathrm{\&Sigma;}}}H\left(i\right))$

Here, H (i) is a histogram in the frame, and k is a decision threshold.

■ is the calculating pixel average gray in active zone, is vision signal mean value.

$\mathrm{VL}=\frac{1}{\mathrm{MN}}\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{M-1}{\mathrm{\&Sigma;}}}x(i,j)$

Here, N is a line number, and M is a columns.

● the bandwidth of video signal.When calculating video bandwidth,

■ at first utilizes FFT by row rated output spectrum, then each row power spectrum in the frame is asked average.

$P\left(f\right)=\frac{1}{N}\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}{P}_i\left(f\right)$

Here P _i(f) be the capable power spectrum of i, N is a line number

■ obtains the bandwidth of this two field picture according to the frame inner average power spectrum: the low frequency of bandwidth is made as 0, and high frequency points is defined as the lower limit that makes average power spectra reach 95% frequency

$f_{\mathrm{high}}=\underset{f}{\max }(\underset{i=0}{\overset{f}{\mathrm{\&Sigma;}}}P\left(i\right)<\frac{19}{20}\underset{i=0}{\overset{\&infin;}{\mathrm{\&Sigma;}}}P\left(i\right))$

f _low＝0

● the vision signal signal to noise ratio.The vision signal snr computation is utilized method of wavelet, and flow process is illustrated by Figure 12.

■ at first utilizes the Daubechies small echo to carry out multiple dimensioned decomposition picture frame;

$W_{2^i}f(x,y)=f*{\mathrm{\&psi;}}_{2^j}(x,y)$

■ adopts two kinds of approach to carry out Noise Estimation based on the wavelet coefficient after decomposing is comprehensive:

◆ a kind of method that adopts the noise singular point, promptly investigate the correlation of the different scale coefficient of wavelet decomposition of same pixel point and confirm singular point;

$S(x,y)=\frac{W_{2^j}(x,y)}{W_{2^{j+1}}(x,y)}$

S _s＝{(x，y)|S(x，y)＞t _s}

◆ another kind utilizes the method for threshold decision, and the noise variance of promptly estimating according to previous frame (σ) is as threshold parameter, investigates the coefficient after the wavelet decomposition.

t _c＝kσ

$S_c=\left\{(x,y)\right|W_{2^j}(x,y)<t_c\}$

◆ combine the result of two kinds of Noise Estimation, adopt its combined information, estimated noise energy calculates the signal to noise ratio of vision signal at last.

S＝S _t⌒S _c

$P_{\mathrm{noise}}=\underset{(x,y)\&Element;S}{\mathrm{\&Sigma;}}{W}_{2^j}(x,y)$

$\mathrm{SNR}=10*\left(\frac{\underset{(x,y)\&Element;I}{\mathrm{\&Sigma;}}f(x,y)-P_{\mathrm{noise}}}{P_{\mathrm{noise}}}\right)$

● audio signal frame mean value.Audio signal mean value is defined as the mean value of amplitude of the audio fragment of corresponding certain frame of video.

$\mathrm{AL}=\frac{1}{N}\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}x\left(i\right)$

● audio signal bandwidth.Audio signal bandwidth is calculated based on the FFT conversion, the frequency between the power spectrum 5%～95% after the audio fragment FFT conversion is defined as the bandwidth of this audio fragment.

$f_{\mathrm{low}}=\underset{f}{\min }(\underset{i=0}{\overset{f}{\mathrm{\&Sigma;}}}P\left(i\right)>\frac{1}{20}\underset{i=0}{\overset{\&infin;}{\mathrm{\&Sigma;}}}P\left(i\right))$

$f_{\mathrm{high}}=\underset{f}{\max }(\underset{i=0}{\overset{f}{\mathrm{\&Sigma;}}}P\left(i\right)<\frac{19}{20}\underset{i=0}{\overset{\&infin;}{\mathrm{\&Sigma;}}}P\left(i\right))$

● the audio signal signal to noise ratio.The audio frequency signal to noise ratio is utilized the method for FFT conversion, and flow process is illustrated by Figure 13.During calculating,

■ supposes that at first (HFS) is noise bandwidth beyond the audio bandwidth

$f_{\mathrm{high}}=\underset{f}{\max }(\underset{i=0}{\overset{f}{\mathrm{\&Sigma;}}}P\left(i\right)<\frac{19}{20}\underset{i=0}{\overset{\&infin;}{\mathrm{\&Sigma;}}}P\left(i\right))$

■ utilizes audio bandwidth information then, and the audio power spectrum calculates noise power spectrum, thereby calculates the audio frequency signal to noise ratio.

$P_{\mathrm{noise}}=\underset{i=f}{\overset{f_{\mathrm{sample}}/2}{\mathrm{\&Sigma;}}}p\left(i\right)$

$P_N=\frac{f_{\mathrm{sample}}/2}{f_{\mathrm{sample}}/2-f_{\mathrm{hight}}}*P_{\mathrm{noise}}$

$\mathrm{SNR}=10*\log \left(\frac{\underset{i=0}{\overset{f_{\mathrm{sample}}/2}{\mathrm{\&Sigma;}}}p\left(i\right)-P_N}{P_N}\right).$

Two, introduce exemplary real-time example with the lower part

Present embodiment be run on Linux platform, multithreading, based on the software architecture of plug-in unit.

Fig. 1 illustrates present embodiment software architecture figure.Can know by figure,

● present embodiment software mainly is made up of 4 modules:

■ main program module, this module major function be application initialization, Memory Allocation, dynamic load, following each sub-function module is set.This module can comprise startup by remote computer through socket control, suspend, stop, disposing and be provided with etc.

■ data acquisition submodule, this module are the dynamic base card format, by the main program dynamic load, operate in the independent thread.Major function is that through operation bottom data analog input card, image data is to input shared drive (" input Buffer ")

■ low layer quality information is analyzed submodule, and this module is the dynamic base card format, by the main program dynamic load, operates in the independent thread.Major function does, reads " input Buffer " and looks audio content, carries out the low layer quality analysis, and analysis result is saved in output shared drive (" output Buffer ").Through Socket, " output Buffer " content is outputted results to reception server (medium recovery system) simultaneously.The information that the low layer quality information is analyzed submodule to be provided has:

◆ video wavelet transformation domain coefficient

◆ audio frequency FFT coefficient in transform domain

◆ single frames defects detection provisioning information (being used for high layer information analyzing subsystem comprehensive analysis processing)

◆ video essential information (frame mean value, bandwidth, signal to noise ratio)

◆ audio frequency essential information (frame mean value, bandwidth, signal to noise ratio)

■ video defects detection submodule, this module is the dynamic base card format, by the main program dynamic load, operates in the independent thread.This module major function does; Look audio content based on reading " input Buffer "; Analyze submodule preanalysis partial information in conjunction with the low layer quality information, carry out video defective content detection, and testing result is saved in output shared drive (" output Buffer ").Through Socket, " output Buffer " content is outputted results to reception server (medium recovery system) simultaneously.Video defects detection submodule has:

◆ the inefficacy frame defect detects

◆ all kinds of capable inefficacy frames detect, comprise,

● the capable inefficacy frame of I type detects

● the capable inefficacy frame of II type detects.

● the capable inefficacy frame of III type detects

● the capable inefficacy frame of IV type detects.

◆ cut type defects detection.

◆ interframe roll type defects detection.

◆ cross-color type defects detection.

Fig. 2 illustrate main type of this enforcement use-case with and call and derived relation (solid line is " calling "; Dotted line is " derivation "), wherein,

◆ LibSocket.a and Libpthread.a are standard Socket and thread library under the Linux

◆ Thread is a C++ encapsulation to Libpthread

◆ Plugin is derived from by Thread, increases interfaces such as dynamic base loading, environment setting, execution, end

◆ FrameCapture is derived from by Plugin, realizes the video/audio acquisition function

◆ MediaInfo is derived from by Plugin, realizes low layer quality information analytic function

◆ MediaDefect is derived from by Plugin, realizes video defects detection function

◆ MediaAnalysis is a main program, can pass through the Socket Long-distance Control

◆ MTBuffer is responsible for data management.

Fig. 3 illustrates the system architecture of video defects detection.Can know that by figure system's frame defect that at first lost efficacy detects, detect like this frame and be the inefficacy frame, then detect and accomplish; Otherwise carry out different classes of defects detection respectively; At last, testing result is gathered and export.

Claims (3)

1. full-automatic multimedia material defects detection and mass analysis method is characterized in that this method comprises:

1st, the foundation of media materials defect model collection;

2nd, the foundation of media materials quality analysis index set;

3rd, the detection of media materials defective; The detection algorithm of described media materials defective is:

3.1st, inefficacy frame defect model detection algorithm utilizes inter-frame correlation information, detects failpoint set in the frame; According to the distribution and the density of failpoint, judge whether this frame is the inefficacy frame then;

3.2nd, the capable inefficacy frame defect of I type model detection algorithm is 1 dimensional signal with frame greyscale image transitions to be detected, and wherein line number is an independent variable, and every capable grey scale pixel value accumulated value is a dependent variable; On this basis of signals, investigate interframe, line-to-line correlation information, and combine motion compensation in the ranks then, the final inefficacy row of judging such defective that detects;

3.3rd, the capable inefficacy frame defect of II type model detection algorithm is 1 dimensional signal with frame greyscale image transitions to be detected, and wherein line number is an independent variable, and every capable grey scale pixel value accumulated value is a dependent variable; Then on this basis of signals,,, detect defective candidate row in the frame in conjunction with frame-to-frame correlation information according to correlation information in the ranks in the frame; From detecting the row beginning of losing efficacy, search up and down adopts the method for region growing to detect all defect zone in this frame at last;

3.4th, the capable inefficacy frame defect of III type model detection algorithm utilizes line-to-line correlation property and frame-to-frame correlation information, detects candidate row; Because this type of defective possesses displacement in the ranks, when calculating line-to-line correlation property, candidate row is searched in its adjacent ± 5 row intervals, and the correlation maximum between calculating two row is as line-to-line correlation property; This candidate row possesses the characteristics of interior high correlation of row and interframe low correlation; According to area distribution, the degree of correlation of candidate row, confirm the existence zone of such defective then;

3.5th, the capable inefficacy frame defect of IV type model detection algorithm through detecting left side black characteristic area in the frame, goes out the defective candidate regions with image segmentation, and promptly the left side detects the set of the row of black region; Utilize the interframe histogram comparison of candidate regions then, confirm the existence zone of such defective;

3.6th, cut type defect model detection algorithm in frame, detect the linear object of width less than 4 pixels, and binaryzation is as template; On this template basis, utilize the Hough conversion then, detect the x wire shaped object, filter out the linear object of all the other directions; At last, utilize morphology open-close operator to carry out motion compensation, x wire shaped object concentration filter falls because the line that object of which movement causes in the frame stays because the line that defective produces from detecting; Utilize median filter method to filter out isolated pixel at last, stay the position that scratch defects exists;

3.7th, interframe roll type defect model detection algorithm utilizes in the frame correlation in the ranks, the part that two frames are connected before and after detecting; In continuous frame group, follow the tracks of the position of detecting this part then, till this part disappears, thereby obtain the position of frame roll and the frame fragment of rolling thereof;

3.8th, cross-color type defect model detection algorithm is 1 dimensional signal with 2 dimension image transitions, and wherein line number is an independent variable, and every capable grey scale pixel value accumulated value is a dependent variable; Then on this basis, be unit with " OK ", according to frame-to-frame correlation, detect the zone of color exception;

4th, media materials quality analysis; Said media materials mass analysis method is:

4.1st, video signal frame mean value and effective coverage detection method; Frame interior pixel gray value histogram; With its 5% pixel value is decision threshold, investigates laterally and the longitudinal boundary district, if border row or be listed as all pixel values all less than this threshold value; Then be judged as the marginal zone of no signal, remaining area is an active zone; Calculating pixel average gray in active zone is vision signal mean value;

4.2nd, bandwidth of video signal evaluation method at first utilizes FFT by row rated output spectrum, then each row power spectrum in the frame is asked average; Obtain the bandwidth of this two field picture according to the frame inner average power spectrum: the low frequency of bandwidth is made as 0, and high frequency points is defined as the lower limit that makes average power spectra reach 95% frequency;

4.3rd, vision signal signal-to-noise ration estimation method at first utilizes the Daubechies small echo to carry out multiple dimensioned decomposition picture frame; Adopt two kinds of approach to carry out Noise Estimation according to the wavelet coefficient after decomposing is comprehensive: a kind of method that adopts the noise singular point, promptly investigate the correlation of the different scale coefficient of wavelet decomposition of same pixel point and confirm singular point; Another kind utilizes the method for threshold decision, and the noise variance of promptly estimating according to previous frame is as threshold parameter, investigates the coefficient after the wavelet decomposition; In conjunction with the result of two kinds of Noise Estimation, adopt its combined information, estimated noise energy calculates the signal to noise ratio of vision signal at last;

4.4th, audio signal frame mean value calculation method, audio signal mean value are defined as the mean value of amplitude of the audio fragment of corresponding certain frame of video;

4.5th, audio signal bandwidth evaluation method, audio signal bandwidth calculate based on the FFT conversion, the frequency between the power spectrum 5%～95% after the audio fragment FFT conversion are defined as the bandwidth of this audio fragment;

4.6th, audio signal signal-to-noise ration estimation method supposes that at first the HFS beyond the audio bandwidth is a noise bandwidth, utilizes audio bandwidth information then, and the audio power spectrum calculates noise power spectrum, thereby calculates the audio frequency signal to noise ratio;

5th, preserve the detection and the quality analysis result of media materials defective.

2. method according to claim 1 is characterized in that described media materials defect model collection of the 1st step comprises:

1.1st, inefficacy frame defect model, the video frame content of this model lacks fully, is substituted by some rambling signals;

1.2nd, the capable inefficacy frame defect of I type model, the video defective of this model is a unit with " OK ", impaired capable information lacks fully, non-impaired capable information keeps fully, and the row of damage with do not have special relevancy in the ranks;

1.3rd, the capable inefficacy frame defect of II type model, the video defective of this model is a unit with " OK ", impaired capable information lacks fully, and in the ranks information correlation is extremely strong in the affected area;

1.4th, the capable inefficacy frame defect of III type model, the video defective of this model is a unit with " OK ", impaired capable information lacks fully, and the capable internal information regularity of affected area squints to the right, forms the black triangle of a no signal thus on the affected area left side;

1.5th, the capable inefficacy frame defect of IV type model; The video defective characteristics of this model move to left for whole frame; Stay a no signal black region on whole frame the right, defective is a unit with " OK " simultaneously, impaired capable message part disappearance; And affected area is with respect to regional parallel the moving to right of full frame, and impaired capable left part also has a no signal black region simultaneously;

1.6th, cut type defect model, the video defective characteristics of this model are not lost for whole frame information fully, but increase the horizontal cut shape content of fixed-site or variation on the video image;

1.7th, interframe roll type defect model, the characteristics of the video defective of this model do, shows the parts of images of adjacent two frames in the frame simultaneously, and the motion up or down of rolling in time of two interframe partitioning portions, time remaining at 0.5 second between the several seconds;

1.8th, cross-color type defect model, the notable feature of the video defective of this model be a certain be that the field color of unit is unusual with " OK ", its reason is Sequential Color and Memory system formula signal section phase information disappearance.

3. method according to claim 1 is characterized in that described media materials quality analysis index set of the 2nd step comprises:

2.1st, video signal frame mean value and effective coverage are detected;

2.2nd, the bandwidth of video signal;

2.3rd, vision signal signal to noise ratio;

2.4th, audio signal frame mean value;

2.5th, audio signal bandwidth;

2.6th, audio signal signal to noise ratio.

Images