Embodiment
Below, with reference to accompanying drawing, an example of applicability picture conversion method for detecting of the present invention is specifically described.
Fig. 1 is that picture conversion of the present invention detects and the overall flow figure of representative frame bootstrap technique.Below, with reference to Fig. 1, picture conversion of the present invention is detected and the representative frame bootstrap technique carries out the explanation of outline; And to picture conversion method for detecting and representative frame bootstrap technique, will be in the back, with reference to Fig. 2 to Fig. 4, carry out more concrete narration.No matter picture conversion method for detecting of the present invention and representative frame bootstrap technique are in TV station or at digital television receiver, can carry out.
At first, in the stage 100, from program contents table (EPG), select out classification to current program.In general, the image of digital television broadcasting will be categorized into physical culture, news, Video Music program etc.And,, can from the category classification module of EPG table, obtain the information of this each visual classification.
Then, in the stage 110, decision is used for the classification of being selected out is detected picture conversion, and at least more than one physical features is selected method.The physical features that utilization is determined is selected method, removes to detect current frame and whether has carried out picture conversion (stage 120).At this moment,, so utilize the query image that stored away and the characteristic information of defined, will remove to guide corresponding picture (stage 130) if detected picture conversion.Then, in the picture that is directed, go again to store away (stage 140) after input user's explanation or the note.
In addition,,, or be through with the stage 140, so just go to judge whether to also have next frame (stage 150) if do not detect picture conversion in the stage 120; If there is not next frame, so just go to finish.If not so, so just move to (stage 160) after the next frame, and turn back to the stage 120, again whether the picture conversion of next frame is detected then.
So, by the present invention, can from the category classification module, select out information to each program, and utilize this information classification, will select method with optimal physical features by each classification and detect picture conversion.
Below, with reference to Fig. 2, applicability picture conversion method for detecting of the present invention is more specifically illustrated.
At first, in the stage 200, select out classification to current program from EPG table.Then, from the method for detecting table of each classification, select out each physical features that is used for classification is detected program conversion and select method (stage 210).
Category method for detecting table has behind the data base system, the method that detects with optimal each physical features according to each classification.The physical features method of selecting out each image has local variation, and edge histogram, entropy coding, camera motion detect etc.For example in fact, as the news image, interview, the image of discussion etc. is that a kind of action is little, the image that illuminating effect is also little; In order to detect the picture conversion of this image, preferably go to use pixel histogram and local variation's method.In addition, the music program image many to illuminating effect, that action is also many then will use edge histogram and average information metering method.
Below table 1, show the category of various images and optimal various physical features is selected method.
[table 1]
Select in the method at physical features, local variation, histogram, the entropy coding method is to utilize difference to detect similar figure between each frame, and camera motion then utilizes by Up (making progress), Down (downwards), Left (left), Right (to the right), Zoom-In (furthering), each motion vector value of Zoom-Out (pushing away far away) is selected the picture feature of publishing picture.
By table 1 as can be known, if the football race in order to detect the optimal physical features method for detecting of picture conversion, just utilizes local variation, histogram, camera motion so.In addition, if the cartoon program so preferably utilizes local variation to detect the scene conversion.Below, the physical features that specifies is separately selected method.
(1) local variation (Local Variance)
At first, obtain local variation to luminance signal after, obtain number again above the macro block of local variation's critical value.Then, select out the ratio (DL) of all pieces and above-mentioned macro block number, and utilize this value to select the physical features of the picture of publishing picture.
(2) histogram (Histogram)
Obtain the histogram of past frame and present frame, and select out similar degree by histogrammic intersection point.At this moment, if similar degree does not reach the feature critical value, so just be judged as picture conversion.
Fig. 3 is used for illustrating histogrammic figure.Below, with reference to Fig. 3, the method for trying to achieve the histogram intersection point is described.
The X coordinate representation brightness stage of Fig. 3, the number of Y coordinate representation pixel.And solid line on the figure and dotted line are represented the histogram separately of past frame and present frame respectively, and the part of representing with bar chart is two histogram intersection points between the frame.And if the stage of colored Ci was divided into for 16 stages, the little value part in each stage is exactly histogrammic intersection point so.
At this moment, can try to achieve according to mathematical expression 1 histogram intersection between two frames.
[mathematical expression 1]
In the formula, Fm and Fm+1 are two continuous frames, and Hci (Fm) and Hci (Fm+1) are the numbers to pixel in the image of colored Ci.
By the mathematical expression 2 of utilizing the value of so trying to achieve, the similar figure all to image will be removed to determine.By the similar degree between 2, two frames of mathematical expression, will appear in 0.0~1.0 the scope.
[mathematical expression 2]
Then, select out difference Dh by mathematical expression 3.
[mathematical expression 3]
Dh=1-h(Q∩T)
(3) average information (Entropy)
Utilize mathematical expression 4, remove to ask the difference De of the average information of continuous two frames.
[mathematical expression 4]
D
e=E
m+E
m-1
(4) camera motion (Camera Motion)
To the motion of 6 video cameras, i.e. Left (left), Right (to the right), Down (downwards), Up (making progress), Zoom-In (furthering), Zoom-Out (push away far away) selects out its motion vector, and with these values and prescribed motion vector critical value compare; If surpassed critical value, so just think the motion of video camera, it from detecting, picture conversion is got rid of.
The method for detecting of four above-mentioned physical features nothing but illustrative in order to help the understanding of the present invention, can also append the method for detecting of more physical features certainly.
In addition, by the stage 210 of Fig. 2, as mentioned above, after selecting out the physical features method for detecting of related category, again from the method for detecting table of category, select out the weighted value and the corresponding critical value of all categories (stage 220) of each physical features method for detecting of being selected out.Then, utilize each physical features method for detecting to go to calculate whole difference (stage 230) to present frame.Below, the process of calculating whole difference is carried out specific description more.
At first, select out earlier local variation to classification separately, histogram, and the weighted value separately of entropy coding.Table 2 illustration by the weighted value and the critical value of each category setting.
[table 2]
Then, utilize above-mentioned mathematical expression, ask DL, Dh, the De of corresponding each category item.Table 3 is to utilize above-mentioned mathematical expression to try to achieve the difference of corresponding each category item.
[table 3]
Then,, utilize specific physical features method for detecting to try to achieve difference, and difference be multiply by corresponding weighted value try to achieve weighted difference, and each weighted difference combined try to achieve whole difference (Dtotal) each project by the classification of correspondence image.Be that the situation of a music is that example describes now with classification.With mathematical expression 5, can be in the hope of the whole difference (Dtotal) of music categories.
[mathematical expression 5]
D
total=w
hD
h+w
eD
e
In the formula, wh and we be, as putting down in writing at table 2, be classification when being music histogram and the weighted value separately of entropy coding.Histogram that Dh and De are selected out and entropy coding value.
As mentioned above,, try to achieve whole difference (Dtotal), so just whole difference (Dtotal) is compared (stage 240) with the critical value τ of its corresponding classification present frame if pass through the stage 230 of Fig. 2.At this moment,, so just be judged as and do not carry out picture conversion, just remove termination routine if whole difference does not surpass critical value.
In addition, in the stage 240,, so just go to judge whether being suitable for to camera motion in the physical features method for detecting of related category if whole difference has surpassed critical value.If the classification of inapplicable camera motion so just is judged as and has carried out picture conversion.
But, if be suitable for the classification of camera motion, so just remove to calculate the motion vector of 6 above-mentioned video cameras, and judge wherein whether have at least to surpass a motion vector critical value (stage 250).At this moment, surpass if having, so just being judged as is camera motion, rather than picture conversion, does not go to carry out detect (stage 270) of picture conversion.If in the stage 250, the camera motion vector does not surpass the motion vector critical value, so just is judged as and has carried out picture conversion (stage 260).
Below, with reference to Fig. 4,, promptly utilize the representative frame bootstrap technique of picture conversion method for detecting to carry out specific description to the present invention.
Representative frame bootstrap technique of the present invention is, if detected picture conversion, so just goes to detect the similarity between the representative frame of representing picture separately, so that remove to generate guidance information, allows the user watch collected similar scene.
At first, if detected picture conversion according to the picture conversion method for detecting, representative frame (stage 400) then, is just selected out in the frame of selecting out before going so to select out and the interval of present frame in this interval.Then, detect the physical characteristic information (stage 410) of the physical features method for detecting of related category from mark sheet of all categories.Then, utilize the characteristic information of being selected out, will from representative frame, detect its feature (stage 420).
Then, from the summary image table, select out the query image (stage 430) that has the representative feature of related category.Then, the feature of going to judge the representative feature of the query image of being selected out and representative frame whether similar (stage 440).
If not similar mutually, so just, remove to guide frame according to characteristic information then to the explanation or the note of representative frame input and stored user.
If similar mutually, so just remove to judge the instruction manual or the note (stage 450) that whether have existed in the summary image table query image.If there is user's explanation, so just go from above-mentioned summary image table, select and store corresponding instruction manual (stage 460), go to guide representative frame then.If there is no user's explanation so just from user's input and direction memory, also stores away the feature of representative frame, and comes channeling conduct (stage 470) according to this.
By representative frame bootstrap technique of the present invention, the characteristic information of ISP's transmission will be removed to receive, and be deposited in the mark sheet of all categories, again it is come to each scene classification back channeling conduct, so can be to the similar picture of user's automatic regeneration as inquiry (Query).At this moment, characteristic information can be by forming to the histogram distribution of the ad-hoc location of a frame or to the value of whole histogram distribution.
And by representative frame bootstrap technique of the present invention, the user can only remove to watch required scene or required frame from the program of particular category.In fact, the user can singly go to watch anchorman's appearance scene in news program, or the frame that only goes to collect players for the match's seat in football match is watched for example.
In other example to representative frame bootstrap technique of the present invention, can be when carrying out scene conversion and detect, will be according to method for detecting separately and the combination of the physical features value of being selected out, with detected length of an interval degree information to detecting in the past with next, carry out data base system as guidance information, and it is utilized when guiding automatically, so can carry out the guiding of representative frame more accurately.
In fact, in the football match program, if detected the picture conversion of utilizing local variation, homogeneous texture (Homogeneous Texture) and histogram will occur in succession with the order in spectators, Athlete Seating, place etc. so for example; If detect interval short frame, during 10 seconds, occur repeatedly, so just this picture, be directed to such an extent that split screen or shooting picture go.Perhaps, in news program, event histogram is each representative frame very similarly, distributes all over, so just can be directed on anchorman's the frame.
In addition, if receive anchorman's frame representative picture, the positional information in contents directory by the ISP, some in the histogram feature, so can be in the quilt representative frame of selecting out, the frame that only goes to retrieve the anchorman is classified, and channeling conduct.
Representative frame bootstrap technique of the present invention, because the user can append note to the representative frame that is detected, so when the user watches the contents directory of identical category afterwards, just the frame that has carried out note as inquiry, from current contents directory, detect the similarity of the representative frame that has been detected; If this similarity height so just can utilize note can automatically go channeling conduct.
As mentioned above, in detailed description of the present invention, concrete example has been described; But in not breaking away from technology category of the present invention, various deformation is arranged certainly.In fact, physical features is selected method, except in this manual by illustrative, can also append for example.And though proposed to select the mathematical expression that method is come calculated difference according to each physical features, this mathematical expression can become.Therefore, scope of the present invention is not the example that is confined to be illustrated, but should define by the described claim in back and with the content of claims equalization.
By the present invention, can utilize the classification information that obtains from the EPG table, remove to be suitable for suitable physical features by each classification and detect picture conversion.
In addition, if in broadcast program, detected picture conversion, can guide representative frame with optimal method to each classification so.