JP2009272961A - Content evaluation method, device and program and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide more detailed quality information by analyzing at least one of image, audio and music signals in contents and numerically and accurately measuring the quality thereof for each segment of the contents. <P>SOLUTION: The present invention includes; extracting as an analytic signal image information or audio information in contents or at least one of image information and audio information; storing the analytic signal in a storage means; acquiring the analytic signal from the storage means; referencing a rule storage means storing rules as condition determination sentences, using an analytic signal, of which the generation frequency is varied in accordance with a difference in the qualities of the contents; and calculating, for outputting, a quality value while using a rule sufficiency scale calculated on the basis of the rules corresponding to the analytic signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a content evaluation method and apparatus, a program, and a computer-readable recording medium, and in particular, a content evaluation method and apparatus for automatically evaluating the quality of multimedia content such as moving images, sounds, and music, and the like. The present invention relates to a program and a computer-readable recording medium.

  Currently, the number of multimedia contents is increasing. Multimedia contents mainly include video, audio, and music contents. Since these are all time media, it takes about the same time length as the contents to grasp the contents. Under such a demand for reducing the time cost, there is a demand for a technology that can grasp the contents in advance without viewing the contents.

  Even if it is content in a word, various information is included, but it is recognized that the quality of the content is very important information. For example, commercial dramas and movies are created by professional creators and photographic equipment, and have relatively high image quality, sound quality, and story quality. On the other hand, home videos and the like taken at home are often produced by amateur authors, and therefore have relatively low quality.

  In this way, if the quality of the content can be known, it is possible to estimate in advance what purpose the content is created for and what kind of content it is. come.

  As a technology for evaluating the quality of content using content information, there is a technology that focuses on the movement of the camera at the time of video formulation, and determines that the quality of a segment that includes a sudden movement or camera shake is low (for example, , See Patent Document 1).

In addition, as related technologies, a method for detecting speech segments and music segments (for example, see Patent Document 2), a method for specifying an appearance region of a telop (for example, see Patent Document 3), and a region in which a face is photographed is detected. Method (for example, see Patent Document 4), moving image search method (for example, see Patent Documents 5 and 6), fundamental frequency and power extraction method (for example, Non-Patent Document 1), video structuring method (for example, Non-Patent Document) Reference 2), etc. exist as known techniques.
JP 2000-261757 A Japanese Patent Laid-Open No. 10-187182 Japanese Patent Laid-Open No. 11-328423 JP 2005-157911 A JP 2002-245051 A JP 2006-60796 A Japanese Patent Laid-Open No. 2005-311676 Takumi Furui, “Digital Audio Processing Chapter 4, 4.9 Pitch Extraction”, Tokai University Press, September 1985, pp. 57-59. Yukinobu Taniguchi, Akito Akutsu, Yoshinobu Tonomura, Panorama Excerpts: Video list by automatic panorama image generation / layout, IEICE Transactions D-II, Vol. J82-D-II, NO.3, pp. 390-398, 1999.

  In the prior art, the quality of content is determined using only the movement of the camera. However, since this is included in many videos such as dramas, movies, sports videos and general home videos, when measuring quality based only on this, there is a problem that the accuracy is low. there were. In addition, the binary determination of whether the quality is low or not is limited.

  For the above reasons, there is a problem that the detailed quality of content cannot be provided only by the conventional technology.

  The present invention has been made in view of the above points. By analyzing at least one of an image, a sound, and a music signal in content, and measuring the quality numerically and accurately for each segment of the content. Another object of the present invention is to provide a content evaluation method and apparatus and program capable of providing more detailed quality information, and a computer-readable recording medium.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1) is a content evaluation apparatus that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extraction means 20 for extracting at least one of image information or audio information or image information and audio information in the content as an analysis signal and storing it in the storage means 40;
A rule storage means 70 for storing a rule that is a condition determination sentence using an analysis signal whose frequency of occurrence changes due to a difference in content quality;
Quality value calculation means for acquiring an analysis signal from the storage means 40, referring to the rule storage means 70, and calculating and outputting a quality value using a rule satisfaction measure calculated based on a rule corresponding to the analysis signal 60.

Further, the present invention (Claim 2) is provided in the rule storage means 70.
As a condition judgment sentence,
When there is a cut point during a series of conversations, a condition judgment sentence is stored that evaluates the quality of the content highly, and evaluates the content quality low when there is camera work during the series of conversations.

The present invention (Claim 3) is a content evaluation apparatus that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extraction means for extracting at least one of image information, audio information, or image information and audio information in the content as an analysis signal, and storing it in the storage means;
The analysis signal is read from the storage means, and at least one of the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, and the power level ratio among the analysis signals is used. Quality value calculation means for calculating and outputting a quality value using the feature quantity scale calculated in this manner.

The present invention (Claim 4) is a content evaluation device that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extraction means for extracting at least one of image information, audio information, or image information and audio information in the content as an analysis signal, and storing it in the storage means;
A rule storage means for storing a condition judgment sentence that evaluates the quality of the content highly when there is a cut point during a series of conversations, and evaluates the quality of the content low when there is camera work during the series of conversations;
A first quality value obtained by obtaining an analysis signal from the pre-storage means, referring to the rule storage means, and calculating and outputting a quality value using a rule satisfaction measure calculated based on a rule corresponding to the analysis signal A calculation means;
The analysis signal is read from the storage means, and at least one of the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, and the power level ratio among the analysis signals is used. Second quality value calculating means for calculating and outputting a quality value using the feature quantity scale calculated in this manner.

  FIG. 2 is a diagram for explaining the principle of the present invention.

The present invention (Claim 5) is a content evaluation method for analyzing a content by analyzing at least one of an image, a sound, and a music signal included in the content,
An analysis signal extraction step (step 1) in which the analysis signal extraction unit extracts at least one of image information, audio information, image information, and audio information in the content as an analysis signal and stores it in the storage unit. When,
The quality value calculation means obtains an analysis signal from the storage means, and refers to the rule storage means that stores a rule that is a condition determination sentence using an analysis signal whose occurrence frequency varies depending on the quality of the content. And a quality value calculating step (step 2) for calculating and outputting a quality value using a rule satisfaction measure calculated based on the rule corresponding to.

Further, according to the present invention (Claim 6), in the quality value calculation step (Step 2),
Refers to rule storage means that stores a condition judgment statement that evaluates content quality high when there is a cut point during a series of conversations, and evaluates content quality low when there is camerawork during a series of conversations To do.

The present invention (Claim 7) is a content evaluation method for analyzing a content by analyzing at least one of an image, a sound, and a music signal included in the content,
An analysis signal extraction step in which the analysis signal extraction unit extracts at least one of image information or audio information or image information and audio information in the content as an analysis signal, and stores the analysis signal in a storage unit;
The quality value calculation means reads the analysis signal from the storage means, and among the analysis signal, the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, and the power level ratio A quality value calculating step of calculating and outputting a quality value using a feature amount scale calculated using at least one of the following.

The present invention (Claim 8) is a content evaluation method for analyzing content by analyzing at least one of an image, a sound, and a music signal included in the content,
An analysis signal extraction step in which the analysis signal extraction means extracts at least one of image information or audio information or image information and audio information in the content as an analysis signal, and stores it in the storage means;
The first quality value calculation means obtains the analysis signal from the storage means, evaluates the quality of the content highly when there is a cut point during a series of conversations, and if there is camera work during the series of conversations, the content First, a rule storage means that stores a condition determination sentence that evaluates the quality of the image at a low level is calculated, and a quality value is calculated and output using a rule satisfaction measure calculated based on the rule corresponding to the analysis signal. A quality value calculation step;
The second quality value calculation means reads the analysis signal from the storage means, and among the analysis signals, the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, the power level A second quality value calculating step of calculating and outputting a quality value using a feature amount scale calculated using at least one of the ratios;
I do.

  The present invention (Claim 9) is a content evaluation program for causing a computer to function as each means constituting the content evaluation apparatus according to any one of Claims 1 to 4.

  The present invention (Claim 10) is a computer-readable recording medium storing the content evaluation program according to Claim 9.

  As described above, according to the present invention, a feature that changes the frequency of appearance printing due to a difference in content quality is extracted in advance from an image, a sound, and a music signal, and a condition judgment sentence or an evaluation expression for content quality evaluation is obtained. create. These evaluation feature quantities are extracted from the content to be evaluated, and the evaluation value of the content is calculated using a condition determination sentence and an evaluation formula.

  For features whose frequency of appearance changes due to differences in content quality, for example, editing work (for example, insertion of a telop) is generally performed by using the fact that the quality of the content becomes higher when editing processing is performed. When it is detected that the editing operation has been performed by performing the feature detection process for detecting that the content has been implemented, an evaluation value that highly evaluates the quality of the content is assigned. Also, when creating high-quality content, it was created using multiple cameras, taking advantage of the high frequency of using multiple types of video equipment (for example, using multiple cameras). A feature quantity that can be understood is used for evaluation.

  As described above, according to the present invention, a rule having a feature whose appearance frequency changes due to a difference in content quality as a condition determination is created in advance, and a rule satisfaction measure is calculated to evaluate video quality. As a result, the accuracy of the evaluation result is improved.

  In addition, by evaluating the quality of content for each segment of video, it is possible to perform correct evaluation even for video that is a combination of high-quality video material and low-quality video material.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows the configuration of the content evaluation apparatus according to the first embodiment of the present invention.

  The content evaluation apparatus shown in the figure includes a content storage unit 10, an analysis signal extraction unit 20, a segment division unit 30, an analysis signal memory 40, a segment memory 50, a quality value calculation unit 60, and a rule storage unit 70.

  In the example shown in the figure, when content is input to the analysis signal extraction unit 20 and the segment division unit 30, the content is read from the content storage unit 10, but the present invention is not limited to this example. A display device such as a liquid crystal screen for displaying a quality value output from the quality value calculation unit 60, and an input device or a content evaluation device itself. It is assumed that an output device capable of presenting information according to processing is connected, and various information including output data, processing progress, processing result segments, and the like are output and displayed.

  The analysis signal memory 40 and the segment memory 50 are, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, and the like, and a CD (Compact Disk), a DVD (Digital Versatile Disk) as necessary. ) Or the like. In addition to the analysis signal memory 40 and the segment memory 50, a storage device can be provided as necessary. For example, the content evaluation device according to the present invention is incorporated in a general-purpose PC (Personal Computer). If the storage device can be replaced by an external device, such as when it is used, it does not need to be included.

  The analysis signal extraction unit 20 extracts an audio signal included in the content data and stores it in the analysis signal memory 40.

  The segment dividing unit 30 divides the content into segments based on the audio signal.

  The quality value calculator 60 performs a quality value (Q value) calculation process for each segment, and outputs the result to an output device (not shown).

  FIG. 4 is a flowchart (part 1) of the evaluation method according to the embodiment of the present invention.

  The content evaluation method in the present embodiment is roughly divided into three steps.

Step 10) Analysis signal extraction processing step:
The analysis signal extraction unit 20 analyzes and extracts the moving image and / or the audio signal from the content read from the content storage unit 10 and outputs it to the analysis signal memory 40.

Step 11) Segment division step:
This processing is performed as necessary. The segment dividing unit 30 divides the input content into one or more segments, and outputs the start time and segment length of each segment to the segment memory 50. Here, the segment means the entire content or a partial section in the content. This step is introduced as necessary, but when the start time and segment length of each segment are given in advance or manually, or within the range not related to the technology of the present invention, or the entire content is a segment There is no need to introduce it in some cases.

Step 12) Quality value calculation processing step:
The quality value calculation unit 60 refers to the rule in the rule storage unit 70 for each segment based on the analysis signal acquired from the analysis signal memory 40, the segment start time and the segment length of each segment acquired from the segment memory 50. Calculate the value and output it.

  Hereinafter, the processing of each step will be described in detail.

  The analysis signal processing step of step 10 will be described.

  When the content is input as digital data, the analysis signal extraction unit 20 is divided into an image signal and an audio signal. Of this data, an image signal, an audio signal, or both are extracted as analysis signals as necessary and stored in the analysis signal memory 40.

  This analysis signal may be an image signal alone or an audio signal alone. The subsequent processing can be executed, for example, when only an audio signal is used, when only an image signal is used, or when both are used.

  Next, the segment division processing step of Step 11 will be described.

  Segment division refers to a process of dividing content zero or more times based on image signals, audio signals, or both included in the content.

  This step does not need to be executed as shown in FIG. 5 when a segment is given in advance or when it is not necessary to define a segment.

  Further, the signal used for this processing is not necessarily the same as the analysis signal described above.

  First, a method for generating a segment using an image signal will be described.

  When using an image signal, a segment is generated using structured information.

  As the structured information and its extraction method, there are various extraction methods such as those described in Non-Patent Document 2 described above, that is, cut points, camera work, and the like.

  Of these, any number of pieces of arbitrary information may be used, but a cut point is preferably used, and this may be used immediately after that as a segment boundary. In this case, the segment is generated as a shot sandwiched between the front and rear by the cut point.

  Further, when camera work is used in combination and camera work with a relatively large amount of motion is detected in a shot, the start time or end time of the camera work may be used as a boundary.

  Next, an example of a method for generating a segment using only an audio signal will be described.

  An analysis window having a predetermined fixed window width for the audio signal, for example, 50 ms (milliseconds), and a shift interval, for example, 30 ms is set. Using this analysis window as a unit, the audio signal is divided into a correlated signal and an uncorrelated signal. Here, a correlated signal is a signal having a high autocorrelation function value, such as speech and music by humans and animals, and conversely, an uncorrelated signal is a signal that is not a correlated signal, that is, white. A signal with a low autocorrelation function value such as noise.

  Whether the audio signal included in each analysis window is a correlated signal or an uncorrelated signal can be classified as follows, for example.

  The autocorrelation function value of the audio signal is calculated, and if it is a value exceeding the threshold value, it is regarded as a correlated signal, otherwise it is regarded as an uncorrelated signal. The method of giving this threshold value may be given as a constant in advance, for example, 0.7, or based on the ratio of the time when a constant correlated signal exists and the time when an uncorrelated signal exists, as a reference. A threshold value closest to this ratio may be determined.

  Next, among the analysis windows determined to be correlated signals, a segment is configured by a section configured by continuous correlated windows (analysis windows determined to be correlated signals). By executing this process, it is possible to handle continuous speech of humans and animals, music, and the like as a group of sections, so that it is possible to generate a segment whose meaning can be understood by the viewer.

  An example of a method for configuring a segment will be described.

  The segment configuration is executed by a process for determining a boundary between segments.

Assume that a set of correlated windows F in the content is {F ₁ , F ₂ ,..., F _N } in order from the earliest time. Here, N is the total number of correlated windows.

Next, the correlated window _F i adjacent on the time _{axis, F i + 1} of the time interval, i.e., the end time F _iend of _{F i,} the start time F _{i + 1 start} of the _{F i + 1} is the next analysis window, the time The difference F _{i + 1start} −F _iend is calculated.

Next, the calculation result is compared with a predetermined threshold value, and if it is larger than this, it is considered that F _i and F _{i + 1} are correlated windows belonging to different segments, and the boundary of the segment is defined between these two. And

  By repeating this process for all analysis windows, correlated windows with a time difference can be made into different segments, and as a result, a series of correlated signals without a time difference are combined into the same segment. Can do.

For example, in the example shown in FIG. 6, F _{j + 1} −F _j = T1, F _{j + 2} −F _{j + 1} = T2. If the threshold T _th is set as T 1 <T _th <T 2, two segments BA and BB are configured with T 2 as a boundary.

As the threshold value T _tb is set to a lower value, the number of boundaries increases, so the number of segments to be generated increases. Conversely, as the threshold value T _tb is increased, the number of boundaries decreases, The number of segments to be reduced will decrease.

As a special case, the segment may not be divided at all by setting the threshold value T _tb to a very high value, for example, a value equal to or greater than the time length of the entire content.

Therefore, as described above, the segment in the present invention may be the entire content, and the subsequent processing can be executed even when the threshold value T _tb is set such that the content is not divided at all. The step 11 may not be executed because it does not make sense.

  Further, the correlated signal may be further classified in detail, for example, a speech signal or a music signal by a person or an animal, and these may be divided to generate a segment. In this case, spectrum information can be used as a reference for separating them.

  For example, as a method for classifying the speech audio signal and the music signal, the method described in Patent Document 3 can be used.

  A more detailed segment can be generated by such processing.

  A case will be described in which segment generation is executed using both the last audio signal and image signal.

  For example, in the segment generation using the above-described audio signal, there is a problem that section division cannot be performed on a portion without a correlated window. Therefore, both segment generation using an audio signal and image information, for example, segment generation using a cut point may be applied. By this method, it is possible to define a fine section that cannot be achieved by segment generation by only one side.

  In addition, when effective segment generation cannot be performed by using only one of them, for example, when there is no correlated window in the content or when there is no structured information of an image, it is possible to perform section division complementarily. Is possible.

  Through the above processing, one or more segments can be generated from the content.

  It should be noted that the start time of each segment and its time length are acquired and stored in the segment memory 50 in any of the methods described here and other examples.

  Next, the quality value calculation processing step of step 12 will be described.

  FIG. 7 shows the configuration of the quality calculation unit in one embodiment of the present invention.

  The quality calculation unit 60 includes a feature amount extraction unit 61, a feature amount scale calculation unit 62, a conditional sentence determination feature amount extraction unit 63, a conditional sentence determination unit 64, a rule satisfaction measure scale calculation unit 65, a quality value calculation unit 66, a feature An amount storage unit 601, a feature amount scale (FS) storage unit 602, a conditional sentence determination feature amount storage unit 603, and a rule satisfaction measure scale (RS) storage unit 604.

  FIG. 8 is a flowchart of the quality value calculation process in one embodiment of the present invention.

  Step 12 is a step in which the quality value calculation unit 60 calculates and outputs a quality value (hereinafter referred to as a Q value) that serves as a reference for classifying the segment for each segment based on the analysis signal of each segment.

  The Q value is calculated based on at least one of the two criteria of the feature amount scale FS and the rule sufficiency scale RS.

  In the following, the processing for calculating the feature quantity scale FS and the rule satisfaction scale RS for one segment will be described in detail. When performing processing for each segment, the start time of the segment for which calculation processing is performed and the time length of the segment are acquired from the segment memory 50, and the segment corresponding to the segment for which calculation processing is performed is included in the analysis information. A section is specified, and feature amount extraction processing (S31) or conditional sentence determination processing (S33) is performed from the analysis information of the specified section. In addition, as thresholds and rules necessary for calculating the FAS and the rule sufficiency measure RS, those stored in the rule storage unit 70 in advance are used.

  First, the feature amount scale FS will be described.

  The feature amount scale FS is an index indicating the high quality that can be determined from the feature amount obtained from the analysis information of the analysis signal memory 40. What kind of feature value is used is determined in advance. The feature amount scale FS is calculated through step 31 (feature amount extraction processing step) and step 32 (feature amount scale calculation processing step).

  Step 31) The feature amount extraction unit 61 acquires an analysis signal from the analysis signal memory 40, extracts a predetermined feature amount from the analysis signal, and stores the value of each feature amount in the feature amount storage unit 601.

The feature quantity to be used is, for example, when using an image signal as an analysis signal,
・ Shot length (SB)
-Movement amount (MQ)
-Color histogram (CH)
In the case where an audio signal is used as the analysis signal, for example,
・ Pitch change (DP)
・ Power level ratio (PR)
Etc. are suitable.

  A method for extracting various feature amounts will be described.

<Shot length SB>
The shot length SB indicates the time length of the above-described shot. What is necessary is just to obtain | require this as the duration of the area pinched | interposed between this, after detecting a cut point using the method of the above-mentioned nonpatent literature 2, etc. Video produced by professionals is known to have few long shots and an average shot length of 6-7 seconds. The shot length (SB) is extracted as a feature amount. For example, as shown in FIG. 9, if the shot length is divided like a professional shot, the quality is high, and the shot is greatly separated from the professional one. If so, the quality can be considered low.

<Movement MQ>
For the extraction of the motion amount MQ, the camera parameters calculated at the time of camera work detection described in Non-Patent Document 2 and the movement amounts in the x and y directions can be used. Preferably, the norm is calculated and scalarized. By calculating the degree of camera shake and the smoothness of camera work as the amount of movement (MQ), as shown in FIG. 9, there is a difference between high-quality content and content that is not.

<Color histogram CH>
The color histogram CH divides each image into one or more regions, quantizes average color values, saturation values, lightness values, or arbitrary color information such as RGB and luminance values of each region, It can be calculated by counting the number of appearances. In the case of edited video, the material shot at multiple locations and subjects is carefully cut and edited, so the hue between shots often changes greatly, but in the case of unedited video, shots shot at the same place Since there is a continuous color change, the color change is small (for example, the ground color is dominant in many shots in athletic meet videos). In addition, in the case of content that accurately controls the amount of light or light caused by sunlight or lighting, the subject is clearly photographed and the image has a variety of color changes. As a whole, whiteness is applied, and the luminance value is uniformly high.

<Pitch change DP>
For the pitch change DP, for example, after the pitch F0 is extracted by the pitch extraction method described in Non-Patent Document 1, the time change is calculated as a difference amount. As shown in FIG. 11, a high quality recording of voice / music using a sound collecting microphone or the like tends to have a relatively smooth pitch change and a small difference amount. Those recorded without such consideration include many discontinuous points (jumps) in the pitch, and the amount of difference tends to increase.

<Power level ratio PR>
The power level ratio PR is calculated by, for example, calculating the rms value of the amplitude of the speech waveform as power,
PR = (Average power of the part where the pitch is extracted)
/ (Average power of the part where the pitch is not extracted)
Calculate As shown in FIG. 12, as in the case of the pitch change, those in which the recording is considered tend to have a low power level in the portion where the pitch is not extracted, but those in which the recording is not considered , The power level will be higher. As a result, the former PR tends to be high and the latter is low.

  The above-described feature amount may be selected and used according to the analysis information to be used.

  Step 32) The feature quantity scale calculation unit 62 calculates the feature quantity scale FS from the value of each feature quantity passed from the feature quantity extraction unit 61 and stores it in the feature quantity scale (FS) storage unit 602.

  Below, an example of the process which calculates the feature-value scale FS is described.

  The feature amount scale FS calculates a statistical amount such as an average value and a variance for each feature amount to be used, and is determined based on this value.

  For example, the average value SBA and variance value SBV may be calculated as shot length statistics, and the average value PDA and variance value PDV may be calculated as pitch change PD statistics.

  In order to calculate the feature amount scale FS from these statistics, one or more functions for calculating the feature amount scale FS from each statistic are used. As this function, for example, a linear function may be used, or an arbitrary function such as a upper and lower bounded function such as a sigmoid function or a Gauss function, or a nonlinear function such as a multilayer perceptron or a support vector machine may be used. Good. Further, these functions may be used in combination.

  Hereinafter, an example will be described in which an average value SBA of shot lengths and an average value PRA of power level ratios are employed as feature amounts, and the feature amount scale FS is calculated using each sigmoid function and linear function.

For the average value SBA of the shot length, the contribution FS ^SBA related to the feature amount scale FS is calculated from the ^SBA as follows using a sigmoid function.

ここで、α、βは、予め定めておく定数であり、例えば、α＝１、β＝６など、任意の値としてよい。この例では、プロフェッショナルが制作した映像の平均ショット長がβ＝６であることを仮定し、平均ショット長ＳＢＡが６に近いほどＦＳ^SBAが大きくなる。

Here, α and β are predetermined constants, and may be arbitrary values such as α = 1 and β = 6. In this example, it is assumed that the average shot length of the video produced by the professional is β = 6, and the FS ^SBA increases as the average shot length SBA is closer to 6.

For the average value PRA of the power level ratio, the contribution FS ^PRA from PRA to FS is calculated as follows using a linear mapping.

ここで、γ、εは、予め定めておく定数であり、例えば、γ＝1.0、ε=-0.5など、任意の値としてよい。この場合、パワーレベル比が高い値を取るほどクオリティが高いものであることを想定しており、ＦＳ^PRAが大きくなる。

Here, γ and ε are predetermined constants, and may be arbitrary values such as γ = 1.0 and ε = −0.5, for example. In this case, it is assumed that the higher the power level ratio, the higher the quality, and the FS ^PRA increases.

In the above, the feature amount scale FS is calculated based on the calculated FS ^SBA and FS ^PRA .

  Here, an example of calculating the feature amount scale FS using a linear function will be described.

  For example, the feature quantity scale FS is calculated as follows.

ここで、η、λは、予め定めておく定数であり、例えば、η＝0.6、λ=0.4など、任意の値としてよい。

Here, η and λ are predetermined constants, and may be arbitrary values such as η = 0.6 and λ = 0.4, for example.

  The example of the method for calculating the feature amount scale FS has been described above, but it goes without saying that the same processing can be performed using any combination of feature amounts and any function other than the example described here.

  Preferably, the feature amount scale FS that is finally calculated is calculated so as to fall within the range of 0 to 1 as described above.

  Next, how to obtain the rule sufficiency measure RS will be described.

  The rule sufficiency measure RS is a value calculated depending on whether or not a rule set in advance in the rule storage unit 70 as a reference for determining quality is satisfied. In each rule stored in the rule storage unit 70, video and audio features that are generated differently between high-quality video and low-quality video are set as performance sentences. The rule sufficiency measure RS is calculated through the condition determination feature quantity extraction process in step 30, step 33 (condition determination process step), and step 34 (rule sufficiency measure calculation process step).

  Step 30) In the conditional sentence determination feature amount extraction processing in the conditional sentence determination feature amount extraction unit 63, the feature amount used for the conditional determination in step 33 is extracted from the analysis information (analysis signal). Note that the determination condition in step 33 is defined in advance. Therefore, the feature quantity to be extracted in step 30 is also clarified in advance. In step 30, an existing feature amount extraction process necessary for calculating the rule sufficiency measure RS may be used as a preprocess. Further, the conditional sentence determination feature quantity extraction unit 63 calculates the feature quantity used in step 33 from the feature quantity extraction process described in the feature quantity extraction process (step 31) used in the feature quantity scale FS calculation. May be selected.

  The rule sufficiency measure RS is calculated using one or more rules set in the rule storage unit 70 in advance. Each rule R that determines the rule sufficiency measure RS is described in an if-then format. This is configured based on information obtained by analyzing analysis information, and is described as follows.

"If conditional statement CS then RS addition"
When this conditional statement CS is satisfied, it is determined to be true, and the value is added to the RS by the RS added point described in theen. At this time, the initial value of RS is set to 0.

  Step 33) First, the conditional statement determination unit 64 determines whether or not the segment to be processed satisfies the conditional statement CS.

As an example of the rule R, for example,
R1: "then +0.4 with if telop"
R2: “if face is up then +0.2”
R3: “if the face is facing the direction of the camera during a series of conversations -0.3”
R4: “if there is a cut point in a series of conversations, then +1.0”
R5: “if there is a camera work during a series of conversations, then -0.6”
Etc. are set in advance.

  Here, R4 and R5 are rules based on how well the shooting environment is prepared. Specifically, “R5” is intended to evaluate the camera work for pointing the camera to the conversation speaker, which occurs when shooting with one camera. For this reason, a negative RS addition of −0.6 is given as an index indicating that the photographing environment is not prepared (the number of photographing equipment is not sufficient). On the other hand, an image having a feature such as “R4” can be generated using at least two cameras in an environment that does not require camerawork when switching the speaker. Therefore, a positive RS addition of +1.0 is given as an index indicating that the shooting environment is in place.

  “R1” is a feature that extracts a feature indicating that video editing has been performed under the assumption that a segment that has undergone video editing has high quality, and is added to information indicating that video editing has been performed. It is intended to give an additional RS score. Specifically, since telops are generally inserted by video editing, telops are detected, and RS addition points when there are telops are set to a positive value of +0.4.

  "R2" and "R3" are rules that are statistically common in high-quality content.

  As described above, in order to determine whether or not the conditional statement of the rule set in the rule storage unit 70 is satisfied, it is necessary to perform analysis necessary for determining each conditional statement.

  For example, an example of this analysis method will be described for each of the rules R1 to R5 listed here.

  For example, the telop in the rule R1 is detected by the conditional statement determination unit 64 as to whether or not there is a telop in the segment to be processed by the method described in Patent Document 3, and one or more telops are detected. In this case, it is determined that the conditional statement of rule R1 is satisfied (true). Whether or not there is an R2 face up is determined by detecting the face area in the segment to be processed by the method 9 described in Patent Document 4 and the number of face area pixels is equal to or greater than a predetermined threshold. In the case of the number (that is, when the occupation ratio of the face area in the image is equal to or greater than a predetermined threshold), it is determined that there is a face up in the segment to be processed, and the conditional statement of rule R2 is It determines with satisfy | filling, and stores in RS memory | storage part 604. FIG.

  As for R3, R4, and R5, first, a series of conversation sections are detected from the segments to be processed by the method described in Patent Document 2. In the case of R3, the direction of the face is determined by applying the method described in Patent Literature 4 to the video section corresponding to the time zone detected as a series of conversation sections, and the face direction and the camera. When the angle formed by the optical axis direction is within a predetermined angle, it is determined that the conditional statement of rule R3 is satisfied and stored in the RS storage unit 604.

  In the case of R4, a cut point is detected using a method described in Non-Patent Document 2 for a video section corresponding to a series of conversation sections, and rule R4 is detected when one or more cut points are detected. Is stored in the RS storage unit 604.

  If it is R5, the camera work is detected using the method described in Non-Patent Document 2 or the like for the video section corresponding to the series of conversation sections, and when one or more camera works are detected, the rule of R5 It is determined that the conditional statement is satisfied.

  Step 34) Next, the rule sufficiency measure calculation process in the rule sufficiency measure calculator 65 will be described.

  The rule sufficiency measure RS is calculated by adding an RS additional point to the rule determined to be true after the above condition is determined. For example, in the above example, when it is determined that a certain segment is true for the two rules R1 and R4, the RS is 0.4 + 1.0 = 1.4. In addition to the examples given here, rules regarding information obtained from analysis information may be added as appropriate.

  Furthermore, although the above-mentioned rule is a rule regarding a segment, since it is equal to the entire content when the segment is not divided, a rule for the entire content may be set.

For example,
R6: “if-the total length of the content is 30 seconds or less then-0.9”
R7: “the content bit rate of the content is 9.8 Mbps or more then +0.9”
R8: “if content audio sampling rate is less than 22.050 kHz then −0.8”
Etc. may be set.

  In particular, many of these rules can be referred to as content properties without special analysis. Accordingly, when the content quality value is calculated only from such rules, the analysis method described so far need not be performed.

  Step 35) The quality value calculation process in the quality value calculation unit 66 will be described.

  Based on the FS and RS obtained above, the Q value of the segment is calculated.

  There are various classification methods. Here, an example of calculation using FS, RS, and a sigmoid function will be described.

  For example, the Q value is calculated based on the following mathematical formula.

ここで、ψ、φは予め定める定数であり、例えば、ψ＝０．２、φ＝０．８などとすればよい。この例では、Ｑ値は、０〜１までの範囲を取る。

Here, ψ and φ are predetermined constants, for example, ψ = 0.2, φ = 0.8, and the like. In this example, the Q value ranges from 0 to 1.

  In this example, the example in which the Q value is determined using both FS and RS has been described. However, for example, only one of FS and RS may be used. The function for calculating the Q value may be a linear function or other nonlinear function.

  In this way, as shown in FIG. 13, since the Q value can be calculated for each segment, this Q value is output as the quality value.

  By calculating the Q value for each segment, it is possible to correctly evaluate the quality even for a video in which high-quality video material and low-quality video material are mixed.

  At this time, the information to be output may be output in a list format together with the start time and time length of each segment, as shown in FIG. 14, for example.

  In the example shown above, it can be determined that the higher the Q value, the higher the quality. Therefore, for example, in order to determine a binary value of “high” and “low”, it is determined that a segment having a Q value of 0.5 or higher is high in quality, and a segment having a quality of less than 0.5 is low in quality. You may judge.

  In addition, in order to classify the quality into “very high”, “slightly high”, “slightly low”, and “very low”, the Q value is 0.75 or more and the Q value is 0.5 or more, respectively. The determination may be made in four stages: less than 0.75, Q value of 0.25 or more and less than 0.5, and Q value of less than 0.25.

  By performing such classification, for example, it is possible to request segments belonging to a specific quality among the segments and collect and display only the corresponding segments. For example, when editing using segments included in multiple videos and creating one high-quality video, only those belonging to “very high” and “slightly high” are collected from the classified segments and displayed. You can also.

  There are various ways of classification other than the above, but it goes without saying that an appropriate form may be taken as appropriate.

  In addition, although this invention is a method of evaluating quality, it is not only the use which shows the evaluated quality directly. Some examples are described below.

<Use as content search technology>
In the conventional content search technology, for example, as disclosed in the moving image search methods described in Patent Document 5 and Patent Document 6, the score is based on the similarity ST of the feature amount of the moving image segment and the representative image. In many cases, the results are calculated and the results of ranking in the order of the scores are presented. However, such a method cannot evaluate the quality of content. Some users are particularly interested in viewing high-quality ones. For such users, satisfactory results cannot be obtained only with the conventional moving image search technology.

  Therefore, by considering the quality evaluated by the method of the present invention, not only by the similarity ST but also by preferentially presenting a higher quality, such a moving image that is also beneficial to such a user. Provide search technology.

  As an example of the processing method, for example, contents ranked by similarity ST according to Patent Literature 5, Patent Literature 6, and the like are divided into predetermined sections, for example, (ranked from 1st place, sections for every 30th place, etc.) ) And may be re-sorted in descending order of Q value for each section.

  Alternatively, a new score SC may be calculated from the similarity ST and the Q value calculated by Patent Document 5, Patent Document 6, and the like, and the ranking may be reconfigured based on this score SC.

As an example of this method, for example, using a linear function,
SC = Ψ × ST + Φ × Q value and the like may be calculated.

  At this time, Ψ and Φ are arbitrary constants, for example, Ψ = 0.5, Φ = 0.5, and the like.

  Further, it is not always necessary to use a linear function, and an arbitrary function such as a nonlinear function may be used. Preferably, a function that monotonically increases with respect to the ST and Q values is used.

  In the example shown in FIG. 15, scores with similarity, Q value, Ψ = 0.5, and φ = 0.5 are calculated for eight contents. The table in FIG. 5A shows the results obtained by the conventional method ranking in descending order with respect to the similarity. In this example, a low Q value indicating quality appears at the top of the ranking, and does not satisfy the intention of a user who wants to watch a high quality one. FIG. 5B shows the result of ranking according to the score considering both the similarity and the Q value. As a result, among those having a high degree of similarity, those having a high Q value appear at the top, which satisfies the user's intention.

  Similarly, there is a content recommendation technique for recommending completely new content similar to the content that has been viewed by the user. In this case as well, it is possible to use the same processing method, such as recommending a higher quality and higher quality.

<Use of detection technology as preliminary processing>
Conventionally, a detection technique for automatically detecting a specific section from content has been invented. For example, the technique described in the patent document “Japanese Patent Laid-Open No. 2008-22142” discloses a technique for detecting only a pitching scene from a baseball video. With such a technique, it is difficult to obtain the intended effect depending on the content to be processed. In the above example, since it is a technique for detecting a pitching scene of a baseball video, the effect cannot be obtained for content other than the baseball video.

  Therefore, for example, when the above-described detection technology is used for a content database that includes content other than baseball video, if preparatory processing that narrows down what seems to be baseball video can be introduced in advance, the effect can be reduced or wasted. This is advantageous because it can save processing. In this example, baseball video is taken up as content, but the same can be said for movies, dramas, news, various sports videos, and the like.

  The content as described above is mainly created by professional creators, and the quality is high. Therefore, by narrowing down high-quality content in advance by the technology of the present invention, for example, the detection technology as described above can be supported and effective processing can be realized.

  In addition, although patent document "Unexamined-Japanese-Patent No. 2008-22142" was illustrated here as an example, it cannot be overemphasized that it is applicable with respect to the arbitrary detection techniques which can consider the reduction of the same effect.

  As described above, according to the present embodiment, the quality of the segment can be calculated and presented as a numerical value by analyzing various information of the content based on the analysis signal in the content. This provides a solution to the problem that the accuracy of the evaluated quality is low and the number of classifications is small in the prior art.

  Further, the operation of the components of the content evaluation apparatus shown in FIG. 3 can be constructed as a program and installed in a computer used as the content evaluation apparatus for execution or distributed via a network. .

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims. For example, XML (eXtensible Markup Language) data including information on the generated segment may be generated. If it is general-purpose XML data, the usability of the output result generated by the present invention can be improved.

  The present invention is applicable to all techniques for evaluating the quality of video.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the content evaluation apparatus in one embodiment of this invention. It is a flowchart (the 1) of the evaluation method in one embodiment of this invention. It is a flowchart (the 2) of the evaluation method in one embodiment of this invention. It is a figure showing an example of the partial content production | generation process in one embodiment of this invention. It is a block diagram of the quality value calculation part in one embodiment of this invention. It is a flowchart of the quality value calculation process in one embodiment of the present invention. It is an example of shot length SB in one embodiment of the present invention. It is an example of the amount of movement MQ in one embodiment of the present invention. It is an example of pitch change DP in one embodiment of the present invention. It is an example of power level ratio PR in one embodiment of the present invention. It is an example of Q value in one embodiment of the present invention. It is an example of the output result in one embodiment of this invention. It is an example of a search result.

Explanation of symbols

10 content storage unit 20 analysis signal extraction unit, analysis signal extraction unit 30 segment division unit 40 storage unit, analysis signal memory 50 segment memory 60 quality value calculation unit, quality value calculation unit 61 feature quantity extraction unit 62 feature quantity scale calculation unit 63 Condition sentence determination feature quantity extraction section 64 Condition sentence judgment section 65 Rule satisfaction level scale calculation section 66 Quality value calculation section 70 Rule storage means, rule storage section 601 Feature quantity storage section 602 Feature quantity scale (FS) storage section 603 Condition sentence Feature storage unit for determination 604 Rule satisfaction scale (RS) storage unit

Claims (10)

A content evaluation apparatus that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extraction means for extracting at least one of image information or audio information or image information and audio information in the content as an analysis signal, and storing the analysis signal in a storage means;
A rule storage means for storing a rule that is a condition determination sentence using an analysis signal whose frequency of occurrence varies depending on the quality of content;
Quality value calculation that obtains the analysis signal from the storage means, refers to the rule storage means, calculates a quality value using a rule satisfaction measure calculated based on a rule corresponding to the analysis signal, and outputs the quality value Means,
A content evaluation apparatus comprising: The rule storage means includes
As the condition judgment sentence,
When there is a cut point during a series of conversations, a condition judgment sentence that evaluates the quality of the content highly, and evaluates the content quality low when there is camera work during a series of conversations, is stored.
The content evaluation apparatus according to claim 1. A content evaluation apparatus that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extracting means for extracting image information or audio information in the content, or at least one of image information and audio information as an analysis signal, and storing it in a storage means;
The analysis signal is read from the storage means, and at least one of the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, and the power level ratio among the analysis signals is read out. A quality value calculating means for calculating and outputting a quality value using a feature amount scale calculated using
A content evaluation apparatus comprising: A content evaluation apparatus that analyzes at least one of an image, a sound, and a music signal included in content and evaluates the content,
Analysis signal extracting means for extracting image information or audio information in the content, or at least one of image information and audio information as an analysis signal, and storing it in a storage means;
A rule storage means for storing a condition judgment sentence that evaluates the quality of the content high when there is a cut point during a series of conversations and evaluates the quality of the content low when there is camera work during the series of conversations;
First obtaining the analysis signal from the storage means, referring to the rule storage means, and calculating and outputting a quality value using a rule satisfaction measure calculated based on a rule corresponding to the analysis signal Quality value calculation means,
The analysis signal is read from the storage means, and at least one of the shot length of the image signal, the amount of motion, the color histogram, the pitch change, the power level, the pitch change of the audio signal, and the power level ratio among the analysis signals is read out. Second quality value calculating means for calculating and outputting a quality value using a feature amount scale calculated using
A content evaluation apparatus comprising: A content evaluation method for analyzing at least one of an image, a sound, and a music signal included in content and evaluating the content,
An analysis signal extraction step in which the analysis signal extraction unit extracts at least one of image information or audio information or image information and audio information in the content as an analysis signal, and stores the analysis signal in a storage unit;
Quality value calculating means refers to a rule storage means that acquires the analysis signal from the storage means, and stores a rule that is a condition determination sentence using an analysis signal whose occurrence frequency changes due to a difference in content quality, A quality value calculating step for calculating and outputting a quality value using a rule satisfaction measure calculated based on a rule corresponding to the analysis signal;
The content evaluation method characterized by performing. In the quality value calculating step,
The rule storage means for storing a condition judgment sentence that evaluates the quality of the content highly when there is a cut point during a series of conversations and evaluates the content quality low when there is camera work during a series of conversations The content evaluation method according to claim 5, which is referred to. A content evaluation method for analyzing at least one of an image, a sound, and a music signal included in content and evaluating the content,
An analysis signal extraction step in which the analysis signal extraction unit extracts at least one of image information or audio information or image information and audio information in the content as an analysis signal, and stores the analysis signal in a storage unit;
Quality value calculation means reads the analysis signal from the storage means, and among the analysis signals, shot length of image signal, motion amount, color histogram, pitch change, power level, pitch change of audio signal, power level ratio A quality value calculating step of calculating and outputting a quality value using a feature amount scale calculated using at least one of
The content evaluation method characterized by performing. A content evaluation method for analyzing at least one of an image, a sound, and a music signal included in content and evaluating the content,
An analysis signal extracting unit that extracts at least one of image information or audio information or image information and audio information in the content as an analysis signal, and stores the analysis signal in a storage unit;
The first quality value calculation means acquires the analysis signal from the storage means, and when there is a cut point during a series of conversations, evaluates the quality of the content highly, and there is camera work during the series of conversations. In this case, the rule storage means storing the condition judgment sentence that evaluates the quality of the content is referred to, and the quality value is calculated and output using the rule satisfaction measure calculated based on the rule corresponding to the analysis signal. A first quality value calculating step;
Second quality value calculation means reads the analysis signal from the storage means, among the analysis signal, shot length of the image signal, motion amount, color histogram, pitch change, power level, pitch change of the audio signal, A second quality value calculating step of calculating and outputting a quality value using a feature amount scale calculated using at least one of the power level ratios;
The content evaluation method characterized by performing.   The content evaluation program for functioning a computer as each means which comprises the content evaluation apparatus of any one of Claims 1 thru | or 4.   A computer-readable recording medium storing the content evaluation program according to claim 9.

Images