KR20160107734A

KR20160107734A - Method for classifying objectionable movies using duration information and apparatus for the same

Info

Publication number: KR20160107734A
Application number: KR1020150030919A
Authority: KR
Inventors: 정치윤
Original assignee: 한국전자통신연구원
Priority date: 2015-03-05
Filing date: 2015-03-05
Publication date: 2016-09-19

Abstract

Disclosed are a method and an apparatus for classifying harmful video by using duration information of video. An apparatus for classifying harmful video by using duration information of video according to an aspect of the present invention comprises: a video determination group selection unit which selects a determination group, in which a sampling period of an analysis target video, a harmfulness determination method and classification features have been defined, for an input video by analyzing duration information of the input video; a frame extraction unit which extracts frames of the analysis target video from the input video in accordance with the sampling period determined by the video determination group selection unit; a harmfulness analysis unit which extracts a plurality of video features from the frames of the analysis target video extracted by the frame extraction unit, and determines harmfulness of the frames of the analysis target video by analyzing the video features; a scene change detection unit which determines whether a scene change occurs by analyzing similarity between the frames of the analysis target video extracted by the frame extraction unit; and a video harmfulness determination unit which determines harmfulness of the input video by selectively using pieces of information, collected by the harmfulness analysis unit and the scene change detection unit, based on the determination group selected by the video determination group selection unit.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for classifying a harmful video using temporal information of a moving picture,

The present invention relates to a harmful moving picture classification method and apparatus, and more particularly, to a harmful moving picture classification method and apparatus that classifies a harmful moving picture into a plurality of moving picture groups using time information of moving pictures, The present invention relates to a method and apparatus for effectively classifying an object.

The sharing of video files through the internet such as web hard and P2P has been rapidly increased, making it easier for adolescents to access the harmful videos, resulting in various social problems. In order to protect youth from harmful contents, it is necessary to determine the harmfulness of the video when the video is uploaded from a file sharing site such as web hard or P2P, and to block it from the source. Core technology for detecting the harmfulness of the video file Technology.

There have been many researches on how to judge the harmfulness of a video file. In general, the method of judging harmfulness of a moving image includes a step of extracting a plurality of images from a moving image, a step of analyzing the harmfulness of the extracted image, And determining the harmfulness of the moving picture using the motion picture.

In order to improve the classification performance in the method of judging the harmfulness of the existing video file, various methods of extracting various visual characteristics at the stage of analyzing the harmfulness of images have been studied. As the number of features to be extracted increases, It takes a lot of time to extract the features. In particular, the processing time is not a problem when extracting a plurality of features from a single image. However, in order to determine the harmfulness of a moving image file, the processing time is rapidly increased when a plurality of frames are extracted from a moving image.

As another method to improve the classification performance, there have been many studies on the method of combining the motion information of the whole video and the characteristics of the audio as well as the harmfulness information of the image in the step of judging the harmfulness of the moving image. However, there is a disadvantage in that it takes a long time to extract the audio from the whole moving picture, or extract and analyze the motion information from the whole moving picture.

The video files can be divided into a plurality of groups according to duration. For example, if you have a video like a movie, you have a very long video group with a duration of over 60 minutes, an ad, a movie trailer, a video group with less than 5 minutes of short duration such as a UCC video, and an intermediate video group such as a drama, .

In the case of a harmless video belonging to a video group having a short playback time, a scene changes suddenly and frequently changes in order to provide a lot of information in a short time such as an advertisement and a movie trailer. On the other hand, in the case of a harmful video belonging to a video group having a short playback time, there are many images captured using a fixed camera or a single camera, so that the scene does not change rapidly and is almost the same.

Existing technologies have difficulty in improving the detection performance by using the same classification characteristic and judgment method without using the characteristic of the group according to the moving picture time. Also, in order to improve the detection performance, a plurality of classification features and judgment methods are used, which increases the time required for judging a moving picture file.

In order to solve the problems of the related art described above, the present invention provides a method and apparatus for classifying a harmful video effectively by using classification information and a determination method appropriate for the characteristics of each group after classifying the video into a plurality of video groups using time information of the video And to provide a method for the same.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a harmful motion classification apparatus using temporal information of a moving picture, the apparatus comprising: means for analyzing time information of an input moving picture, A moving picture judgment group determining unit for determining a judgment group in which a judgment method and a classification feature are defined; A frame extracting unit for extracting an analysis target image frame from the input moving image according to a sampling period determined by the moving image determining group determining unit; A harmfulness analyzer for extracting a plurality of video features from the analysis target image frame extracted by the frame extracting unit and analyzing the plurality of video features to determine the hazard of the analysis target image frame; A scene change detection unit for analyzing a similarity between the analysis target image frames extracted by the frame extraction unit and determining whether there is a scene change; And a video harmfulness determining unit for determining the harmfulness of the input moving image by selectively using the information collected by the harmfulness analyzing unit and the scene change detecting unit according to the determination group determined by the moving image determining group determining unit.

As described above, according to the present invention, when judging a harmful moving picture, a judgment group of a moving picture is set by utilizing characteristics according to time of moving picture, and the accuracy of the judgment of moving picture is applied by applying different analysis and judgment methods to each group. Can increase

Further, in the case of a moving image having a short playback time, a threshold value for judging a moving image is adjusted by using a prior knowledge such as a feature that a scene changes rapidly and often in order to provide a lot of information at a limited time such as an advertisement or a movie trailer Thereby reducing the possibility that a harmless video is judged to be a harmful video.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating a configuration of a harmful moving picture classification apparatus using time information of moving pictures according to an embodiment of the present invention. FIG.
FIG. 2 is a flowchart illustrating a process of determining a decision group in the motion decision group determination unit of FIG. 1. FIG.
3 is a flowchart illustrating a harmful moving picture classification method using moving picture time information according to the first embodiment of the present invention.
FIG. 4 is a flowchart illustrating a harmful moving picture classification method using moving picture time information according to a second embodiment of the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram illustrating a configuration of a harmful moving picture classification apparatus using time information of moving pictures according to an embodiment of the present invention.

Referring to FIG. 1, a harmful motion classification apparatus using time information of a moving picture according to an embodiment of the present invention includes a moving picture judgment group determination unit 100, a frame extraction unit 200, a harmfulness analysis unit 300, A detection unit 400, and a video harmfulness determination unit 500.

The moving picture judgment group determination unit 100 analyzes the time information of the input moving picture and determines a judgment group defining a sampling period, a harmfulness determination method, and a classification characteristic of the analysis target image from the input moving picture.

The video files can be divided into a plurality of groups according to duration. For example, in the case of movies and commercial pornography, since the playing time of a moving picture is very long, it is possible to have a statistical meaning even if a frame is sampled and analyzed at a uniform interval.

However, in the case of a short commercial, a movie trailer, or a UCC image, the reproduction time is short. Therefore, when the same number of images are sampled and inspected, a similar frame is repeatedly analyzed.

For this reason, in the case of a moving image having a short playback time, it is necessary to change the sampling method. Also, in the case of a moving image having a short playback time, since the scenes change rapidly in order to provide a lot of information at a limited time, such as an advertisement or a movie trailer, there is a characteristic that changes frequently. It is necessary.

For example, assuming that 100 images are always extracted from a video file to determine the hazard, a video file having a playback time of 100 minutes is extracted one video at a time, and a video file having a playback time of 100 seconds is recorded at 1 One at a time. Since the contents of an image do not change greatly for 1 second in a moving image file, if a sampling is performed at intervals of 1 second, a similar image is repeatedly extracted. Therefore, it is necessary to define a minimum sampling interval. Assuming that the minimum sampling interval is 5 seconds, 20 images are extracted at intervals of 5 seconds for a moving image file of 100 seconds.

According to the embodiment of the present invention, the moving image judgment group is defined as being classified into three. The number of video judgment groups can be changed according to the setting, and the scope of rights of the present invention is not limited to the three video judgment groups. For convenience of explanation, it is assumed that three moving image judgment groups are set in FIGS. 1 to 3. FIG.

In the judgment group 1, when the reproduction time of the moving picture is spicy, in the judgment group 3, the reproduction time of the moving picture is very short. In the judgment group 2, in the case of the moving picture having the reproduction time between the judgment group 1 and the judgment group 3 (Sampling, cycle, hazard determination method, classification characteristic, etc.) are defined.

For example, in the case of the judgment group 1, the sampling period T for extracting a frame to be analyzed is calculated according to Equation (1) below because the duration of the moving picture is long. In Equation (1), N denotes the total number of frames to be extracted from the moving picture, and duration denotes the playing time of the moving picture.

[Equation 1]

T = duration (s) / N

In the case of the judgment group 1, it takes a long time to use the scene change detection function because the moving picture reproduction time is long. Since there is no difference between the harmful image and the harmless image, the harmfulness is judged using only the image information .

On the other hand, the judging group 2 and the judging group 3 include a moving image having a reproduction time which can not extract the total number of frames N necessary for the analysis in the minimum sampling period D defined beforehand. For example, if a predefined minimum sampling period is 5 seconds and the total number of extracted frames is 100, this corresponds to a moving image having a playback time of 500 seconds or less.

In addition, the distinction between the judgment group 2 and the judgment group 3 is determined according to the number of frames which can be extracted in the minimum sampling period and the number of the minimum sequential judgment frames which are defined in advance and the comparison result.

For example, if the number of frames that can be extracted in the minimum sampling period is larger than the predetermined minimum sequence number of frames to be determined in advance, it belongs to the judgment group 2, and if not, belongs to the judgment group 3.

On the other hand, when sequentially analyzing and analyzing image frames to be analyzed from input moving images, it is a sequential determination method to make a judgment when the number of frames is more than a certain number of frames having statistical significance. At this time, the number of frames having a statistical significance is defined as the minimum number of sequential frames. The sequential determination method reduces the time required to scan a video file because it does not have to check all the frames.

The moving images belonging to the judgment group 2 are analyzed using the sequential judgment method, and the moving pictures belonging to the judgment group 3 do not use the sequential judgment method because the reproduction time is very short. In addition, because the moving picture belonging to the judgment group 3 includes many advertisements, CFs, etc., scene change detection is performed, and the information is used to judge the harmfulness of the moving picture.

FIG. 2 is a flowchart illustrating a process of determining a decision group in the motion decision group determination unit of FIG. 1. FIG. Hereinafter, a process of determining a decision group in the video decision group determination unit according to an embodiment of the present invention will be described with reference to FIG. 1 and FIG.

The moving picture judgment group determining unit 100 analyzes the playing time of the input moving picture in step S100 and compares the playing time of the analyzed moving picture with a value obtained by multiplying the minimum sampling period D of the moving picture by the total number of extracted frames N (S220).

If it is determined in step S220 that the playback time of the moving image is greater than the value obtained by multiplying the minimum sampling period D of the moving image by the total number of extracted frames N, it is determined that the input moving image belongs to the determination group 1.

The sampling period of the moving image belonging to the judgment group 1 is determined according to the moving image playback time, and the sequential moving image determination method is used in the moving image analysis, and scene change detection is not performed.

If it is determined in step S220 that the reproduction time of the moving image is smaller than the product of the minimum sampling period D of the moving image and the total number of extracted frames N, the number of extracted frames is compared with the preset minimum sequence number of frames to be determined (S230).

The number of extracted frames is calculated by dividing the reproduction time of the moving picture by the minimum sampling period (D). If it is determined in step S230 that the extracted frame number is larger than the minimum number of sequential frames, it is determined that the input moving image belongs to the judgment group 2. The moving picture belonging to the judgment group 2 is sampled using the minimum sampling period. In the moving picture analysis, the sequential moving picture determination method is used, and scene change detection is not performed.

As a result of the determination in step S230, if the extracted frame number is smaller than the minimum number of sequential frames, it is determined that the input moving image belongs to the judgment group 3. The moving picture belonging to the judgment group 3 is sampled using the minimum sampling interval. In the moving picture analysis, the sequential moving picture determination method is not used and the scene change detection information is used for the harmful moving picture determination.

Referring back to FIG. 1, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to a sampling period determined by the moving image determining group deciding unit 100.

For example, when it is determined that the input moving image belongs to the determination group 1, the analysis target image frame in the input moving image is extracted according to the sampling period calculated in the above Equation (1). If it is determined that the input moving image belongs to the determination group 2 or the determination group 3, the analysis target image frame is extracted according to a predetermined minimum sampling period.

The harmfulness analysis unit 300 extracts a plurality of video features from the analysis subject image frame extracted by the frame extraction unit 200 and analyzes the plurality of video features to determine the hazard of the analysis subject image frame . The plurality of video features extracted here may include information such as color, texture, shape, and the like.

The scene change detection unit 400 compares video characteristics such as color, texture, and shape extracted from the analysis target image frame extracted by the frame extraction unit 200 with the video characteristics of the previous image, .

The scene change detection unit 400 generates information such as a scene change time, an interval between scene changes, and a frequency of scene change in an input moving image, and transmits the generated information to a video deterioration determining unit 500 to be described later.

The video harmfulness determining unit 500 determines the harmfulness of the input moving image by using the information collected from the harmfulness analyzing unit 300 and the scene change detecting unit 400.

At this time, the video harmfulness determining unit 500 selectively uses the information collected from the harmfulness analyzing unit 300 and the scene change detecting unit 400 based on a judgment group to which the input moving image belongs.

For example, when the input moving image belongs to the judgment group 1 or the judgment group 2, the video harmony judging unit 500 does not use the information collected by the scene transformation detecting unit 400, .

Alternatively, when the input moving image belongs to the judgment group 3, the video harmfulness judgment unit 500 uses the information collected in the scene change detection unit 400 and the information collected in the harmfulness analysis unit 300 to input Determine the hazard of video.

If it is determined that the input moving image belongs to the judgment group 1 or the judgment group 2, the video harmony judging unit 500 judges whether the number of image frames I and the number of harmful frames O, And determines that the input video is a harmful video if the ratio is greater than or equal to a predetermined threshold value and determines that the input video is a harmless video if the ratio is greater than or equal to a certain threshold. Here, the determination as to whether or not the image frame is harmful in the analysis target image frame is made using the result analyzed by the harmfulness analysis unit 300 described above.

&Quot; (2) "

Alternatively, if it is determined that the input moving image belongs to the judgment group 3, the video harmony judging unit 500 judges whether the input moving image is good or harmless by using the information received from the scene change detection unit 400, It is judged secondarily.

As a result of the determination, if the input moving image is primarily determined to be harmless video, it is finally determined whether the input moving image is good or bad using Equation (3) below.

&Quot; (3) "

For example, the video harmony judging unit 500 judges whether or not the video input / output of the input moving image has occurred by using information such as the time at which the scene conversion received from the scene change detecting unit 400 occurs, the interval between scene changes, It primarily judges whether it is harmless.

At this time, the video harmony judging unit 500 uses the characteristic that the frequency of the scene change is high and the scene change interval is narrow in the case of the harmless moving picture.

As a result of the determination, if the input moving image is primarily determined to be harmless video, the video harmfulness determining unit 500 increases the threshold value by adding a predetermined offset to a preset threshold value, To determine whether the input video is good or bad. That is, when it is primarily determined that the video is a harmless video, the offset is added to the threshold to increase the probability that the input moving image can be determined as harmless.

Alternatively, if the video harmfulness determining unit 500 determines that the input moving image is primarily a harmful video, the harmfulness of the input moving image is determined using Equation (2).

In the foregoing, the specific configuration of the harmful moving picture classification apparatus using time information of the moving picture according to the present invention and the functions and operations of the respective constitutions have been described. Hereinafter, referring to FIGS. 3 and 4, a description will be made of a harmful moving picture classification method using moving picture time information according to another embodiment of the present invention.

3 is a flowchart illustrating a harmful moving picture classification method using moving picture time information according to the first embodiment of the present invention.

Referring to FIGS. 1 and 3, the moving image determining group determining unit 100 determines an input moving image determining group (S310). Each judgment group is defined with a judgment policy for analyzing the moving images belonging to the group or judging whether or not it is good or harmless.

For example, in determining the determination group of the input moving image, the moving image determination group determination unit 100 may use the reproduction time information of the input moving image. That is, the result of comparing the playback time of the moving image with the value obtained by multiplying the minimum sampling period (D) of the moving image by the total number of extracted frames (N), and comparing the number of extracted frames of the input moving image with the preset minimum- According to the result, the input moving image can be classified into the corresponding judgment group.

If the input moving image is determined to be the judgment group 1 (S321), it is determined whether the input moving image is null / non-harmful according to the judgment policy defined in the judgment group 1.

For this, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to the sampling cycle defined in the judgment group 1 (S331), and the hazard analysis unit 300 extracts, / A plurality of video features for harmlessness are extracted and analyzed (S341). Accordingly, it is determined whether or not each analysis target image frame is harmful, and the plurality of video features may include information such as color, texture, and shape.

The video harmfulness determining unit 500 analyzes the harmfulness of the moving image using Equation (2) that can determine the hazard of the moving picture before extracting the frame by the total number of extracted frames (N) when sufficient hazard judgment information is accumulated And terminates the input video by applying the sequential test method of terminating the input video (S351).

If the input moving image is determined to be the determination group 2 (S323), it is determined whether the input moving image is good or bad according to the determination policy defined in the determination group 2.

For this, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to the sampling cycle defined in the judgment group 2 (S333). At this time, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to a predetermined minimum frame extraction period.

The harmfulness analysis unit 300 extracts and analyzes a plurality of video features for judging whether the video frames are analyzed in step S343. Accordingly, it is determined whether or not each analysis target image frame is harmful, and the plurality of video features may include information such as color, texture, and shape.

The video harmfulness determining unit 500 analyzes the harmfulness of the moving image using Equation (2) that can determine the hazard of the moving picture before extracting the frame by the total number of extracted frames (N) when sufficient hazard judgment information is accumulated And terminates the input video by applying the sequential test method of terminating the input video (S353).

If the input moving image is determined to be the determination group 3, it is determined whether the input moving image is good or bad according to the determination policy defined in the determination group 3.

For this, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to the sampling cycle defined in the judgment group 3 (S335). At this time, the frame extracting unit 200 extracts an analysis target image frame from the input moving image according to a predetermined minimum frame extraction period.

The harmfulness analyzing unit 300 extracts and analyzes a plurality of video features for judging whether a video object is an object or a harmless object in each analysis object image frame (S345). Accordingly, it is determined whether or not each analysis target image frame is harmful, and the plurality of video features may include information such as color, texture, and shape.

In addition, the scene change detection unit 400 detects whether or not the scene change of the analysis target image frame is performed (S355). Accordingly, information such as the time at which the scene change occurs in the input moving image, the interval between scene changes, and the frequency of the scene change is generated and transmitted to the video hazard determiner 500.

In step S365, the video harmony judging unit 500 judges whether the scene change is detected using the information collected from the harmfulness analyzing unit 300 and the scene change detecting unit 400 And determines the hazard of the input moving image (S375).

4 is a flowchart illustrating a harmful moving picture classification method using moving picture time information according to a second embodiment of the present invention.

4, the input moving picture is classified into two judgment groups, unlike the method shown in FIG.

Referring to FIGS. 1 and 4, the moving picture determining group determining unit 100 analyzes the playing time of the input moving picture (S410). When the playing time of the moving picture is less than the minimum sampling interval D and the total number of frames N, (Step S420). In step S420, the determination group is divided into a moving image that is larger than the time obtained by multiplying the moving image and the other moving images.

If the playback time of the moving image is larger than the time obtained by multiplying the minimum sampling interval D by the total frame number N, the frame extracting unit 200 extracts an analysis from the input moving image according to the sampling cycle defined in Equation (2) A target image frame is extracted (S431).

Then, the harmfulness analyzing unit 300 extracts and analyzes a plurality of video features for judging whether the video object is an object or an uninhabitable object (S441). Accordingly, it is determined whether or not each analysis target image frame is harmful, and the plurality of video features may include information such as color, texture, and shape.

If the sufficient hazard judgment information is accumulated, the video harmfulness judging unit 500 judges the harmfulness of the moving picture by using Equation (2) which can determine the hazard of the moving picture before extracting the frame by the total number of extracted frames N Analyzing and terminating the input video by applying a sequential test method (S451).

As a result of the determination in step S420, if the playback time of the moving image is shorter than the time obtained by multiplying the minimum sampling interval D by the total frame number N, the frame extracting unit 200 extracts the minimum sampling period D defined in advance The analysis target image frame is extracted from the input moving image (S433).

Then, the harmfulness analyzing unit 300 extracts and analyzes a plurality of video features for judging whether the video frames are analyzed in step S443. Accordingly, it is determined whether or not each analysis target image frame is harmful, and the plurality of video features may include information such as color, texture, and shape.

Next, the scene change detection unit 400 detects whether the scene of the analysis target image frame is transformed (S453). Accordingly, information such as the time at which the scene change occurs in the input moving image, the interval between scene changes, and the frequency of the scene change is generated and transmitted to the video hazard determiner 500.

In step S463, the video harmony judging unit 500 judges whether the scene change is detected using the information collected by the harmfulness analyzing unit 300 and the scene change detecting unit 400 And judges the harmfulness of the input moving image (S473).

According to the present invention, in determining a harmful video, a determination group of a video is set by utilizing characteristics according to time of the video, and the accuracy of video determination can be increased by applying different analysis and determination methods to each group

Meanwhile, the harmful moving picture classification method using moving picture time information according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media storing data that can be decoded by a computer system. For example, there may be a ROM (Read Only Memory), a RAM (Random Access Memory), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device and the like. The computer-readable recording medium may also be distributed and executed in a computer system connected to a computer network and stored and executed as a code that can be read in a distributed manner.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalents shall be construed as being included within the scope of the present invention.

100: video judgment group determination unit 200: frame determination unit
300: Hazard analysis part 400: scene change detection part
500: video hazard judgment unit

Claims

A moving picture judgment group determination unit for analyzing time information of an input moving picture and determining a judgment group in which a sampling period, a harmfulness judgment method and a classification characteristic of the analysis target image are defined from the input moving picture;
A frame extracting unit for extracting an analysis target image frame from the input moving image according to a sampling period determined by the moving image determining group determining unit;
A harmfulness analyzer for extracting a plurality of video features from the analysis target image frame extracted by the frame extracting unit and analyzing the plurality of video features to determine the hazard of the analysis target image frame;
A scene change detection unit for analyzing a similarity between the analysis target image frames extracted by the frame extraction unit and determining whether there is a scene change; And
And a video harmfulness determining unit for determining the harmfulness of the input moving image by selectively using the information collected by the harmfulness analyzing unit and the scene change detecting unit according to the determination group determined by the moving image determining group determining unit,
And the time information of the moving picture.