KR20220108915A - Real-time harmfulness inspection apparatus and method for video content - Google Patents

Abstract

The present invention relates to an apparatus and method for examining real-time harmfulness of video content. The method includes the steps of: receiving a real-time streaming video file from a video server according to a user's video content request; determining the necessity of inspecting the harmfulness of the video file by checking the certificate included in the received streaming video file; if a harmfulness test is required, extracting a plurality of image frames per second from the video file based on the performance of a user terminal and the resolution of an image by starting the harmfulness test during streaming of the video file; examining the harmfulness of each of the plurality of extracted image frames; determining the harmfulness of the current image frame by obtaining a correlation between a harmfulness test result of the current image frame and a harmfulness test result of the plurality of image frames in the previous specific section; and reflecting a reading result in a corresponding section of the image being played so that safe image frames are normally played and harmful image frames are stopped or covered.

Description

Real-time HARMFULNESS INSPECTION APPARATUS AND METHOD FOR VIDEO CONTENT of video content

The present invention relates to image content harmfulness inspection technology, and more particularly, a real-time image content harmfulness inspection apparatus capable of inspecting for harmfulness in real time along with the reproduction of streamed image content and immediately reflecting the result on the image being reproduced. and methods.

With the advent of the interactive media era in which anyone can produce and consume media content such as video, real-time communication is possible and sharing and spreading speed are fast, resulting in an explosive increase in the use of video content creation and streaming.

Since anyone can easily upload content, sexually suggestive and provocative videos such as pornography and violence are also increasing, which is emerging as a social problem. Accordingly, the development of related technologies for screening and blocking the harmfulness of image information has been in progress. Representatively, there are keyword-based blocking technology, hash list-based blocking technology, feature-based blocking technology, and content-based blocking technology.

However, in performing a harmfulness test on a video frame extracted from a real-time streaming video, there is a problem in that it is difficult to immediately reflect the test result in the image being reproduced due to the time difference between the frame being reproduced on the screen and the inspection frame. For example, an image to be blocked may have already been played and passed.

On the other hand, if a single frame is extracted from an image that is being reproduced at multiple frames per second and a hazard inspection is performed, the inspection time can be reduced by minimizing the load that may occur. In the case where the images are extracted by overlapping, unexpected and unintended test results may occur.

Korean Patent Registration No. 10-0670815 (Jan. 11, 2007) Korean Patent Registration No. 10-0930529 (2009.12.01)

An embodiment of the present invention is to provide an apparatus and method for real-time harmfulness inspection of video content that can check whether a streamed video content is harmful in real time along with the reproduction and immediately reflect the result on a reproduced video.

An embodiment of the present invention determines the number of frames extracted from a real-time streaming video according to the performance of the user terminal and reads the harmfulness test result with respect to the extracted image frames to secure the reliability of the test result. An object of the present invention is to provide an apparatus and method for real-time harmfulness inspection of video content.

An embodiment of the present invention is to provide an apparatus and method for real-time harmfulness testing for video content, capable of predicting and pre-blocking harmfulness of a subsequent image frame through association between harmfulness test result values.

In embodiments, the video content real-time harmfulness inspection apparatus plays a video server providing a real-time streaming video file, and a video file streamed from the video server, extracts a plurality of image frames per second from the video file, checks for harmfulness, and inspects and a user terminal that reads the result and blocks the harmful image of the video being reproduced.

The user terminal may include an image content receiving unit, a harmfulness checking unit configured to start a harmfulness test of the moving image file at a buffering time for playing the moving image file of the image content received from the image content receiving unit, or to inspect the harmfulness of a reproduced image in real time; An inspection result reading unit that reads the inspection result based on the correlation between the inspection result values of the harmfulness inspection unit and determines the harmfulness of the current image of the reproduced image, and a video file of the image content received from the image content receiving unit is reproduced and the It may include an image reproduction processing unit for blocking the harmful image by reflecting the harmfulness determination of the inspection result reading unit on the reproduced image.

The video content receiving unit may receive a real-time streaming video file from the video server in response to a user's request for video content, check a certificate of the received streaming video file, and determine the necessity of a harmfulness check of the video to be played back.

The harmfulness inspection unit extracts at least two or more image frames per second according to the performance of the user terminal and the resolution of the image from the streaming video file, examines the harmfulness of each of the extracted image frames, and frames the harmfulness inspection result. It can be saved with timecode information.

The inspection result reading unit may calculate a correlation between the harmfulness test result of the current image frame with a plurality of immediately preceding image frames and read the harmfulness of the current image frame based on the calculated association.

The inspection result reading unit may calculate the correlation by obtaining an average value of the inspection result value of the current image frame and result values of image frames extracted within a specific time immediately preceding the location of the current image frame and tested for harmfulness.

The inspection result reading unit may apply a weight to the risk in the ratio of safety to risk among the hazard test result value and read the image frame at the boundary of safety or danger as a harmful image.

The inspection result reading unit compares the similarity between the first image frame and the immediately preceding image frame when the deviation between the harmfulness inspection result values of the image frames extracted in the same time period is greater than the preset standard, and determines that the scene change is abrupt when the similarity is less than or equal to the reference value and the harmfulness can be read according to the similarity test result for the current image frame.

Among the embodiments, the method for real-time harmfulness inspection of video content includes receiving a real-time streaming video file from a video server in response to a user's request for video content, and the necessity of checking the harmfulness of the video file by verifying a certificate included in the received streaming video file. determining, when a harmfulness check is required, starting a harmfulness check during streaming of the video file and extracting a plurality of image frames per second from the video file based on the performance of the user terminal and the resolution of the image, the extracted checking the harmfulness of each of the plurality of image frames, and reading the harmfulness of the current image frame by obtaining a correlation between the harmfulness test result of the current image frame and the harmfulness test result of the plurality of image frames within the immediately preceding specific section; and and reflecting the reading result in the corresponding section of the image being reproduced to reproduce normally in the case of a safe image frame, and stop or cover the reproduction in the case of a harmful image frame.

The disclosed technology may have the following effects. However, this does not mean that a specific embodiment should include all of the following effects or only the following effects, so the scope of the disclosed technology should not be construed as being limited thereby.

The apparatus and method for real-time harmfulness inspection of video content according to an embodiment of the present invention can inspect whether or not harmfulness is present in real time together with the reproduction of streamed video content and immediately reflect the result in the image being reproduced, so that harmful images can be obtained at an accurate time. can be blocked or hidden.

A video content real-time harmfulness inspection apparatus and method according to an embodiment of the present invention determines the number of frames extracted from a real-time streaming video according to the performance of a user terminal and reads the extracted image frames in association with the harmfulness inspection result. The reliability of the inspection results can be secured.

The apparatus and method for real-time harmfulness testing for video content according to an embodiment of the present invention can predict and pre-block the harmfulness of a subsequent image frame through association between harmfulness test result values, thereby preventing exposure of harmful scenes in advance.

1 is a diagram for explaining an apparatus for real-time harmfulness detection of video content according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of a user terminal of an apparatus for real-time harmfulness detection of video content according to an embodiment of the present invention.
3 is a flowchart illustrating a method for real-time harmfulness inspection of video content according to an embodiment of the present invention.
4 is a flowchart for explaining a process of a hazard test and a test result reading process according to an exemplary embodiment.

Since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiment described in the text. That is, since the embodiment is capable of various changes and may have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, it should not be understood that the scope of the present invention is limited thereby.

On the other hand, the meaning of the terms described in the present application should be understood as follows.

Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected to the other component, but it should be understood that other components may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" to another element, it should be understood that the other element does not exist in the middle. On the other hand, other expressions describing the relationship between elements, that is, "between" and "between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The singular expression is to be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprises" or "have" refer to the embodied feature, number, step, action, component, part or these It is intended to indicate that a combination exists, and it should be understood that it does not preclude the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

Identifiers (eg, a, b, c, etc.) in each step are used for convenience of description, and the identification code does not describe the order of each step, and each step clearly indicates a specific order in context. Unless otherwise specified, it may occur in a different order from the specified order. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer-readable codes on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. . Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. In addition, the computer-readable recording medium may be distributed in a network-connected computer system, and the computer-readable code may be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Terms defined in the dictionary should be interpreted as being consistent with the meaning of the context of the related art, and cannot be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present application.

1 is a diagram for explaining an apparatus for real-time harmfulness detection of video content according to an embodiment of the present invention.

Referring to FIG. 1 , the video content real-time harmfulness testing apparatus 100 may include a video server 110 and a user terminal 130, and perform video playback and real-time harmfulness check between them, and display the results. It can be implemented to be reflected in an image that is being played immediately.

The video server 110 may be implemented as a computing device capable of transmitting video content to the user terminal 130 through a network. In an embodiment, the video server 110 may store a plurality of video contents in the form of video files in a streaming service platform and provide real-time streaming video data. Here, the video server 110 may be connected to the user terminal 130 through a wired network or a wireless network such as Bluetooth or WiFi.

The user terminal 130 may request image content from the video server 110 and may correspond to a computing device capable of receiving and playing a real-time streaming video file of the requested image content. The user terminal 130 may be implemented as a smart phone, a notebook computer, or a computer, but is not necessarily limited thereto, and may be implemented in various devices such as a tablet PC, IPTV, and the like. The user terminal 130 extracts image frames from the image content to be reproduced for the harmfulness test, and determines the number of image frames per second to be extracted according to the performance and resolution of the user terminal 130 to increase the reliability of the harmfulness test. In one embodiment, the user terminal 130 can minimize the time difference due to the inspection by first starting the harmfulness test during the buffering time for video playback of online streaming, or in real time when there is no buffering, and can display the test result immediately. It can be reflected in the video being played. Here, the user terminal 130 may stop or prohibit the reproduction of the video file according to the harmfulness test result, and block or cover a specific section to prevent exposure of harmful scenes.

2 is a block diagram illustrating a configuration of a user terminal in an apparatus for real-time harmfulness detection of video content according to an embodiment of the present invention.

Referring to FIG. 2 , the user terminal 130 may include an image content receiving unit 210 , a harmfulness inspection unit 230 , an inspection result reading unit 350 , an image reproduction processing unit 270 , and a control unit 290 .

The image content receiver 210 may receive image content from the video server 110 . In an embodiment, the image content receiving unit 210 may receive a real-time streaming video file of the corresponding image content by network connection with the video server 110 in response to a user's request for image content. The video content receiving unit 210 may determine the necessity of checking the harmfulness of the video to be played by checking the certificate of the received streaming video file. The video content receiving unit 210 may determine that if the streaming video file has already been tested as a safe video in the process of posting it to the video server 110 and there is authentication, it is not necessary to check the harmfulness of the video to be played back. The video content receiving unit 210 may determine that the harmfulness test is necessary when the streaming video file is not authenticated, when the authentication is present but reliability is low, when the authentication period has expired, or when the user unconditionally requests the test. In an embodiment, the image content receiving unit 210 may check the reliability of the corresponding image content through metadata included in the streaming video file. Here, the image content receiving unit 210 may check reliability by comparing the addresses of the website on which the image content is uploaded and the streaming website from the metadata, and if the addresses of the two sites match, it may be determined that there is reliability. have.

The received video file may be tested for harmfulness and reproduced by the harmfulness inspection unit 230 and the image reproduction processing unit 270 , respectively.

The harmfulness inspection unit 230 may perform a harmfulness inspection on the real-time streaming video file received by the image content receiving unit 210 . According to the performance of the user terminal 130 and the resolution of the reproduced image, the harmfulness inspection unit 230 extracts at least two or more image frames per second from the reproduced video file to perform the harmful content inspection. .

Frame is the smallest unit of video, and most videos show images at 15 to 30 frames per second. Time code is used to indicate the position from the start point of a specific frame, which is usually expressed in the form of hours:minutes:seconds;frames. For example, if the time code of a specific frame is 00:12:34:15, it means that it is at the position of 12 minutes 34 seconds 15 frames.

In an embodiment, the harmfulness checking unit 230 may extract at least 2 to 29 image frames per second according to the hardware performance of the user terminal 130 and the resolution of the reproduced image. The harmfulness inspection unit 230 may first start the inspection during the buffering time for the reproduction of the real-time streaming video file in the image reproduction processing unit 270 . If the network speed of the user terminal 130 is fast, the video of online streaming may be immediately played, but in general, a playback delay of about 2 to 3 seconds may occur. Here, the harmfulness test unit 230 may first start the examination for the image requiring the harmfulness examination during the buffering time for reproduction. The harmfulness inspection unit 230 may inspect the harmfulness of the image being reproduced in real time when there is no buffering.

The harmfulness check unit 230 may perform a harmfulness check on each of the plurality of extracted images in seconds, and store the check result together with a frame time code. Here, as for the harmfulness test method, the harmfulness test may be performed through an artificial intelligence learning model based on features or contents based on the user's viewable rating, and various methods may be used without being limited thereto. In one embodiment, the harmfulness test unit 250 may store the harmfulness test result in the format [image name]_[hour minute second]_[checked image number]_[result] format. For example, if the frame position of the image called ‘Goblin’ is 1 hour, 23 minutes and 1 second, the first frame among a plurality of frames extracted for inspection at the location, and the result of the hazard test is safe, it can be saved in the “dokebo_012301_1_safe” format. The hazard test unit 230 may convert the test result into a percentile according to a safe level or a harmful level and represent the result as a numerical value.

The inspection result reading unit 250 may read the harmfulness test result, determine the harmfulness of the corresponding image frame, and reflect it on the reproduced image. In one embodiment, the inspection result reading unit 250 reads the result of the harmfulness inspection so that it is possible to play, stop, block, and the like of the corresponding image.

In an embodiment, the test result reading unit 250 may process the harmfulness test result to be reflected in the image of the reproduced image as it is. For example, when three image frames are extracted in seconds and checked for harmfulness, the first image is dangerous, the second image is also dangerous, and the third image is safe. Hides or stops playback for , and resumes normal playback in the third image. Since the image being reproduced by the local image reproduction processing unit 270 can be controlled in units of milliseconds, the screen can be blocked or unblocked by reflecting the result of the hazard test in units of seconds.

In an embodiment, the inspection result reading unit 250 may process harmful content by reading a correlation between a previous inspection result and a subsequent inspection result based on the harmfulness inspection result. In the case of video content, since it is often made up of a sequence, there may be a similarity between the front and rear frames. When the inspection result of the current image frame is different from the inspection result of the previous image frame, the inspection result reading unit 250 compares the average of the inspection result value of the current image frame and the inspection result value of the plurality of immediately preceding image frames to determine the value of the current image frame. The test results can be judged. Here, the inspection result reading unit 250 may use the average of the inspection result values of the image frames within at least 2 seconds immediately before. If the number of image frames from 2 seconds before to the current time is large, the reliability of the association may be secured by limiting the number. Associativity readings can be unreliable in fast-changing scenes, as an image typically needs to be exposed for at least 0.3 seconds.

The test result reading unit 250 may convert the harmfulness test result into a score for each image. For example, as for the hazard test result, a positive number represents “safe” and a negative number represents “risk”, and it can be expressed numerically according to the degree of safety and hazard. The inspection result reading unit 250 may finally determine the inspection result of the current frame by calculating an average of the inspection result of a predetermined number of frames immediately before and the inspection result of the current frame based on the position of the current inspection frame. For example, if the hazard test score of the current image is 54, the hazard test score of the immediately preceding first image is -85, the hazard test score of the second image is -70, and the hazard test score of the third image is -86. The average value of the hazard test score of the four images is -46.75, so even if the hazard test score of the current image is 54, the average value of -46.75 can be judged as dangerous.

In an embodiment, the test result reading unit 250 may adjust the weight of the result by applying a weight to the hazard test result value. Here, the inspection result reading unit 250 may read an image at the boundary of safety or danger as harmful by applying a weight to the risk in the ratio of safety to risk. Weighting is not performed independently, but is dependent on other reading methods such as correlation reading, allowing the intensity of test results to be adjusted.

In one embodiment, the test result reading unit 250 determines that the screen change is abrupt when the deviation between the test result values of the images extracted in the same time period is greater than a preset standard, and in this case, the related test result is temporarily suspended and the first image and the last image to check the similarity. When a new test result comes out from the similarity checker 230 , the test result reading unit 250 may determine a sudden screen change by comparing it with a plurality of previous test results. For example, from 01 second to scene 1, the result is “safe 80”, the result of scene 2 is “Danger -90”, and the result of scene 3 is “danger -85”. It can be predicted as a serious case. In this case, the test result reading unit 250 may temporarily withhold the correlation test result and compare the first scene result and the third scene result to check the similarity between the two images. Here, scene 3 corresponds to an image frame currently being reproduced and being read as a result of the harmfulness test, and scene 1 corresponds to a previous image frame of the image immediately preceding the image currently being reproduced. The inspection result reading unit 250 compares the similarity between the two image frames, and when the similarity is less than a reference value, that is, when the similarity is low, the result is displayed on the screen in the form of each image frame, which is a strict inspection method, without following the correlation test result. You can adjust the results. The test result reading unit 250 may display the correlation test result on the screen when the similarity between the two image frames is equal to or greater than a reference value, that is, when a similar image can be viewed. Here, the similarity comparison between the two image frames may be calculated using brightness information and distortion information for each pixel of two images that are objects of similarity comparison. The inspection result reading unit 250 may prevent inspection errors that may occur due to a sudden scene change by comparing the similarity between the two image frames.

The image reproduction processing unit 270 may reproduce the real-time streaming video file received by the image content receiving unit 210 in a user-viewable form and output the screen. The image reproduction processing unit 270 may display the image on the screen so that the user can view the image after about 2 seconds after the start of the examination by the harmfulness inspection unit 230 . Therefore, the hazard test result can be immediately reflected in the reproduced image. The image reproduction processing unit 270 normally reproduces and displays the image content that has passed the user's harmfulness determination standard based on the reading result of the inspection result reading unit 250, and displays the image content that does not pass the user's harmfulness determination standard. It can be displayed by changing it to other video content. For example, content may be displayed through a process such as blurring harmful images. The image reproduction processing unit 270 may process the image to be covered, stopped, stopped, or reproduced by reflecting the harmfulness test reading result for each image frame of the image being reproduced. In an embodiment, the image reproduction processing unit 250 may block processing in a response method according to a result designated by an administrator according to the content and number of results such as bad/dangerous/adult. Here, the harmful content processing unit 270 may store in the block list, stop playing the corresponding image, restrict blocking and reconnection of the image file and the corresponding web site, and report this series of processes to the administrator. For example, if the video content being played is for children, if the result is that the currently lewd image is played for 2 seconds or more, the video can be stopped and moved to a safe site with a warning message. When the corresponding video content is stored in the user terminal 130, the video playback may be stopped, and the video content may be played back only after approval of the administrator (parent) or passing authentication/re-authentication. The image reproduction processing unit 270 may report the harmfulness test reading result and measures to the designated terminal of the manager or guardian through a notification message or e-mail or the like.

The control unit 290 inspects the harmfulness of the video content subscribed by streaming in the user terminal 130 in real time, reads the test result, and immediately reflects the result in the image being reproduced. It is possible to control overall operations of the harmfulness inspection unit 230 , the inspection result reading unit 250 , and the image reproduction processing unit 270 , and manage control and data flow between them.

3 is a flowchart illustrating a method for real-time harmfulness inspection of video content according to an embodiment of the present invention.

Referring to FIG. 3 , the user may request video content from the video server 110 through the user terminal 130 (step S310 ). Here, the user terminal 130 may be connected to the video server 110 through a network and may correspond to a computing device having an image playback function. The user terminal 130 may request desired image content through a web browser, an app, a smart device, or the like.

In response to a video content request from the user terminal 130 , the video server 110 may provide a video file of the corresponding video content to the user terminal 130 in real-time streaming.

The user terminal 130 may receive a real-time streaming video file and determine whether to check for harmfulness (step S320). In an embodiment, the user terminal 130 may receive a video file in an online state by streaming through the video content receiving unit 210, and check the reliability authentication through a certificate included in the received video file, and according to the confirmation result You can decide whether or not to check the corresponding video file for harmfulness. Here, the image content receiving unit 210 may determine that the harmfulness test is necessary when the reliability of the video file is not authenticated, when the reliability is low, when the authentication period has expired, or when a harmfulness test is requested by the user. .

The user terminal 130 may start image reproduction and harmfulness check of the streaming video file (step S330). In an embodiment, the user terminal 130 may perform image reproduction processing through the image reproduction processing unit 270 , and may perform harmfulness examination of the image being reproduced through the harmfulness inspection unit 230 . Here, the harmfulness test unit 230 may extract a plurality of image frames in seconds according to the hardware performance of the user terminal 130 and the resolution of the image being reproduced and check the harmfulness. The harmfulness test unit 230 may first start the harmfulness test of the video being streamed at the time it is buffered for playback or, if there is no buffering, perform a hazard test in real time and immediately reflect the test result on the image being reproduced. .

The user terminal 130 may read the harmfulness test result of the extracted images to determine the harmfulness of the corresponding image (step S340). In an embodiment, the user terminal 130 determines the harmfulness of the image as it is as the result of the harmfulness examination through the examination result reading unit 250 , the harmfulness examination result of the current image and the harmfulness examination results of the immediately preceding images. The second reading method that determines the harmfulness of the current image based on the correlation between the second reading method and the third reading method that determines the harmfulness by applying a weight to the risk among the harmfulness test results The harmfulness test result may be read through one method set by the user (administrator or guardian) among the fourth reading methods for determining the harmfulness of the current image based on the similarity. Here, the inspection result reading unit 250 may read the harmfulness inspection result using the second reading method using the association as a default, but is not limited thereto.

The user terminal 130 may stop or mask the reproduction of the dangerous image by reflecting the reading result on the image being reproduced (step S350). In one embodiment, the user terminal 130 may stop the reproduction of the dangerous image among the images currently being reproduced through the image reproducing processing unit 270 according to the result of reading the harmfulness of the image content and replay the safe image. . Here, the blocking method for the danger image may be made in various ways. The user terminal 130 may prevent and block the posting of the corresponding image content by automatically reporting the harmfulness reading result to the manager.

4 is a flowchart for explaining a process of checking a hazard and reading a test result according to an exemplary embodiment.

In FIG. 4 , the harmfulness checking unit 230 extracts a plurality of image frames per second from the real-time streaming video data (step S410 ). Here, the number of extracted frames may be determined by the hardware performance of the user terminal 130 and the resolution of the reproduced image. In an embodiment, the harmfulness test unit 230 may extract at least 2 to 29 image frames per second.

The harmfulness checking unit 230 examines the harmfulness of each of the plurality of extracted image frames (step S420). Here, the harmfulness test may extract features or contents of images and apply them to an artificial intelligence-based model to examine the harmfulness. The harmfulness test unit 230 may store the harmfulness test result together with time code information of the corresponding image. In an embodiment, the harmfulness test unit 230 may store the test result in the following format.

var　tmpResult　=　{　result:　result_text,　data:　request.predictions,　curSeconds:　curSeconds,　sceneNum:　sceneNum　};

Here, tmpResult is the temporary storage, result is the inspection result, data is the converted raw data of the inspection result, curSenconds is the location in the video, and sceneNum is the scene number.

The scene number means the sequence number of a plurality of image frames in seconds extracted as the number of the extracted scene within 1 second from the time of the extracted image.

The inspection result reading unit 250 calculates an average of the inspection result values of the current image frame and the previous predetermined number of image frames when the reading method using the association is set as a default (step S430). Here, the inspection result reading unit 250 obtains the sum of the result values of the image frames extracted within 2 seconds immediately before the harmfulness test from the position of the current image frame and the inspection result value of the current image frame and divided by the number of images. The average value can be calculated.

The inspection result reading unit 250 may read the harmfulness of the current image frame based on the calculated average value. For example, even if the harmfulness test result of the current image frame is safe as 54, if the average value with the previous image frames is -24, the inspection result reading unit 250 is inserted into the current image frame in association with the harmfulness test result of the previous image frames. can be read as harmful. In one embodiment, the inspection result reading unit 250 compares the similarity between the first image frame and the last image frame among the immediately preceding image frames when the deviation of the harmfulness inspection result value between the previous image frames is greater than the reference value, When the similarity falls below the threshold, the harmfulness can be read as it is as a result of the harmfulness test of the current image frame instead of calculating an average value by predicting a sudden change of the screen. For example, when a plurality of extracted image frames are sequentially inspected for hazards, the current image frame 3 is 'dangerous', the previous image frame 1 is 'safe', and the image frame 2 is 'dangerous'. , the inspection result reading unit 250 predicts that there is a change in the scene, and when the difference between the inspection result values of the first and second image frames is greater than or equal to a threshold upper limit (eg, 120), it may be determined as an abrupt scene change. The inspection result reading unit 250 compares the similarity between the first and third image frames, that is, the difference, when it is determined that a sudden scene change is made. If it is 59% or less, it is judged that there is a change in the previous scene rather than a new scene, and the result can be read and the intensity of the result can be adjusted.

An apparatus and method for real-time harmfulness inspection of video content according to an embodiment can inspect online obscene video chatting and obscene video services, which have recently become a problem, in real time, and according to the test result, stop or prohibit playback of a video file or a specific section (Example: smoking scenes, scenes with excessive body exposure, sections with scenes that are difficult to watch with children, etc.)

In addition, if necessary, certain parts can be covered or removed from the scene just before a harmful scene (eg, before smoking, excessive violence, or body exposure) occurs, and the test result is immediately displayed on the image being reproduced. By reflecting this, it is possible to prevent errors at the time of blocking and release due to the time difference between inspection and playback.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

100: video content real-time harmfulness inspection device
110: video server 130: user terminal
210: image content receiving unit 230: harmfulness inspection unit
250: test result reading unit 270: image reproduction processing unit
290: control unit

Claims (9)

a video server that provides real-time streaming video files; and
Real-time harmfulness of video content comprising a user terminal that plays a video file streamed from the video server, extracts a plurality of image frames per second from the video file, checks for harmfulness, and reads the test result to block harmful images of the video being reproduced inspection device.
According to claim 1, wherein the user terminal is
video content receiving unit;
a harmfulness inspection unit which starts a harmfulness test of the moving image file or inspects the harmfulness of the reproduced image in real time during a buffering time for the reproduction of the moving image file of the image content received by the image content receiving unit;
an inspection result reading unit which reads the inspection result based on the correlation between the inspection result values of the harmfulness inspection unit and determines the harmfulness of the current image of the reproduced image; and
Real-time harmfulness of video content, characterized in that it includes an image reproduction processing unit that reproduces a video file of the image content received from the image content receiving unit and blocks harmful images by reflecting the harmfulness determination of the inspection result reading unit on the reproduced image. inspection device.
According to claim 2, wherein the image content receiving unit
Real-time harmfulness inspection apparatus for video content, characterized in that receiving a real-time streaming video file from the video server according to a user's request for video content, and determining the necessity of a harmfulness test of a video to be played back by checking a certificate of the received streaming video file .
The method of claim 2, wherein the hazard inspection unit
From the streaming video file, at least two or more image frames are extracted per second according to the performance of the user terminal and the resolution of the image, and harmfulness is checked for each of the extracted image frames, and the harmfulness test result is combined with frame time code information and Video content real-time harmfulness inspection device, characterized in that it is stored together.
The method of claim 2, wherein the inspection result reading unit
A real-time harmfulness inspection apparatus for video content, characterized in that the harmfulness test result of the current image frame is calculated as a correlation with a plurality of immediately preceding image frames, and the harmfulness of the current image frame is read based on the calculated association.
The method of claim 5, wherein the inspection result reading unit
Image content real-time harmfulness inspection apparatus, characterized in that the correlation is calculated by obtaining an average value of the inspection result value of the current image frame and result values of image frames extracted within a specific time immediately before from the position of the current image frame and inspected for harmfulness.
The method of claim 5, wherein the inspection result reading unit
Image content real-time harmfulness inspection apparatus, characterized in that by applying a weight to the risk in the ratio of safety to risk among the results of the harmfulness test, an image frame at the boundary of safety or danger is read as a harmful image.
The method of claim 5, wherein the inspection result reading unit
If the deviation between the hazard test results of the image frames extracted in the same time period is greater than the preset standard, the similarity between the first image frame and the previous image frame is compared. Image content real-time hazard inspection device, characterized in that it reads the hazard according to the similarity test result.
receiving a real-time streaming video file from a video server according to a user's request for video content;
determining the necessity of checking the harmfulness of the video file through verification of a certificate included in the received streaming video file;
when a harmfulness check is required, starting a harmfulness check during streaming of the video file and extracting a plurality of image frames per second from the video file based on the performance of the user terminal and the resolution of the video;
examining the harmfulness of each of the plurality of extracted image frames;
reading the harmfulness of the current image frame by obtaining a correlation between the harmfulness test result of the current image frame and the harmfulness test result of a plurality of image frames within the immediately preceding specific section; and
and reflecting the reading result in a corresponding section of the image being reproduced to reproduce normally in the case of a safe image frame and stop or cover the reproduction in case of a harmful image frame.

Images