KR102432824B1 - Method for Detecting Harmful Streaming Video in Smart Device, Smart Device and Terminal for Machine Learning - Google Patents

Abstract

The embodiment discloses a harmful streaming image detection method in a smart device that plays a streaming image received from a streaming server. A harmful streaming image detection method in a smart device includes the steps of sequentially extracting image frames from streaming images, examining whether the extracted image frames are harmful based on a pre-trained neural network model, and depending on whether the image frames are harmful and selectively blocking playback of the streaming video.

Description

Method for Detecting Harmful Streaming Video in Smart Device, Smart Device and Terminal for Machine Learning

The present invention relates to a technique for inspecting whether an image is harmful.

With the spread of smart devices and the development of high-speed mobile communication technology, it has become possible to provide real-time video streaming services anytime, anywhere. Accordingly, the media consumption environment is evolving into an environment that uses services such as VoD, real-time personal broadcasting, and video-based SNS using smart devices as well as PCs.

However, such a change in the media service usage environment has the advantage of providing diversity and convenience in media consumption, while also diversifying channels for distributing harmful substances including pornography and violence. That is, as the real-time production and transmission of harmful substances is possible through these various channels, it is becoming more and more difficult to prevent the distribution of illegal substances and the use of juveniles.

Conventionally, as a method for blocking harmful Internet substances, a technology for blocking harmful websites and a keyword-based blocking technology for filtering an image search containing a forbidden word have been developed. However, it is reported that it is possible to distribute and use harmful substances by bypassing the conventional blocking technology, and the error rate of blocking is also high.

In order to compensate for the shortcomings of the prior art, hash list-based blocking technology, feature-based blocking technology, and content-based blocking technology for identifying harmful substances based on the contents of a video have been developed.

Among them, the hash list-based blocking technology is a method of converting summary information (hash) generated from harmful substances into a DB, calculating summary information (hash) of an image to be identified in the future, and comparing it with data stored in the DB. However, in this method, continuous updating of the DB is required, and when the image is deformed, the summary information is also different, making it difficult to identify.

The feature-based blocking technology extracts characteristic information of harmful images called DNA to build a DB, and detects harmful substances by comparing the DNA and DB of images to be identified in the future. This feature-based blocking technology has the advantage of being able to detect even some modified harmful substances for harmful substances included in the DB list. It is not easy to respond to personal broadcasting.

Moreover, because it is practically impossible to build a DB for all images due to the explosion of user uploads and real-time personal broadcasting images, streaming service providers employ monitoring personnel to detect harmful images. However, this method is not preferable in view of the low detection efficiency, mental health problems of monitoring personnel, and high cost of hiring personnel.

Korean Patent No. 10-1237469

The embodiment aims to solve the problems of mental health and high cost of monitoring personnel due to manually examining the harmfulness of streaming video.

The embodiment has an object to enable real-time harmfulness inspection for an explosion of streaming video.

The purpose of the embodiment is to directly inspect a smart device that receives a streaming video without the cost of constructing and operating a separate detection device for inspecting the harmfulness of the streaming video.

An object of the embodiment is to enable a smart device to inspect the harmfulness of a streaming video in real time without degrading the service quality of the streaming video.

A method for detecting harmful streaming images in a smart device that reproduces a streaming image received from a streaming server according to an embodiment comprises the steps of sequentially extracting image frames from a streaming image, and a neural network trained in advance to determine whether the extracted image frames are harmful. It includes the step of checking based on the model and the step of selectively blocking the playback of the streaming video according to whether the image frame is harmful.

In this case, the neural network model is generated by the machine learning terminal and may be downloaded in advance from the machine learning terminal.

At this time, the machine learning terminal generates a first neural network model trained with training data including a harmful image and a non-toxic image, and generates a second neural network model by lightening the generated first neural network model, the neural network model, It may be a second neural network model.

In this case, the neural network model may be an image classification neural network based on a convolutional neural network (CNN).

The method for detecting harmful streaming video in a smart device according to an embodiment further includes, after the step of checking, determining the harmfulness of the streaming image based on the number of image frames in which the harmfulness exists, and the blocking step includes: It may be performed based on the result of determining the harmfulness of the streaming video.

In this case, the method for detecting a harmful streaming image in a smart device according to an embodiment may further include displaying a warning message based on a result of determining the harmfulness of the streaming image.

In this case, the method for detecting a harmful streaming image in a smart device according to an embodiment may further include displaying an alternative image based on a result of determining the harmfulness of the streaming image.

Meanwhile, the extracting may include extracting an image frame at a predetermined extraction cycle, and the extraction cycle may be changed in real time according to at least one of whether the image frame is harmful, CPU occupancy, and remaining battery power.

In this case, the extraction period may be adjusted to be smaller than before when the image frame is checked as harmful.

In this case, the extraction period may be longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery capacity is less than or equal to a predetermined threshold.

A machine learning terminal according to an embodiment includes a memory in which at least one program is recorded, a network interface and a processor for executing the program, wherein the program acquires a harmful image frame and a non-harmful image frame, and the obtained image frame and Generating training data labeled with non-harmful image frames, constructing a neural network, optimizing parameters by training a first neural network model with training data, and lightweighting the optimized first neural network model to generate a second neural network model have.

In this case, the program may download the second neural network model according to a request from at least one smart device through the network interface.

A smart device according to an embodiment includes a memory in which at least one program and a pre-trained neural network model are recorded, a network interface for receiving a streaming image from a streaming server through a network, an interface output unit and a program through which the streaming image is reproduced It includes a processor that executes, wherein the program sequentially extracts image frames from the streaming image, checks whether the extracted image frame is harmful based on a pre-trained neural network model, and selectively streams the image frame according to whether the image frame is harmful may block playback.

In this case, the neural network model may be a second neural network model in which the first neural network model learned from training data including the harmful image and the non-toxic image is lightened.

In this case, the neural network model may be an image classification neural network based on a convolutional neural network (CNN).

Meanwhile, the program may determine the harmfulness of the streaming image based on the number of image frames in which the harmfulness exists and block playback of the streaming image based on the result of determining the harmfulness of the streaming image.

In this case, the program may express a warning message based on the result of determining the harmfulness of the streaming video.

In this case, the program may express an alternative image based on the result of determining the harmfulness of the streaming image.

On the other hand, when the program sequentially extracts the image frames, the image frames are extracted at a predetermined extraction cycle, but the extraction cycle may be adjusted to be smaller than before when the image frame is checked to be harmful.

In this case, the extraction period may be longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery capacity is less than or equal to a predetermined threshold.

According to an embodiment, by examining the harmfulness of streaming video based on artificial intelligence, it is possible to solve the problems of mental health and high cost of monitoring personnel.

According to the embodiment, it is possible to enable real-time harmfulness inspection of the explosive streaming video.

According to an embodiment, the smart device receiving the streaming video may directly test it without the cost of constructing and operating a separate detection device for testing the harmfulness of the streaming video.

According to an embodiment, the smart device may inspect the harmfulness of the streaming video in real time without degrading the service quality of the streaming video. That is, by adjusting the image extraction cycle according to the processing performance and battery state of the smart device, there is no interruption in service and additional battery consumption due to the harmfulness test can be reduced.

1 is a schematic block diagram of a system for detecting harmful streaming images according to an embodiment.
2 is a flowchart illustrating a method for detecting harmful streaming video in a smart device according to an embodiment.
3 is a flowchart for explaining the harmfulness determination step ( S330 ) of the streaming video shown in FIG. 2 .
4 is a flowchart for explaining an operation of adjusting an extraction period according to whether harmfulness is present according to an embodiment.
5 is an exemplary diagram for explaining image frame extraction according to an extraction cycle according to an embodiment.
6 is a flowchart illustrating an operation of adjusting an extraction period according to a remaining battery amount according to an embodiment.
7 is a flowchart illustrating an operation of adjusting an extraction period according to CPU occupancy according to an embodiment.
8 is a block diagram illustrating a computer system according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the spirit of the present invention.

The terminology used herein is for the purpose of describing the embodiment and is not intended to limit the present invention. In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, “comprises” or “comprising” implies that the stated component or step does not exclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted with meanings commonly understood by those of ordinary skill in the art to which the present invention pertains. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, a method for detecting harmful streaming video in a smart device, a smart device, and a machine learning terminal according to an embodiment will be described in detail with reference to FIGS. 1 to 8 .

1 is a schematic block diagram of a system for detecting harmful streaming images according to an embodiment.

Referring to FIG. 1 , in the harmful streaming image detection system, the streaming server 10 , the machine learning terminal 100 , and the smart device 200 are linked through a wired/wireless network.

The streaming server 10 provides a video streaming service to at least one smart device 200 .

The learning engine 110 of the machine learning terminal 100 generates the first neural network model 120 trained with training data including harmful images and non-harmful images.

That is, the learning engine 110 obtains an error by substituting the value output from the first neural network model 120 and the labeled correct answer value into a loss function or a cost function. Thereafter, the training parameter is updated through a backpropagation algorithm in a direction to minimize the cost function of the learning engine 110 .

In this case, the neural network may be an image classification neural network based on a convolutional neural network (CNN). As an example, it may be a mobile (MobileNet) designed for a mobile terminal. However, this is only an example, and the present invention is not limited thereto. That is, the first neural network model 120 may be implemented with various learning algorithms.

In this case, the machine learning terminal 100 further includes a lightweight model conversion unit 130 , which lightens the generated first neural network model 120 to generate a second neural network model 220 . do. Through this, the file size of the second neural network model 220 may be smaller than that of the first neural network model 120 , and the detection speed may be increased. Here, the lightweight model converter 130 may be a Tensor Flow (TF) Lite Converter.

Here, a brief description of the lightweight algorithm may include a model storage format change step, a quantization step, and a pruning step. In this case, in the step of changing the model storage format, the model storage format is changed to efficiently operate in limited computing resources and memory resources. For example, the model storage format may be Flatbuffer.

At this time, in the quantization step, the precision of the numbers used to represent the parameters of the model is reduced. That is, it is possible to reduce the size of the model and improve the detection speed at the cost of decreasing accuracy.

In this case, in the pruning step, it may be advantageous to reduce the size of the model by removing those having a low contribution to inference among parameters of the model. Meanwhile, the machine learning terminal 100 downloads the second neural network model 220 according to a request from at least one smart device 200 through a network interface.

In this case, the machine learning terminal 100 may be operated by a service provider that provides a streaming service to the smart device 200 through the streaming server 10 . Accordingly, the second neural network model 220 may be installed in a streaming service App provided by a service provider and provided to the smart device 200 .

The smart device 200 may be a device capable of playing a streaming image received from the streaming server 10 .

According to an embodiment, the smart device 200 may include an image extraction unit 210 , a neural network model 220 , and a service control unit 240 . Additionally, it may further include an inference unit 230 .

The image extraction unit 210 sequentially extracts image frames from the streaming video. In this case, the image extraction unit 210 may extract image frames at a predetermined extraction cycle, rather than extracting all image frames of the streaming video. In this case, the extraction period of the image frame may be a predetermined period or may be variable. A detailed description thereof will be described later with reference to FIGS. 4 to 7 .

The neural network model 220 checks whether the extracted image frame is harmful. In this case, the harmfulness test model 220 may be a second neural network model 220 in which the first neural network model 120 learned from training data including harmful images and non-toxic images is lightweight.

The service control unit 240 selectively blocks playback of the streaming image according to whether the image frame is harmful.

The inference unit 230 may be located between the harmfulness test model 220 and the service control unit 240 to determine the harmfulness of the streaming video based on the number of image frames in which the harmfulness exists. Accordingly, it is possible to reduce a recognition error through the neural network model 10 . A detailed description thereof will be described later with reference to FIG. 3 .

In this case, the service control unit 240 may block playback of the streaming image based on the result of determining the harmfulness of the streaming image by the inference unit 230 .

In this case, the service control unit 240 may display a warning message based on the result of determining the harmfulness of the streaming video. For example, when it is determined that the streaming video is harmful, the service control unit 240 does not automatically block the streaming video, but informs that the streaming video is harmful so that the smart device user can stop the streaming video by themselves. Alternatively, when an adult user watches a streaming video with their children, measures may be taken to prevent exposure to the children. Alternatively, although the streaming video has not yet been determined to be harmful, when the number of image frames in which the harmfulness exists approaches the threshold, the service control unit 240 may output a message indicating that the streaming video is highly harmful in advance. .

In this case, the service control unit 240 may display an alternative image based on the result of determining the harmfulness of the streaming image.

In this case, the operation of reproducing the streaming image and the operation of inferring the harmfulness of the streaming image based on the neural network model may be performed in parallel and at the same time.

That is, if the streaming video is sequentially reproduced after inferring the harmfulness of the streaming video based on the neural network model, the streaming service may be delayed. Therefore, in the embodiment, by extracting an image frame from the streaming image separately while playing the streaming image, whether harmfulness is inferred based on the neural network model, blocking or replacing the playing streaming image according to the harmfulness inference result, or the streaming image is A warning message can be output during playback.

In this case, the operation of deducing the harmfulness of the streaming video operates at the same time with other modules, for example, the playback module, using a thread, GPU, etc., and when the inference is finished, it notifies whether the inference is terminated so that the system can utilize the inference result. Meanwhile, as described above, when the image extraction unit 210 sequentially extracts image frames, the extraction period may be varied. To this end, the inference unit 230 further includes a scheduler 231 , calculates the extraction period in real time, and feeds it back to the image extraction unit 210 .

In this case, the extraction period may be changed in real time according to at least one of whether the image frame is harmful, CPU occupancy, and battery remaining.

That is, the extraction period can be adjusted to be smaller than before when the image frame is checked as harmful. That is, the image frame is extracted with a short cycle only when the possibility of the harmful streaming image is high, and when the possibility of the harmful streaming image is small, the image frame is extracted with a long cycle, so that the neural network model 220 and the inference unit 230 are Unnecessary load can be reduced.

In this case, the extraction period may be longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery capacity is less than or equal to a predetermined threshold. That is, this is to prevent deterioration of the service quality of the streaming video due to CPU occupancy and battery consumption due to the harmfulness test of the streaming video of the smart device 200 .

2 is a flowchart illustrating a method for detecting harmful streaming video in a smart device according to an embodiment. Here, the content overlapping with the above-described content will be omitted.

Referring to FIG. 2 , the method for detecting harmful streaming images in a smart device according to an embodiment includes the steps of sequentially extracting image frames from a streaming image ( S310 ), and a neural network model pre-trained to determine whether the extracted image frames are harmful. It includes a step of checking based on ( S320 ) and a step of selectively blocking playback of a streaming video according to whether the image frame is harmful ( S340 ).

At this time, the harmful streaming image detection method in the smart device according to the embodiment further includes a step (S330) of determining the harmfulness of the streaming image based on the number of image frames in which the harmfulness exists after the step of checking (S320) may include A detailed description thereof will be described later with reference to FIG. 3 .

In this case, the blocking step ( S340 ) may be performed based on the result of determining the harmfulness of the streaming video. In addition, although not shown in the drawings, the method may further include displaying a warning message based on the result of determining the harmfulness of the streaming video. In addition, although not shown in the drawings, the method may further include displaying an alternative image based on the result of determining the harmfulness of the streaming image.

Meanwhile, in the extraction step ( S310 ), the image frame may be extracted at a predetermined extraction cycle. In this case, the extraction period may be changed in real time according to at least one of whether the image frame is harmful, CPU occupancy, and battery remaining. In this case, the extraction period may be adjusted to be smaller than before when the image frame is checked as harmful. In this case, the extraction period may be longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery capacity is less than or equal to a predetermined threshold. A detailed description thereof will be described later with reference to FIGS. 4 to 7 .

3 is a flowchart for explaining the harmfulness determination step ( S330 ) of the streaming video shown in FIG. 2 .

Referring to FIG. 3 , as the harmfulness test result of the image frame is output, the apparatus 200 increases the number K of the analyzed consecutive image frames by '1' (S331).

Then, the device 200 checks whether harmfulness exists in the K-th image frame (S332).

If there is no harmfulness in the image frame as a result of the check in S332, the device 200 determines whether the number (K) of the image frame to be inspected has reached a predetermined number (N) in advance (S333), and selects according to the determination result After initializing (S334) the number (K) of the image frame to be inspected, the process proceeds to step S310. That is, the harmfulness of the image frame is determined every predetermined number (N).

On the other hand, if harmfulness exists in the image frame as a result of checking S332, the device 200 increases the number (M) of image frames in which the harmfulness exists by '1' (S335).

Then, the apparatus 200 determines whether the number (K) of the image frame to be examined has reached a predetermined number (N) in advance (S336).

If it is determined in S336 that the number (K) of the image frames to be inspected is less than the predetermined number (N), the apparatus 200 performs S310 again.

On the other hand, if it is determined in S336 that the number (K) of image frames to be inspected is a predetermined number (N), the apparatus 200 determines that the number (M) of image frames in which harmfulness exists is equal to or greater than the threshold ( th _absc ). It is determined whether it is recognized (S337).

If it is determined in S337 that the number M of harmful image frames is equal to or greater than the threshold th _absc , the device 200 determines that the streaming video is harmful (S338).

On the other hand, if it is determined in S337 that the number M of harmful image frames is not equal to or greater than the threshold th _absc value, the device 200 initializes the number M of harmful image frames and the number K of target image frames. After (S339), the process proceeds to S310.

4 is a flowchart for explaining an operation of adjusting an extraction period according to whether harmfulness is present according to an embodiment, and FIG. 5 is an exemplary diagram for explaining image frame extraction according to an extraction period according to an embodiment.

Referring to FIG. 4 , the scheduler 231 determines whether a harmful image frame is generated ( S411 ).

As a result of the determination in S411, if the image frame is harmful, the scheduler 231 reduces the extraction period (S412). That is, if it is checked that the harmfulness of the image frame is present, the streaming image is highly likely to be harmful. As shown in FIG. 5 , it is necessary to extract the image frame with a short period T1 to increase the detection speed.

On the other hand, if it is determined in S411 that there is no harmfulness of the image frame, the scheduler 231 reduces the extraction period (S413). That is, if there is no harmfulness of the image frame, there is a high possibility that the streaming image is not harmful. As shown in FIG. 5 , the detection speed should be reduced by extracting the image frame with a long period T2. That is, it is possible to prevent deterioration of the quality of the streaming service by reducing the CPU load and battery consumption of the smart device due to unnecessary inference.

6 is a flowchart illustrating an operation of adjusting an extraction period according to a remaining battery amount according to an embodiment.

Referring to FIG. 6 , the scheduler 231 checks whether the battery level of the smart device is sufficient ( S421 ).

If it is determined in S421 that the remaining battery power of the smart device is sufficient, the scheduler 231 decreases the extraction period (S422). On the other hand, if it is determined in S411 that the remaining battery power of the smart device is insufficient, the scheduler 231 increases the extraction period (S213). That is, when the remaining battery power is insufficient, this is to prevent the quality of the streaming service from being deteriorated due to battery consumption for inspection of harmful substances in the streaming video.

7 is a flowchart illustrating an operation of adjusting an extraction period according to CPU occupancy according to an embodiment.

Referring to FIG. 7 , the scheduler 231 determines whether the CPU occupancy is equal to or greater than a predetermined threshold ( S431 ).

If the CPU occupancy of the smart device is sufficient as a result of the determination in S431, the scheduler 231 reduces the extraction period (S432). On the other hand, if it is determined in S431 that the remaining battery power of the smart device is insufficient, the scheduler 231 reduces the extraction period (S433).

The configuration and operation of the machine learning terminal 100 and the smart device 200 described above may be implemented by the computer system shown in FIG. 8 . As described above, the learning apparatus 100 and the automatic notation generating apparatus 200 may be implemented on one computer system, and may be implemented as a separate computer system to be able to communicate with each other through a network interface 1060 to be described later. may be

8 is a block diagram illustrating a computer system according to an embodiment of the present invention.

Referring to FIG. 8 , an embodiment of the present invention may be implemented in a computer system such as a computer-readable recording medium. As shown in FIG. 8 , the computer system 1000 includes one or more processors 1010 , a memory 1030 , a user interface input device 1040 , and a user interface output device 1050 that communicate with each other via a bus 1020 . and storage 1060 . In addition, computer system 1000 may further include a network interface 1070 coupled to network 1080 . The processor 1010 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1030 or the storage 1060 . The memory 1030 and the storage 1060 may be various types of volatile or non-volatile storage media. For example, the memory 1030 may include a ROM 1031 or a RAM 1032 .

As described above, in the method for detecting harmful streaming video in a smart device, the smart device and the machine learning terminal according to the present invention, the configuration and method of the embodiments described above are not limitedly applicable, but the embodiments are All or part of each of the embodiments may be selectively combined and configured so that various modifications may be made.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: Streaming Server
100: machine learning terminal 110: learning engine
120: hazard test model 130: lightweight model conversion unit
200: smart device 210: image extraction unit
220: hazard test model 230: reasoning unit
230: scheduler 240: service control unit

Claims (20)

In the method for detecting harmful streaming video in a smart device that plays a streaming video received from a streaming server,
sequentially extracting image frames from the streaming video;
examining whether the extracted image frame is harmful based on a pre-trained neural network model; and
Including the step of selectively blocking the playback of the streaming video according to whether the image frame is harmful,
The neural network model is
The first neural network model generated by the external machine learning terminal is a lightweight second neural network model,
The extraction step is
Extracting an image frame at a predetermined extraction cycle,
The extraction cycle is
changed in real time according to at least one of whether the image frame is harmful, CPU occupancy, and remaining battery power;
A method for detecting harmful streaming video in a smart device. The method of claim 1, wherein the neural network model comprises:
As generated by a machine learning terminal,
A method of detecting harmful streaming video in a smart device, which is downloaded in advance from a machine learning terminal. The method of claim 2, wherein the machine learning terminal comprises:
generating a first neural network model trained with training data including harmful images and non-harmful images;
Create a second neural network model by lightening the generated first neural network model,
The neural network model is
A second neural network model, a method for detecting harmful streaming images in a smart device. The method of claim 1 , wherein the neural network model comprises:
Convolutional Neural Network (CNN)-based image classification neural network algorithm that is generated based on, a harmful streaming image detection method in a smart device. The method of claim 1,
After the step of checking, further comprising the step of determining the harmfulness of the streaming image based on the number of image frames in which the harmfulness exists,
The steps to block
A method for detecting harmful streaming images in a smart device, which is performed based on the result of determining the harmfulness of the streaming image. 6. The method of claim 5,
The method for detecting harmful streaming video in a smart device further comprising the step of displaying a warning message based on the result of determining the harmfulness of the streaming video. 6. The method of claim 5,
A method for detecting harmful streaming images in a smart device, further comprising the step of expressing an alternative image based on the harmfulness determination result of the streaming image. delete The method of claim 1, wherein the extraction period is
When the image frame is inspected as harmful, the method for detecting harmful streaming video in a smart device is adjusted to be smaller than before. The method of claim 1, wherein the extraction period is
A method of detecting harmful streaming images in a smart device that takes longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery is less than or equal to a predetermined threshold. a memory in which at least one program is recorded;
network interface; and
a processor for executing a program;
the program
Generate a first neural network model trained with training data including harmful images and non-toxic images, and lighten the generated first neural network model to generate a second neural network model,
In order to lighten the neural network model,
changing the model storage format;
a quantization step that reduces the precision of numbers representing parameters of the model; and
A machine learning terminal that performs a pruning step of selecting and removing model parameters according to the contribution of inference. 12. The method of claim 11, wherein the program comprises:
A machine learning terminal for downloading a second neural network model according to a request from at least one smart device via a network interface. a memory in which at least one program and a pre-trained neural network model are recorded;
a network interface for receiving a streaming image from a streaming server through a network;
an interface output unit for playing a streaming video; and
a processor for executing a program;
the program
sequentially extracting image frames from streaming video,
Inspects whether the extracted image frame is harmful based on a pre-trained neural network model,
Selectively blocks the playback of streaming video according to the harmfulness of the image frame,
The neural network model is
The first neural network model generated by the external machine learning terminal is a lightweight second neural network model,
program,
Extracting an image frame at a predetermined extraction cycle,
The extraction cycle is
A smart device that changes in real time according to at least one of whether the image frame is harmful, CPU occupancy, and remaining battery power. The method of claim 13, wherein the neural network model comprises:
A smart device, wherein the first neural network model trained with training data including harmful images and non-harmful images is a lightweight second neural network model. The method of claim 13, wherein the neural network model comprises:
A smart device, an image classification neural network based on a convolutional neural network (CNN). 14. The method of claim 13, wherein the program
Determining the harmfulness of the streaming video based on the number of image frames in which the harmfulness exists,
In blocking the playback of the streaming video, a smart device that is performed based on the result of determining the harmfulness of the streaming video. 17. The method of claim 16, wherein the program comprises:
A smart device that displays a warning message based on the harmfulness judgment result of the streaming video. 17. The method of claim 16, wherein the program comprises:
A smart device that displays an alternative video based on the harmfulness judgment result of the streaming video. 14. The method of claim 13, wherein the program
In sequentially extracting image frames, the image frames are extracted at a predetermined extraction cycle,
The extraction cycle is
A smart device that is scaled smaller than before if the image frame is checked as harmful. 14. The method of claim 13, wherein the extraction period is:
A smart device that takes longer than before when the CPU occupancy is greater than or equal to a predetermined threshold or when the remaining battery level is less than or equal to a predetermined threshold.

Images