KR102453919B1 - Method, device and system for verifying of guide soundtrack related to cultural content based on artificial intelligence - Google Patents

Abstract

The present invention provides a method for verifying a guide sound source related to cultural content based on artificial intelligence which can secure the accuracy of the content of a guide sound source and manage the quality of audio files. According to one embodiment of the present invention, the method for verifying a guide sound source related to cultural content based on artificial intelligence is performed by a device and comprises: a step of receiving a first audio file and first metadata from a first user terminal; a step of inspecting the quality of the first audio file; a step of extracting sound information for each section through sampling from the first audio file if it is verified that the first audio file does not have problems as a result of inspecting the quality of the first audio file; a step of checking keywords through speech recognition in sound information extracted for each section, and generating a first list which is a keyword list related to the first audio file based on the keywords checked for each section; a step of generating a first input signal based on the first metadata; a step of inputting the first input signal into a pre-trained first artificial neural network to acquire a first output signal; a step of generating a second list which is a keyword list related to the first metadata based on the first output signal; a step of combining the first list and the second list to generate a third list; a step of generating a second input signal based on the third list; a step of inputting the second input signal into a pre-trained second artificial neural network to acquire a second output signal; and a step of generating verification results for the first audio file based on the second output signal.

Description

AI-based cultural content-related guide Sound source verification method, device and system {METHOD, DEVICE AND SYSTEM FOR VERIFYING OF GUIDE SOUNDTRACK RELATED TO CULTURAL CONTENT BASED ON ARTIFICIAL INTELLIGENCE}

The following embodiments relate to a technology for verifying a guide sound source related to cultural content based on artificial intelligence.

Recently, the field of activity has expanded to the form of actively creating and distributing content beyond the level of users collecting and editing information. Accordingly, there are attempts by users to create and share guide sound sources related to cultural contents such as history, art, and art.

However, there are many cases where there is a problem with the audio file uploaded by the user, so quality control for the audio file is required, and since the guide sound source produced by the user may contain untrue content, A verification process is also required.

Therefore, there is an increasing demand for automatically verifying the audio file, which is a guide sound source, to ensure the accuracy of the content of the guide sound source, and to manage the quality of the audio file, and research on related technologies is required. do.

Korean Patent Registration No. 10-2014639 Korean Patent Registration No. 10-1253225 Korean Patent No. 10-1041039 Korean Patent Registration No. 10-0976082

According to an embodiment, the first audio file and the first metadata are received from the first user terminal, the quality of the first audio file is checked, and the quality of the first audio file is checked as a result of the first audio file. When it is confirmed that there is no problem in the file, sound information is extracted for each zone through sampling from the first audio file, and keywords are identified through voice recognition from the sound information extracted for each zone, and based on the keywords identified for each zone, A first list that is a list of keywords related to a first audio file is generated, a first input signal is generated based on the first metadata, and the first input signal is input to a pre-trained first artificial neural network for a first output obtaining a signal, and based on the first output signal, generating a second list that is a list of keywords related to the first meta data, combining the first list and the second list to generate a third list, and a third list generates a second input signal based on the The purpose of this is to provide a verification method, apparatus and system for a guide sound source related to artificial intelligence-based cultural content that generates a result.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

According to an embodiment, there is provided a method of verifying a guide sound source related to cultural content based on artificial intelligence, performed by a device, the method comprising: receiving a first audio file and first metadata from a first user terminal; checking the quality of the first audio file; extracting sound information for each zone through sampling from the first audio file when it is confirmed that there is no problem in the first audio file as a result of checking the quality of the first audio file; identifying a keyword from the sound information extracted for each zone through voice recognition, and generating a first list that is a keyword list related to the first audio file based on the identified keyword for each zone; generating a first input signal based on the first metadata; obtaining a first output signal by inputting the first input signal to a pre-trained first artificial neural network; generating a second list that is a keyword list related to the first meta data based on the first output signal; generating a third list by combining the first list and the second list; generating a second input signal based on the third list; obtaining a second output signal by inputting the second input signal to a pre-trained second artificial neural network; and based on the second output signal, generating a verification result for the first audio file.

The second artificial neural network determines whether each keyword included in the first list is included in the second list based on the second input signal, and adds the keyword included in the first list to the second list. It is divided into a 1-1 list confirmed to be included and a 1-2 list confirmed not included in the second list, and the number of keywords included in the 1-1 list is included in the first list A matching rate between the first list and the second list is calculated through a value divided by the number of keywords, and when it is confirmed that the matching rate is greater than a preset reference value, the verification result for the first audio file is verified successfully is analyzed to output the second output signal, and when it is confirmed that the matching rate is smaller than the reference value, the verification result of the first audio file may be analyzed as verification failure and the second output signal may be output.

In the method for verifying the artificial intelligence-based cultural content-related guide sound source, after generating the second list, the first audio file is analyzed as an audio guide of a first exhibition object based on a keyword included in the first list. to do; obtaining a fourth list, which is a list of key keywords set in the first exhibit; and adding the fourth list to the second list.

The second artificial neural network compares the keywords included in the first list with the keywords included in the fourth list based on the second input signal, checks whether they match by keyword, and adds the keywords to the fourth list. When it is confirmed that the included keywords all match the keywords included in the first list, the verification result for the first audio file is analyzed as verification success, the second output signal is output, and the second output signal is included in the fourth list If it is determined that one or more of the keywords do not match the keywords included in the first list, the verification result of the first audio file may be analyzed as verification failure, and the second output signal may be output.

According to an embodiment, the first audio file and the first metadata are received from the first user terminal, the quality of the first audio file is checked, and the quality of the first audio file is checked as a result of the first audio file. When it is confirmed that there is no problem in the file, sound information is extracted for each zone through sampling from the first audio file, and keywords are identified through voice recognition from the sound information extracted for each zone, and based on the keywords identified for each zone, A first list that is a list of keywords related to a first audio file is generated, a first input signal is generated based on the first metadata, and the first input signal is input to a pre-trained first artificial neural network for a first output obtaining a signal, and based on the first output signal, generating a second list that is a list of keywords related to the first meta data, combining the first list and the second list to generate a third list, and a third list generates a second input signal based on the By generating the result, it is possible to automatically verify the audio file, which is the guide sound source, to ensure the accuracy of the content of the guide sound source, and to manage the quality of the audio file.

On the other hand, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is a diagram schematically showing the configuration of a system according to an embodiment.
2 is a diagram schematically showing the configuration of an apparatus according to an embodiment.
3 is a flowchart illustrating a process of verifying a guide sound source related to cultural content based on artificial intelligence according to an embodiment.
4 is a flowchart illustrating a process of verifying an audio file according to a keyword matching degree according to an exemplary embodiment.
5 is a flowchart illustrating a process of verifying an audio file according to whether a key keyword is included, according to an exemplary embodiment.
6 is a flowchart illustrating a process of extracting and determining sound information for each zone according to an exemplary embodiment.
7 is a flowchart illustrating a process of controlling policy settings of a host firewall based on artificial intelligence according to an embodiment.
8 is a flowchart illustrating a process of detecting an approach estimated as an attack and providing a countermeasure according to the number of attacks, according to an exemplary embodiment.
9 is a diagram for explaining learning of an artificial neural network according to an embodiment.
10 is a diagram for explaining learning of an artificial neural network according to another embodiment.
11 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

Terms used in the examples are used for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

The embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like.

In an embodiment, the artificial intelligence (AI) system is a computer system that implements human-level intelligence, and is a system in which a machine learns and makes decisions on its own, unlike an existing rule-based smart system. The more the AI system is used, the better the recognition rate and the more accurate understanding of user preferences, and the existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithm technology that categorizes/learns characteristics of input data by itself, and element technology uses machine learning algorithms such as deep learning to simulate functions such as cognition and judgment of the human brain. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technology for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

In general, in order to apply the machine learning algorithm to real life, learning is performed in the Trial and Error method due to the characteristics of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in the actual physical external environment, so instead, the actual physical external environment is implemented on a computer and learning is performed through simulation.

1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1 , a system according to an embodiment may include a plurality of user terminals 100 and devices 200 capable of communicating with each other through a communication network.

First, the communication network may be configured regardless of its communication mode, such as wired and wireless, and may be implemented in various forms so that communication between a server and a server and communication between a server and a terminal are performed.

Each of the plurality of user terminals 100 may be a desktop computer, a notebook computer, a tablet, a smart phone, or the like. Preferably, as shown in FIG. 1 , the first user terminal 110 , the second user terminal 120 , the third user terminal 130 , etc. may be smartphones, and may be employed differently depending on the embodiment. .

Each of the plurality of user terminals 100 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The plurality of user terminals 100 may be configured to communicate with the device 200 by wire or wireless.

Each of the plurality of user terminals 100 is connected to a web page operated by a person or group that provides a service using the device 200 , or is developed and distributed by a person or group that provides a service using the device 200 . One application can be installed. Each of the plurality of user terminals 100 may be linked with the device 200 through a web page or an application.

Each of the plurality of user terminals 100 may access the device 200 through a web page or an application provided by the device 200 .

Hereinafter, for convenience of description, the operation of the first user terminal 110 will be mainly described, but other user terminals such as the second user terminal 120 may perform the operation of the first user terminal 110 instead. to be.

The device 200 may be a self-server owned by a person or organization that provides services using the device 200, a cloud server, or a peer-to-peer (p2p) set of distributed nodes. may be The device 200 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The device 200 may include at least one artificial neural network that performs an inference function.

The device 200 may be configured to communicate with the plurality of user terminals 100 by wire or wireless, and controls the operation of each of the plurality of user terminals 100 , and displays which information on the screen of each of the plurality of user terminals 100 . You can control what is displayed.

Meanwhile, for convenience of explanation, only the first user terminal 110 , the second user terminal 120 , and the third user terminal 130 are shown in FIG. 1 among the plurality of user terminals 100 , but the number of terminals is not It may vary depending on the example. As long as the processing capacity of the apparatus 200 allows, the number of terminals is not particularly limited.

According to one embodiment, in the present invention, in terms of service, it is possible to provide a service so that the user can directly upload the content of the audio guide through the application, and automate the process of first inspecting the properties and quality of the uploaded audio. quality control can be streamlined by allowing the uploader to recognize and correct error information.

In addition, in the present invention, from the technical point of view, it is possible to extract the characteristics of the audio guide original data, design a validator based on this, and vectorize the user's upload history data, content error frequency, test result values, etc. A unique Audio Semantics Network can be rescued based on sampling, and data storage can be made more efficient by deriving timecode-based error information after automated quality inspection after upload.

According to an embodiment, the device 200 may verify the guide sound source related to the cultural content based on artificial intelligence. To this end, the apparatus 200 may include a plurality of artificial neural networks that have been pre-trained to perform a machine learning algorithm.

In the present invention, artificial intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, perceptual ability, etc., and implements it with a computer, and the concepts of machine learning, symbolic logic, etc. may include Machine Learning (ML) is an algorithm technology that classifies or learns characteristics of input data by itself. Artificial intelligence technology is an algorithm of machine learning that can analyze input data, learn the results of the analysis, and make judgments or predictions based on the results of the learning. In addition, technologies that use machine learning algorithms to simulate functions such as cognition and judgment of the human brain can also be understood as a category of artificial intelligence. For example, technical fields such as verbal comprehension, visual comprehension, reasoning/prediction, knowledge expression, and motion control may be included.

Machine learning may refer to the process of training a neural network model using experience of processing data. With machine learning, computer software could mean improving its own data processing capabilities. The neural network model is constructed by modeling the correlation between data, and the correlation may be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data, and repeating this process to optimize parameters of a neural network model can be called machine learning. For example, a neural network model may learn a mapping (correlation) between an input and an output with respect to data given as an input/output pair. Alternatively, the neural network model may learn the relationship by deriving regularity between the given data even when only input data is given.

The artificial intelligence learning model or neural network model may be designed to implement a human brain structure on a computer, and may include a plurality of network nodes that simulate neurons of a human neural network and have weights. A plurality of network nodes may have a connection relationship with each other by simulating a synaptic activity of a neuron through which a neuron sends and receives a signal through a synapse. In the AI learning model, a plurality of network nodes can exchange data according to a convolutional connection relationship while being located in layers of different depths. The artificial intelligence learning model may be, for example, an artificial neural network model, a convolutional neural network model, or the like. As an embodiment, the AI learning model may be machine-learned according to a method such as supervised learning, unsupervised learning, reinforcement learning, or the like. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. may be used.

Among them, CNNs are a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data of the two-dimensional structure. Compared with other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained through standard back-passing. CNNs are easier to train than other feed-forward neural network techniques and have the advantage of using fewer parameters.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increased size of available training data and the availability of computational power, combined with advances in algorithms such as piecewise linear units and dropout training, have greatly improved many computer vision tasks. For huge data sets, such as those available for many tasks today, overfitting is not important, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. To this end, a distributed, scalable implementation of deep neural networks may be used.

2 is a diagram schematically showing the configuration of an apparatus according to an embodiment.

Referring to FIG. 2 , the device 200 may include a WEB client, Audio Encoder, Automatic Audio Validator, Machine Learning Engine, Audio Feature DB, Uploader history DB, and Content DB.

The WEB client can perform functions such as upload UI, error reporting, and file manager. The WEB client can provide a web page so that audio files and metadata can be uploaded via the web. Here, the metadata may be generated by input of tags and keywords, and may include extensions, tags, digital rights management (DRM) information, setting information, and the like.

Audio Encoder may perform a function of checking Audio Feature, Metadata, Sound Waveform, Timecode, and the like. That is, the audio encoder may collect and vectorize metadata, waveform information, time code, audio characteristic information, and the like.

Automatic Audio Validator utilizes the information stored in the Audio Feature DB and Uploader history DB, compares it with the existing audio characteristic information table and upload history data, measures the vector value of the input audio and the distance between each cluster, and based on the measurement result It is possible to distinguish between positive and negative.

Automatic Audio Validator classifies positives and negatives for each feature or feature extraction section (F1, F2, F3, etc.) of an audio file, and then classifies it as a positive sample if the entire area is positive, such as D1 and D3. If one or more regions are negative, such as D2 and D4, it can be classified as a negative sample.

Audio files classified as positive samples may be determined as Pass and transmitted to the Content DB, and audio files classified as negative samples may be determined as Fail and reported as timecode-based error contents.

Audio Feature DB and Uploader history DB can be learned by Machine Learning Engine.

3 is a flowchart illustrating a process of verifying a guide sound source related to cultural content based on artificial intelligence according to an embodiment.

Referring to FIG. 3 , first, in step S301 , the device 200 may receive a first audio file and first metadata from the first user terminal 110 . Here, the first audio file is a file in which a guide sound source related to cultural content is recorded, and the first metadata is data describing the first audio file, and may include keywords, tag information, and the like.

Specifically, the device 200 may provide a web page for uploading the first audio file and input of the first metadata to the first user terminal 110, and the first audio file is selected through the web page. and when the first metadata is input, the first audio file and the first metadata may be received from the first user terminal 110 .

In step S302 , the device 200 may check the quality of the first audio file.

Specifically, the device 200 may determine whether the first audio file is a normal audio file by checking the extension of the first audio file, and analyze the waveform of the first audio file to determine whether the first audio file is a user It is possible to determine whether the voice is a recorded audio file, and through the process of analyzing the frequency of the first audio file to determine whether the first audio file is an audio file in which the user's voice is recorded, quality can be checked.

When it is determined that the first audio file is a normal audio file and the user's voice is recorded as an audio file, the apparatus 200 determines that there is no problem in the first audio file and may check the quality of the first audio file.

When it is determined that the first audio file is an abnormal audio file or an audio file in which the user's voice is not recorded, the device 200 determines that there is a problem in the first audio file and may check the quality of the first audio file. have.

In step S303 , as a result of checking the quality of the first audio file, the device 200 may determine whether a problem has occurred in the first audio file.

If it is confirmed that there is a problem with the first audio file in step S303 , the device 200 provides the first user terminal 110 with a web page for re-uploading the first audio file to the first user terminal 110 . can Thereafter, returning to step S301 , the device 200 may receive the first audio file from the first user terminal 110 again.

If it is determined in step S303 that there is no problem in the first audio file, in step S304 , the device 200 may extract sound information for each zone from the first audio file through sampling.

For example, the device 200 may extract a sound recorded for 2 seconds from the first audio file once every 20 seconds as sound information, and extract the sound information extracted from the first zone as the first sound information, , the sound information extracted from the second zone may be extracted as the second sound information. In this case, the first zone may be a zone recorded for the first 2 seconds in the first audio file, and the second zone may be a zone recorded for 2 seconds after 20 seconds from the first zone.

In step S305 , the device 200 may identify a keyword from the sound information extracted for each zone through voice recognition.

For example, when the first sound information and the second sound information are extracted from the first audio file, the device 200 may identify the first keyword from the first sound information through voice recognition, and from the second sound information The second keyword may be identified through voice recognition.

In step S306 , the device 200 may generate a first list based on the keywords identified for each zone.

For example, when the first keyword is identified in the first sound information and the second keyword is identified in the second sound information, the device 200 may set a first list that is a keyword list including the first keyword and the second keyword. can create

Meanwhile, in step S307 , the device 200 may generate a first input signal based on the first metadata.

For example, the device 200 may identify a keyword describing the first audio file based on the first metadata, and may generate the first input signal by encoding the identified keyword.

In step S308 , the device 200 may obtain a first output signal by inputting the first input signal to the first artificial neural network.

Specifically, the device 200 may include a pre-trained first artificial neural network. The first artificial neural network is an algorithm that receives a first input signal generated through metadata, selects a keyword that is inferred to be related to the metadata, and outputs a first output signal generated from the selected keyword list. can

In step S309 , the device 200 may generate a second list based on the first output signal. Here, the second list may mean a keyword list related to the first meta data.

For example, the apparatus 200 may obtain a first output signal from the first artificial neural network by inputting a first input signal generated based on the first metadata to the first artificial neural network, and the first output signal When the output values identified through . are identified as “first keyword” and “second keyword”, a second list that is a keyword list including the first keyword and the second keyword may be generated.

The first artificial neural network may be trained to select and analyze keywords related to metadata through metadata. The first artificial neural network may be learned through a method described later with reference to FIG. 9 . Through this, the first artificial neural network may select a keyword that is inferred to be related to the metadata, and may output a list of the selected keywords.

In step S310 , the device 200 may generate a third list by combining the first list and the second list. In this case, the device 200 may match the first list and the second list to generate a matching result as the third list.

In step S311 , the device 200 may generate a second input signal based on the third list.

For example, the device 200 may generate a second input signal by encoding a keyword included in the third list.

In operation S312 , the device 200 may obtain a second output signal by inputting the second input signal to the second artificial neural network.

Specifically, the device 200 may include a pre-trained second artificial neural network. After receiving the second input signal generated through the matched list, the second artificial neural network infers the verification result for the audio file according to the keyword matching degree, and outputs the second output signal generated as the inferred verification result. It may be an algorithm.

In operation S313 , the device 200 may generate a verification result for the first audio file based on the second output signal.

For example, the apparatus 200 may obtain a second output signal from the second artificial neural network by inputting the second input signal generated based on the third list to the second artificial neural network, and When the output value checked through the output value is “0”, the verification result for the first audio file is generated as verification failure, and when the output value checked through the second output signal is “1”, the verification result for the first audio file is generated. A verification result can be generated as verification success.

The second artificial neural network may be trained to analyze whether the audio file has been verified through the matched list. The second artificial neural network may be learned through a method described later with reference to FIG. 10 . Through this, the second artificial neural network may analyze and output whether the verification of the audio file succeeds or fails through the matched list. A detailed description related thereto will be described later with reference to FIGS. 4 and 5 .

4 is a flowchart illustrating a process of verifying an audio file according to a keyword matching degree according to an exemplary embodiment.

Referring to FIG. 4 , first, in step S401, when a second input signal is input, the second artificial neural network checks whether each keyword included in the first list is included in the second list based on the second input signal. can Here, the second input signal is a signal generated based on the third list, and the third list is a list in which the first list and the second list are matched and combined.

In step S402 , the second artificial neural network may classify keywords included in the first list into a 1-1 list and a 1-2 list. Here, the 1-1 list is a keyword list confirmed to be included in the second list among keywords included in the first list, and the 1-2 list is a keyword list that is not included in the second list among keywords included in the first list. This is a list of confirmed keywords.

For example, when the first list includes the first keyword, the second keyword, the third keyword, and the fourth keyword, and the second list includes the first keyword and the second keyword, the second artificial neural network is As a result of checking whether each of the first keyword, the second keyword, the third keyword, and the fourth keyword included in the first list is included in the second list, only the first keyword and the second keyword among the keywords included in the first list Since the third keyword and the fourth keyword among the keywords included in the second list and included in the first list are not included in the second list, the first keyword and the second keyword are divided into a 1-1 list, The third keyword and the fourth keyword may be divided into a 1-2 list.

In step S403 , the second artificial neural network may calculate a matching rate between the first list and the second list through a value obtained by dividing the number of keywords included in the 1-1 list by the number of keywords included in the first list. .

For example, when the first list includes the first keyword, the second keyword, the third keyword, and the fourth keyword, and the 1-1 list includes the first keyword and the second keyword, the second artificial keyword The neural network may calculate a matching rate between the first list and the second list as 50% through a value obtained by dividing 2 by 4.

In step S404 , the second artificial neural network may check whether the matching rate is greater than a reference value. Here, the reference value may be set differently depending on the embodiment.

If it is confirmed in step S404 that the matching rate is greater than the reference value, in step S405 , the second artificial neural network may analyze the verification result for the first audio file as verification success and output a second output signal. In this case, the second artificial neural network may generate a second output signal as an output value indicating verification success, and then output the generated second output signal.

If it is determined in step S404 that the matching rate is smaller than the reference value, in step S406 , the second artificial neural network may analyze the verification result for the first audio file as a verification failure and output a second output signal. In this case, the second artificial neural network may generate a second output signal as an output value indicating verification failure, and then output the generated second output signal.

5 is a flowchart illustrating a process of verifying an audio file according to whether a key keyword is included, according to an exemplary embodiment.

First, after generating the first list and the second list, the device 200 may analyze the first audio file as an audio guide for the first exhibition based on keywords included in the first list.

For example, when the first keyword and the second keyword are included in the first list, the device 200 determines that the first keyword and the second keyword are keywords used to describe the first exhibition object, The file may be analyzed as an audio guide of the first exhibit.

The device 200 may acquire a fourth list that is a list of key keywords set in the first exhibit. To this end, the database of the device 200 stores a list of core keywords divided by exhibits, and the device 200 may search the database to obtain a fourth list that is a list of core keywords of the first exhibit. Here, the key keywords may include historical facts about the exhibits, terms essential for describing the exhibits, and the like.

When the fourth list is obtained, the device 200 may add the fourth list to the second list.

Thereafter, the device 200 combines the first list and the second list to generate a third list, and based on the third list, generates a second input signal, and then converts the second input signal to a second artificial neural network. can be entered in

In step S501, when the second input signal is input, the second artificial neural network compares the keywords included in the first list with the keywords included in the fourth list based on the second input signal, and determines whether each keyword matches. can be checked Here, the keyword may be a keyword indicating the characteristics of the exhibit, such as year, date, region, and producer.

Specifically, when the first list includes the first keyword, the second keyword, the third keyword, and the fourth keyword, and the fourth list includes the first keyword and the second keyword, the second artificial neural network is 4 Check whether the first keyword included in the list matches the first keyword included in the first list, and check whether the second keyword included in the fourth list matches the second keyword included in the first list can be checked

For example, when the first keyword is a keyword indicating a year, the second artificial neural network determines whether the first keyword included in the fourth list matches the first keyword included in the first list, and as a result, the fourth list If the first keyword included in the year is 1535 and the first keyword included in the first list is confirmed to be 1535, it is determined that the first keyword included in the fourth list matches the first keyword included in the first list. If it is confirmed that the first keyword included in the fourth list is 1535 and the first keyword included in the first list is 1534, the first keyword included in the fourth list and the first keyword included in the first list It can be confirmed that the first keyword does not match.

In step S502, the second artificial neural network may check whether the keywords included in the fourth list all match the keywords included in the first list.

In step S502 , it may be checked whether the keywords included in the fourth list all match the keywords included in the first list.

If it is confirmed in step S502 that the keywords included in the fourth list match all the keywords included in the first list, in step S503, the second artificial neural network analyzes the verification result for the first audio file as verification success. 2 output signals can be output. In this case, the second artificial neural network may generate a second output signal as an output value indicating verification success, and then output the generated second output signal.

If it is confirmed in step S502 that the keywords included in the fourth list do not all match the keywords included in the first list, and at least one of the keywords included in the fourth list does not match the keywords included in the first list, In step S504 , the second artificial neural network may analyze the verification result for the first audio file as a verification failure and output a second output signal. In this case, the second artificial neural network may generate a second output signal as an output value indicating verification failure, and then output the generated second output signal.

6 is a flowchart illustrating a process of extracting and determining sound information for each zone according to an exemplary embodiment.

Referring to FIG. 6 , first, in step S601 , the device 200 may extract first sound information that is a sound of a first region from a first audio file.

In step S602 , the device 200 may analyze the first sound information and divide the first zone into a first section that is a voice section and a second section that is a breathing section. At this time, the device 200 analyzes the first sound information, divides the voice section in which the user voice is reproduced in the first section into the first section, and removes the breathing section in which the user voice is not reproduced in the first section It can be divided into two sections.

In step S603 , the device 200 may calculate the first ratio through a value obtained by dividing the length of the first section by the length of the first section.

For example, when the length of the first section is 2 seconds and the length of the first section is 1 second, the apparatus 200 may calculate the first ratio as 50% through a value obtained by dividing 1 by 2.

In step S604 , the device 200 may determine whether the first ratio is higher than the first reference ratio. Here, the first reference ratio may be set differently depending on the embodiment.

If it is determined in step S604 that the first ratio is higher than the first reference ratio, in step S610 , the device 200 may determine the sound information extracted from the first zone as the first sound information.

If it is determined in step S604 that the first ratio is lower than the first reference ratio, in step S605 , the device 200 may determine whether the first ratio is higher than the second reference ratio. Here, the second reference ratio may be set to a value smaller than the first reference ratio.

If it is determined in step S605 that the first ratio is higher than the second reference ratio, in step S606, the device 200 extracts second sound information that is the sound of the second zone extending the first zone from the first audio file. can

For example, when the first sound information, which is the sound of the first zone, which is the zone recorded for 2 seconds, is extracted from the first audio file, the device 200 determines that the first ratio is higher than the second reference ratio, In the first audio file, the second sound information that is the sound of the second region that is the region recorded for 4 seconds may be extracted by extending the first region forward and backward by 1 second.

In step S607 , the device 200 may analyze the second sound information to divide the second section into a third section that is a voice section and a fourth section that is a breathing section. At this time, the device 200 analyzes the second sound information, divides the voice section in which the user voice is reproduced in the second section into a third section, and removes the breathing section in which the user voice is not reproduced in the second section It can be divided into 4 sections.

In step S608 , the device 200 may calculate the second ratio through a value obtained by dividing the length of the third section by the length of the second section.

In step S609 , the device 200 may determine whether the second ratio is higher than the first reference ratio.

If it is determined in step S609 that the second ratio is higher than the first reference ratio, in step S611 , the device 200 may determine the sound information extracted from the first zone as the second sound information.

On the other hand, if it is confirmed that the first ratio is lower than the second reference ratio in step S605 or it is confirmed that the second ratio is lower than the first reference ratio in step S609, in step S612, the device 200 is extracted from the first zone It can be determined that there is no sound information.

7 is a flowchart illustrating a process of controlling policy settings of a host firewall based on artificial intelligence according to an embodiment.

First, the device 200 may be connected to a plurality of servers to provide a cultural content platform, and the plurality of servers may include a first server, a second server, and the like. Each of the plurality of servers is a host connected to the device 200 through an internal network, and may transmit/receive data to and from the device 200 through a network connected to the internal network.

Referring to FIG. 7 , in step S701 , the device 200 may receive operating system information of the first server and network setting information of the first server from the first server. Here, the operating system information of the first server may include information for identifying the type and version of the operating system installed and operated in the first server, and the network setting information of the first server is the network setting state of the first server. may include information indicating

In step S702 , the device 200 may analyze the network connection state of the first server based on the operating system information of the first server and the network setting information of the first server. In this case, the device 200 may analyze the network connection state of the first server and generate an analysis result of the network connection state of the first server. Here, the analysis result of the network connection state of the first server may include a result of analyzing in which operating system the first server is operated and through which network port the first server is connected.

In step S703 , the device 200 may apply the analysis result of the network connection state of the first server to the third artificial neural network previously learned in the device 200 .

According to an embodiment, the third artificial neural network may be an algorithm that analyzes and outputs a firewall policy suitable for the server after receiving an analysis result of the server's network connection state.

In step S704 , the device 200 may select a firewall policy suitable for the first server based on the output of the third artificial neural network.

For example, the device 200 applies the analysis result of the network connection state of the first server to the third artificial neural network, and as a result of checking the output of the third artificial neural network, when the output value is “0”, the first When a firewall policy suitable for the server is selected as the first policy and the output value is confirmed as “1”, a firewall policy suitable for the first server may be selected as the second policy.

The third artificial neural network may be trained to analyze a firewall policy suitable for the server through the analysis result of the server's network connection state. The third artificial neural network may be learned through a method described later with reference to FIG. 10 . Through this, the third artificial neural network may analyze and output a firewall policy suitable for the server in consideration of the network connection state of the server.

In step S705, the device 200 may check whether the policy selected in step S704 is included in the firewall policy of the first server. For example, when the first policy is selected as the firewall policy suitable for the first server, the device 200 may check whether the first policy is included in the firewall policy of the first server.

To this end, information on the firewall policy set in each of the plurality of servers may be stored separately for each server in the database of the device 200 . The number of information about the firewall policy may be different depending on the firewall policy set in the server, and when a plurality of firewall policies are set in one server, a priority for each of the plurality of firewall policies may be set .

If it is determined in step S705 that the first policy is included in the firewall policy of the first server, in step S706 , the device 200 may determine that a change to the firewall policy of the first server is not required.

That is, since the first policy selected as the firewall policy suitable for the first server is already set in the first server, the device 200 may determine that maintenance of the firewall policy of the first server is necessary.

After step S706, when a certain period elapses, the process returns to step S701, the device 200 receives the operating system information of the first server and the network setting information of the first server again from the first server, and a firewall policy suitable for the first server can be selected again.

Meanwhile, if it is determined in step S705 that the first policy is not included in the firewall policy of the first server, in step S707 , the device 200 may determine that a change to the firewall policy of the first server is necessary.

That is, since the first policy selected as the firewall policy suitable for the first server is not set in the first server, the device 200 may determine that the firewall policy of the first server needs to be changed.

In step S708 , the device 200 may register by adding the selected policy to the firewall policy of the first server. For example, if the first policy is selected as a firewall policy suitable for the first server, the device 200 may register the first policy by adding the first policy to the firewall policy of the first server. In this case, when the second policy is already registered in the firewall policy of the first server, the first policy may be additionally registered in the firewall policy of the first server. Through this, when a plurality of policies are registered in the firewall policy of the first server, the device 200 may set a priority for each of the plurality of policies.

The process of adding and registering the first policy to the firewall policy of the first server is to add and update the first policy to the information about the firewall policy of the first server stored in the database of the device 200, and to the firewall policy. The setting may be performed through step S709.

In step S709 , when the first policy is additionally registered in the firewall policy of the first server, the device 200 transmits a setting change command for the firewall policy of the first server to the first server, so that the first policy is the first It can be controlled to be set by the server's firewall policy. Here, the setting change command for the firewall policy of the first server is a command for setting by adding the first policy to the firewall policy of the first server, and the first server is configured to change the settings for the firewall policy of the first server through the setting change command. , can be set by adding the first policy as a firewall policy of the first server.

After step S709, when a certain period elapses, the process returns to step S701, the device 200 receives the operating system information of the first server and the network setting information of the first server again from the first server, and a firewall policy suitable for the first server can be selected again.

8 is a flowchart illustrating a process of detecting an approach estimated as an attack and providing a countermeasure according to the number of attacks, according to an exemplary embodiment.

Referring to FIG. 8 , first, in step S801 , the device 200 may receive monitoring information on network traffic of the first server from the first server. Here, the monitoring information on the network traffic of the first server includes monitoring information on which port was connected to and generated traffic from the first server, and how much traffic was generated per hour for each IP address connected to the first server. It may include monitoring information and the like.

In step S802 , the device 200 may apply the monitoring information on the network traffic of the first server to the fourth artificial neural network previously learned in the device 200 .

According to an embodiment, the fourth artificial neural network may be an algorithm that receives monitoring information on network traffic of a server and outputs a detection result of whether an approach estimated as an attack is detected to the server. Here, the attack is an attack on the server, and may include ransomware, hacking, and the like.

In step S803 , the device 200 may detect whether an approach estimated as an attack to the first server is detected based on the output of the fourth artificial neural network.

For example, the device 200 applies the monitoring information for the network traffic of the first server to the fourth artificial neural network, and as a result of checking the output of the fourth artificial neural network, when the output value is “0”, the first server When it is detected that an approach estimated as an attack is not detected and the output value is confirmed as “1”, it may be detected that an approach estimated as an attack is detected by the first server.

The fourth artificial neural network may be trained to analyze whether an approach estimated as an attack is detected to the server through monitoring information on network traffic of the server. The fourth artificial neural network may be learned through a method described later with reference to FIG. 10 . Through this, the fourth artificial neural network may analyze and output whether an approach estimated as an attack is detected to the server in consideration of the change state of the server's network traffic.

In step S804 , the device 200 may check how many times an approach estimated as an attack is detected to the first server during the reference period, and calculate the first number of attacks, which is the number of times the first server has been attacked during the reference period. Here, the reference period may be set differently depending on the embodiment, and steps S801 to S803 may be repeatedly performed during the reference period. Through this, the device 200 may calculate the first number of attacks by checking how many times the approach estimated as an attack is detected by the first server during the reference period.

In step S805 , the device 200 may determine whether the number of first attacks is less than the first reference number. Here, the first reference number may be set differently in proportion to the length of the reference period.

If it is determined in step S805 that the number of first attacks is less than the first reference number, in step S807 , the device 200 may determine the state of the first server as a normal state.

After a certain period of time after step S807, the process returns to step S801, and the device 200 may again receive monitoring information for network traffic of the first server from the first server, and recalculate the first number of attacks.

If it is determined in step S805 that the number of first attacks is greater than the first reference number, in step S806 , the device 200 may determine whether the first number of attacks is less than the second reference number. Here, the second reference number may be set to a value greater than the first reference number.

If it is determined in step S806 that the number of first attacks is less than the second reference number, in step S808, the device 200 may determine the state of the first server as a warning state.

If it is determined in step S806 that the number of first attacks is greater than the second reference number, in step S809, the device 200 may determine the state of the first server as a dangerous state.

In step S810 , when it is determined that the state of the first server is a warning state, the device 200 may transmit a notification message warning of an attack of the first server to the first manager terminal. Here, the first manager terminal means a terminal used by the first manager registered as a manager of the first server, and the database of the device 200 stores contact information of the first manager terminal, and through this, the device 200 may transmit a notification message to the first manager terminal.

In step S811 , when the state of the first server is determined to be in a critical state, the device 200 transmits a connection blocking command of the first port to which the connection is allowed through the first policy to the first server, and in the first server Connection through the first port may be controlled to be blocked.

That is, in the first server, the first policy is set as a firewall policy, and the first policy includes a setting to allow a connection through the first port, so that the network connection state of the first server is connected through the first port. In a state in which traffic is being generated through the first port, when the number of attacks through the first port is too large, the device 200 transmits a connection blocking command of the first port to the first server. , it is possible to control the first server to block the connection through the first port. In this case, the first server may block the connection of the first port through the connection blocking command of the first port, so that traffic does not occur through the first port.

9 is a diagram for explaining learning of an artificial neural network according to an embodiment.

The artificial neural network may be a component included in the device 200 and may be learned through the device 200 or a separate learning device.

After receiving the first input signal generated through the metadata, the first artificial neural network may select a keyword that is inferred to be related to the metadata and output the first output signal generated as the selected keyword list. .

Hereinafter, a process in which the first artificial neural network is learned through the learning device will be described.

First, in step S901, the learning apparatus may acquire training data and a label.

For learning of the first artificial neural network, the learning apparatus may obtain a data set including each keyword information and related keyword information related to each keyword as each training data. Also, the learning apparatus may obtain a degree of relevance between each keyword and a related keyword as a label corresponding to each training data.

In step S902, the learning apparatus may generate an input of the artificial neural network from the training data.

The learning apparatus may use the training data as it is as an input of the first artificial neural network, or may generate an input of the first artificial neural network after a normal process of removing unnecessary information from each training data.

In step S903, the learning apparatus may apply the input to the artificial neural network.

The first artificial neural network included in the device 200 may be an artificial neural network that is learned according to supervised learning. The first artificial neural network may be a convolutional neural network (CNN) or recurrent neural network (RNN) structure suitable for learning through supervised learning.

In step S904, the learning apparatus may obtain an output from the artificial neural network.

The output of the first artificial neural network may be inference of keywords related to metadata. Specifically, the first artificial neural network may select a keyword that is inferred to be related to the metadata, and may output a first output signal generated from the selected keyword list.

In step S905, the learning apparatus may compare the output with the label. The process of comparing the output of the artificial neural network corresponding to the inference and the label corresponding to the correct answer may be performed by calculating a loss function. As the loss function, a known mean squared error (MSE), cross entropy error (CEE), etc. may be used. However, the present invention is not limited thereto, and loss functions used in various artificial neural network models may be used if the deviation, error, or difference between the output of the artificial neural network and the label can be measured.

In step S906 , the learning apparatus may optimize the artificial neural network based on the comparison value. The learning apparatus can gradually match the output of the artificial neural network corresponding to inference and the label corresponding to the correct answer by updating the weights of the nodes of the artificial neural network so that the comparison value becomes smaller and smaller, and through this, the artificial neural network can be optimized to output an inference close to the correct answer. Specifically, the learning apparatus may optimize the artificial neural network by repeating the process of resetting the weight of the artificial neural network so that the loss function corresponding to the comparison value approaches the estimated value of the minimum value. A known backpropagation algorithm, stochastic gradient descent, or the like may be used for optimization of an artificial neural network. However, the present invention is not limited thereto, and a weight optimization algorithm used in various neural network models may be used.

The learning apparatus may train the artificial neural network by repeating such a process.

Through this, after receiving the first input signal generated through the metadata, the first artificial neural network selects a keyword that is inferred to be related to the metadata and outputs the first output signal generated from the selected keyword list can be learned The first artificial neural network may be used to output a keyword that is inferred to be related to the metadata.

10 is a diagram for explaining learning of an artificial neural network according to another embodiment.

According to an embodiment, the artificial neural network may be any one of the second artificial neural network, the third artificial neural network, and the fourth artificial neural network. Here, after receiving the second input signal generated through the matched list, the second artificial neural network infers the verification result for the audio file according to the keyword matching degree, and receives the second output signal generated by the inferred verification result. An algorithm that outputs, the third artificial neural network is an algorithm to analyze and output a firewall policy suitable for the server after receiving the analysis result of the server's network connection state, and the fourth artificial neural network is monitoring information for the server's network traffic After receiving the input, it may be an algorithm that outputs a detection result as to whether an approach estimated as an attack is detected to the server.

The learning device in which the artificial neural network is trained may be the same device as the learning device that provides the verification method of the guide sound source related to the artificial intelligence-based cultural content, or it may be a separate device. Hereinafter, a process in which an artificial neural network is trained will be described.

First, in step S1001, the learning apparatus may generate an input to be input to the artificial neural network. In this case, the learning apparatus may generate a second input signal based on the third list to input to the second artificial neural network, and based on the analysis result of the network connection state of the server to input to the third artificial neural network. to generate an input, and to input to the fourth artificial neural network, the input may be generated based on monitoring information on network traffic of the server.

Specifically, the learning apparatus may perform a process of preprocessing keywords included in the third list. The preprocessed third list may be used as it is as an input to the second artificial neural network, or an input of the second artificial neural network may be generated through a normal process of removing unnecessary information.

In addition, the learning apparatus may perform a process of pre-processing the analysis result of the network connection state of the server. The analysis result of the preprocessed network connection state may be directly used as an input of the third artificial neural network, or an input of the third artificial neural network may be generated through a normal process of removing unnecessary information.

Also, the learning apparatus may perform a process of pre-processing monitoring information for network traffic of the server. The monitoring information on the network traffic of the server on which the preprocessing has been performed may be used as it is as an input of the fourth artificial neural network, or an input of the fourth artificial neural network may be generated through a normal process of removing unnecessary information.

In step S1002, the learning apparatus may apply an input to the artificial neural network. In this case, the learning apparatus applies the input to the second artificial neural network when generating the input of the second artificial neural network, and applies the input to the third artificial neural network when generating the input of the third artificial neural network, and applies the input to the fourth artificial neural network. When the input of the neural network is generated, the input may be applied to the fourth artificial neural network.

The artificial neural network may be an artificial neural network trained according to reinforcement learning. The artificial neural network may be a Q-Network, DQN (Depp Q-Network), or relational network (RL) structure suitable for outputting abstract reasoning through reinforcement learning.

An artificial neural network trained according to reinforcement learning can be updated and optimized by reflecting the evaluation in various rewards. For example, the second artificial neural network is updated and optimized through the first reward and the second reward, the third artificial neural network is updated and optimized through the third reward and the fourth reward, and the fourth artificial neural network is updated and optimized through the fifth reward and through the sixth compensation can be updated and optimized.

For example, as for the first reward, the higher the keyword matching degree, the higher the reward value may be if the verification result for the audio file is inferred to be successful. If inferred, the reward value may increase.

In addition, the third reward may increase as a firewall policy suitable for the server is selected in consideration of the server's network connection state, and the fourth reward may increase as a firewall policy unsuitable for the server is not selected in consideration of the server's network connection state. can rise

In addition, as for the fifth reward, the higher the amount of network traffic of the server, the higher the reward value may be when it detects that an approach that is presumed to be an attack on the server is detected. If an approach presumed to be an attack is detected as being detected, the reward value may be increased.

In step S1003, the learning apparatus may obtain an output from the artificial neural network.

The output of the second artificial neural network may be a second output signal generated as a result of verifying the audio file. At this time, after receiving the second input signal generated through the matched list, the second artificial neural network infers the verification result for the audio file according to the keyword matching degree, and receives the second output signal generated by the inferred verification result. can be printed out.

The output of the third artificial neural network may be information about a firewall policy suitable for the server. In this case, the third artificial neural network may select a firewall policy suitable for the server in consideration of the network connection state of the server, and output information on the firewall policy most suitable for the server.

The output of the fourth artificial neural network is a detection result of whether an approach estimated as an attack is detected by the server. At this time, the fourth artificial neural network analyzes whether an approach estimated as an attack is detected to the server through monitoring information on the server's network traffic, and outputs a detection result as to whether an approach estimated as an attack is detected to the server. have.

In step S1004, the learning device may evaluate the output of the artificial neural network and pay a reward.

The evaluation of the output of the second artificial neural network may be divided into a first reward and a second reward. For example, as the keyword matching degree is higher, the learning device awards a larger amount of first reward as the verification result of the audio file is inferred as success, and when the verification result for the audio file is inferred as failure as the keyword matching degree is low, the learning apparatus 2 You can award a lot of rewards.

The evaluation of the output of the third artificial neural network may be divided into a third reward and a fourth reward. For example, the learning device gives a higher third reward as it selects a firewall policy suitable for the server in consideration of the server's network connection status, and does not select a firewall policy that is not suitable for the server in consideration of the server's network connection status. The more the 4th reward, the more can be awarded.

The evaluation of the output of the fourth artificial neural network may be divided into a fifth reward and a sixth reward. For example, if the learning device detects that the server's network traffic is excessive, the server receives a 5th reward when it detects that an attack presumed to be an attack. Detecting that the presumed approach has been detected may award a large sixth reward.

In step S1005, the learning apparatus may update the artificial neural network based on the evaluation.

In an environment in which the second artificial neural network infers the verification result for the audio file according to the degree of keyword matching, the learning apparatus is configured to maximize the expected value of the sum of the rewards. The second artificial neural network can be updated through the process of optimizing a policy that determines actions to be taken in the fields.

In addition, the learning device performs an action to be taken in specific states so that the expectation value of the consensus of rewards is maximized in an environment in which the third artificial neural network analyzes the firewall policy most suitable for the server. The third artificial neural network can be updated through the process of optimizing a policy that determines actions.

In addition, in an environment in which the fourth artificial neural network analyzes whether an approach estimated as an attack to the server is detected through the monitoring information on the server's network traffic, the learning device expects a sum of rewards (rewards) The fourth artificial neural network can be updated through the process of optimizing the policy that determines the actions to be taken in specific states so that expectation) is maximized.

Meanwhile, the process of optimizing the policy may be performed through the process of estimating the maximum value of the expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function may be performed through stochastic gradient descent (SGD) or the like. The process of optimizing the policy is not limited thereto, and various optimization algorithms used in reinforcement learning may be used.

The learning apparatus may gradually update the artificial neural network by repeating the learning process of the artificial neural network as described above.

Specifically, the learning apparatus may infer the verification result for the audio file according to the keyword matching degree, and train the second artificial neural network to output the inferred verification result.

Through this, after receiving the second input signal generated based on the metadata, the learning apparatus infers the verification result for the audio file according to the keyword matching degree, and outputs the second output signal generated as the inferred verification result can train a second artificial neural network.

That is, when the learning device infers the verification result for the audio file through the matched list, the second artificial neural network can be trained by reflecting reinforcement learning through the first reward, the second reward, etc., and adjusting the analysis criteria. can

In addition, in selecting a firewall policy suitable for the server, the learning device considers all items such as forwarding type, protocol, external allowed IP address, external blocked IP address, virtual port, and operation port, and selects the most suitable firewall policy for the server. After selection, the third artificial neural network that outputs information about the selected firewall policy may be trained.

Through this, after receiving the analysis result of the network connection state of the server, the learning apparatus may learn the third artificial neural network that analyzes and outputs a firewall policy suitable for the server.

That is, when analyzing the firewall policy suitable for the server through the analysis result of the server's network connection state, the learning device reflects reinforcement learning through the third reward, the fourth reward, etc., by adjusting the analysis criteria, 3 Artificial neural networks can be trained.

Also, the learning apparatus may train a fourth artificial neural network that outputs a detection result of whether an approach estimated as an attack is detected to the server through monitoring information on network traffic of the server.

That is, the learning device reflects reinforcement learning through the 5th reward, the 6th reward, etc. when analyzing whether an access estimated as an attack is detected to the server through the monitoring information on the server's network traffic, analysis criteria By adjusting , the fourth artificial neural network can be trained.

11 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

The device 200 according to an embodiment includes a processor 210 and a memory 220 . The processor 210 may include at least one of the devices described above with reference to FIGS. 1 to 10 , or perform at least one method described above with reference to FIGS. 1 to 10 . A person or organization using the apparatus 200 may provide a service related to some or all of the methods described above with reference to FIGS. 1 to 10 .

The memory 220 may store information related to the above-described methods or a program in which methods to be described below are implemented. The memory 220 may be a volatile memory or a non-volatile memory.

The processor 210 may execute a program and control the device 200 . The code of the program executed by the processor 210 may be stored in the memory 220 . The device 200 may be connected to an external device (eg, a personal computer or a network) through an input/output device (not shown), and may exchange data through wired/wireless communication.

The device 200 may be used to train an artificial neural network or to use a learned artificial neural network. The memory 220 may include a learning or learned artificial neural network. The processor 210 may learn or execute an artificial neural network algorithm stored in the memory 220 . The apparatus 200 for learning an artificial neural network and the apparatus 200 for using the learned artificial neural network may be the same or may be separate.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (3)

In a method of verifying a guide sound source related to cultural content based on artificial intelligence, performed by a device,
receiving a first audio file and first metadata from a first user terminal;
checking the quality of the first audio file;
extracting sound information for each zone through sampling from the first audio file when it is confirmed that there is no problem in the first audio file as a result of checking the quality of the first audio file;
identifying a keyword from the sound information extracted for each zone through voice recognition, and generating a first list that is a keyword list related to the first audio file based on the identified keyword for each zone;
generating a first input signal based on the first metadata;
obtaining a first output signal by inputting the first input signal to a pre-trained first artificial neural network;
generating a second list that is a keyword list related to the first meta data based on the first output signal;
generating a third list by combining the first list and the second list;
generating a second input signal based on the third list;
obtaining a second output signal by inputting the second input signal to a pre-trained second artificial neural network; and
Based on the second output signal, comprising the step of generating a verification result for the first audio file,
A method of verifying the guide sound source related to artificial intelligence-based cultural content. According to claim 1,
The second artificial neural network is
Based on the second input signal, it is checked whether each keyword included in the first list is included in the second list,
Divide the keywords included in the first list into a 1-1 list confirmed to be included in the second list and a 1-2 list confirmed not included in the second list,
A matching rate between the first list and the second list is calculated through a value obtained by dividing the number of keywords included in the 1-1 list by the number of keywords included in the first list,
When it is confirmed that the matching rate is greater than a preset reference value, the verification result of the first audio file is analyzed as verification success and the second output signal is output,
If it is confirmed that the matching rate is smaller than the reference value, analyzing the verification result for the first audio file as verification failure and outputting the second output signal,
A method of verifying the guide sound source related to artificial intelligence-based cultural content. According to claim 1,
After generating the second list,
analyzing the first audio file as an audio guide of a first exhibition object based on a keyword included in the first list;
obtaining a fourth list, which is a list of key keywords set in the first exhibit; and
adding the fourth list to the second list,
The second artificial neural network is
based on the second input signal, comparing the keywords included in the first list with the keywords included in the fourth list, and confirming whether they match for each keyword;
When it is confirmed that the keywords included in the fourth list match all the keywords included in the first list, the verification result for the first audio file is analyzed as verification success and the second output signal is output,
If it is confirmed that one or more of the keywords included in the fourth list do not match the keywords included in the first list, the verification result for the first audio file is analyzed as verification failure and the second output signal is output. doing,
A method of verifying the guide sound source related to artificial intelligence-based cultural content.

Images