KR20230123809A - System for expressing act and emotion of character based on sound and facial expression - Google Patents

Abstract

The present invention relates to a method for generating animation expressing motions and emotions of a character. In one embodiment, a voice signal including an arbitrary sentence is converted into text data and syntax analysis is performed to obtain syntax data including the sentence. generating; generating character data including characteristics of a character from the voice signal; selecting a first animation matching the syntax data from among a plurality of gesture animations; selecting a second animation matching the character data from among a plurality of facial animations; and blending a gesture according to the syntax data and a facial expression according to the character data into first and second animations, respectively.

Description

System for expressing act and emotion of character based on sound and facial expression}

The present invention relates to a character animation generation system using a computer, and more particularly, to a character animation generation system for generating animation expressing motions and emotions of a character based on voice and expression.

A digital character included in animation is applied to various fields, and for example, detects the motion of a play actor and animates it into a digital character.

As another example, the digital character is also reproduced as a moving picture synchronized with the sound. For example, Korean Patent Publication No. 2006-0054678 (May 23, 2006) discloses a technique of synchronizing a character to sound.

Patent Document 1: Korean Patent Publication No. 2006-0054678 (published on May 23, 2006) Patent Document 2: Korean Patent Publication No. 2006-0031449 (published on April 12, 2006)

According to an embodiment of the present invention, when a user utters a voice signal including a predetermined sentence, a character animation video is produced by blending motions and facial expressions of characters based on the voice signal or based on the voice signal and the user's facial expression. Disclosed is a system for generating character animation.

According to one embodiment of the present invention, as a method for generating animation expressing motions and emotions of a character, a voice signal including an arbitrary sentence is converted into text data and syntax analysis is performed to generate syntax data including the sentence. doing; generating character data including characteristics of a character from the voice signal; selecting a first animation matching the syntax data from among a plurality of gesture animations; selecting a second animation matching the character data from among a plurality of facial animations; and blending a gesture according to the syntax data and a facial expression according to the character data into first and second animations, respectively.

According to one embodiment of the present invention, a computer readable recording medium on which a computer program for executing the animation generating method is recorded.

According to an embodiment of the present invention, based on voice signals and facial expressions uttered by a user, motions and facial expressions corresponding to each word or phrase of a sentence uttered and emotion are automatically matched and blended to create an automatic and creative character animation image. It has a technical effect that can be crafted with

In addition, since the animation generation technology according to an embodiment of the present invention can implement the user's lines, actions, and emotional expressions as animated characters do, it has technical advantages that are particularly useful and necessary in today's non-face-to-face era.

1 is a block diagram for explaining a character animation generation system according to an embodiment of the present invention;
2 is a diagram illustrating a plurality of gesture animations expressing motions of a character according to a character's emotion classification according to an embodiment;
Fig. 3 is a flow diagram illustrating an exemplary method for generating animations expressing motions and emotions of a character based on voice and facial expressions, according to one embodiment;
4 is a diagram for explaining a process of generating animations expressing motions and emotions of a character according to an embodiment;
5 is a diagram for explaining a character animation generation system in which a wireless care device is integrated;
6 is a diagram for explaining a wireless care method according to an embodiment;
7 is a diagram illustrating a method of estimating a type of a wireless device according to an exemplary embodiment.

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content will be thorough and complete and the spirit of the present invention will be sufficiently conveyed to those skilled in the art.

In this specification, when terms such as first and second are used to describe components, these components should not be limited by these terms. These terms are only used to distinguish one component from another. Embodiments described and illustrated herein also include complementary embodiments thereof.

In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. The expressions 'comprises', 'consists of', and 'consists of' used in the specification do not exclude the presence or addition of one or more other elements in addition to the mentioned elements.

In this specification, the term 'software' refers to technology that moves hardware in a computer, and the term 'hardware' refers to a type of device or device (CPU, memory, input device, output device, peripheral device, etc.) constituting a computer, , the term 'step' means a series of processes or operations connected in time series to achieve a predetermined goal, and the term 'computer program', 'program', or 'algorithm' refers to a set of instructions combined for computer processing and the term 'program recording medium' refers to a computer-readable recording medium on which a program used to install, execute, or distribute a program is recorded.

Terms such as '~unit', '~module', '~unit', '~block', and '~board' used to refer to the components of the invention in this specification refer to those that process at least one function or operation. It may refer to a physical, functional, or logical unit, which may be implemented as one or more hardware, software, or firmware, or may be implemented as a combination of one or more hardware, software, and/or firmware.

In this specification, 'processing device', 'computer', 'computing device', 'server device', and 'server' refer to an operating system such as Windows, Mac, or Linux, a computer processor, memory, application programs, and storage devices (eg For example, HDD, SDD), and may be implemented as a device having a monitor. The computer may be, for example, a device such as a desktop computer, a laptop computer, or a mobile terminal, but these are examples and are not limited thereto. The mobile terminal may be one of mobile wireless communication devices such as a smart phone, a tablet PC, or a PDA.

In this specification, the transmission of information, details, and/or data from element 'A' to element 'B' means that element 'A' transmits directly to element 'B' or that element 'A' transmits information directly to element 'A'. is used in the sense of including transmission to component 'B' through at least one other component.

According to the Animation Industry Promotion Act, animation is "creating a moving image by processing a subject that does not move by itself in the real or imaginary world using various techniques and media such as 2D, 3D, CG, and stop motion." defined as "video". In this specification, "animation" includes this definition and is interpreted as encompassing a motion picture in which at least a part of the body of a real character or a virtual character moves. Also, in this specification, 'animation' is also referred to as 'animation video', 'animation video', 'video', 'image', and the like.

The 'character' referred to in this specification refers to a visual symbol such as a real or fictitious person or animal appearing in an animated image, and for example, a fictional character may mean a main character or avatar of various cartoons, etc. .

A 'gesture' of a character referred to in this specification refers to at least a part of the body such as a character's gesture or hand gesture within an animation. A gesture may simply mean the movement of a part of a character's body, or it may be an expression of a character's inner psychological state or a signal to a certain object. In this specification, gestures are used as the same as 'movement' or 'movement' unless otherwise specified. In addition, sometimes in this specification, 'gesture' may also mean the movement of a character's facial expression or the movement of at least a part of a face such as the eyes, nose, and mouth of a character's face.

The present invention will be described in detail with reference to the following drawings. In describing the specific embodiments below, various specific details have been prepared to more specifically describe the invention and aid understanding. However, readers who have knowledge in this field to the extent that they can understand the present invention can recognize that it can be used without these various specific details. In addition, in describing the invention, it is mentioned in advance that parts that are well-known or commonly used technologies but are not significantly related to the invention are not described in order to prevent confusion in describing the present invention.

1 is a block diagram for explaining a character animation generating system (hereinafter, simply referred to as an “animation generating system” or “system”) according to an embodiment of the present invention. Referring to FIG. 1 , an animation generating system 100 according to an embodiment may include a voice and facial expression analysis module 110 , a visual manager 120 , and an animation archive 130 .

The voice and facial expression analysis module 110 (hereinafter, simply referred to as 'analysis module') may generate phrase data and character data by receiving a voice spoken by a user and analyzing the voice. In an alternative embodiment, the analysis module 110 may generate phrase data and character data by receiving and analyzing a voice uttered by a user and a facial expression of the user.

In one embodiment, the analysis module 110 includes an STT analysis unit 111, a syntax analysis unit 112, a voice analysis unit 113, and a character data generation unit 114. In an alternative embodiment, the analysis module 110 may further include a facial expression analysis unit 115 .

The analysis module 110 may include a voice receiving device such as a microphone in order to receive a voice signal uttered by a user. In addition, in one embodiment, an image capture device such as a camera may be additionally provided to capture and receive a user's facial expression.

The voice signal received by the analysis module 110 may include a predetermined sentence uttered by the user. Here, a 'sentence' does not necessarily include a subject and a verb, and may be a set of one or more words. A voice signal input through a voice receiving device such as a microphone is transmitted to a speech-to-text (STT) analyzer 111 . The STT analyzer 111 is a functional unit that interprets a voice signal and converts it into text data.

The text data generated by the STT analyzer 111 is transferred to the syntax analyzer 112 . The syntax analyzer 112 may generate syntax data by analyzing sentences in the text data by word and/or by syntax.

In one embodiment, 'syntax data' may include a sentence uttered by a user (ie, text data converted from a sentence to text) and parsed data for the sentence (hereinafter referred to as 'meta data'). For example, the meta data may include action information indicating actions corresponding to (matching) each word and/or phrase constituting the sentence, and emotion information indicating emotion corresponding to (matching) each word and/or phrase. can include For example, when a sentence such as 'I'm so happy' is included among the sentences uttered by the user, the syntax analysis unit 112 responds to the word 'joy' by inserting an 'action to display joy' into metadata as motion information. can include In addition, the syntax analysis unit 112 may match the emotion of 'joy' or 'happiness' from the sentence 'so happy', and accordingly, the meta data determines 'joy' or 'happiness' among the types of mood-classified emotions. As information, it can be included in meta data.

In one embodiment, the syntax analyzer 112 may analyze the sentence based on the morpheme, and accordingly, the meta data is tense information about the tense of the verb (eg, information for determining whether it is future, perfect, subjunctive, etc.) may further include. For example, if the sentence uttered by the user includes the word '~ did', it can be determined that the corresponding action has already been completed from the morpheme '~ did', and as another example, the sentence contains the word '~ did'. If included, it can be determined that the corresponding action has not yet been performed through morphological analysis. As another example, when the sentence includes the word 'will do', the syntax analyzer 112 may determine that the corresponding operation has not been performed.

In one embodiment, the syntax analyzer 112 generates motion information so that a character performs an motion corresponding to (matching) each word and/or phrase constituting a sentence. However, depending on the content or nuance of the sentence, the character does not necessarily perform an action consistent with the word (verb) included in the sentence. For example, if the sentence uttered by the user is future tense, such as 'I should have run', or subjunctive, such as 'I would have run', the character does not necessarily perform a running motion, so the syntax analysis unit 112 ' In response to the word 'run', the motion information of the running motion may not be generated (eg, in this case, 'action to think' or 'action to regret' may be generated as motion information rather than 'action to back', Furthermore, in this case, emotions such as 'regret' or 'sadness' may be generated as emotion information), and the syntactic analysis unit 112 may generate motion information and/or emotion information based on the overall meaning or nuance of the sentence. desirable. Therefore, in an embodiment in which the meta data further includes tense information, the syntax analyzer 112 performs an operation corresponding to each word or phrase based on not only the word or phrase of the sentence but also the tense information and the context or nuance of the sentence. Information and/or emotional information may be generated.

The analysis module 110 includes a voice analysis unit 113 to generate information about the emotion of the user who utters the sentence. The voice analysis unit 113 receives the voice spoken by the user, analyzes it according to one or more voice signal analysis criteria such as tone, pitch, and speech speed of the voice, and determines the emotional state of the user (speaker) according to the analysis result. Recognizable.

In order for the voice analyzer 113 to recognize the user's emotion, various conventionally known methods may be applied. For example, the voice analysis unit 113 may consider a change in the frequency of a spoken voice, a speaking speed, and the like. For example, a user's emotional state may be detected by measuring a frequency change (or a change in pitch of a syllable in a voice) of a voice uttered by the user. In one embodiment, the voice analysis unit 113 may recognize one emotional state for all sentences uttered by the user. In a more preferred embodiment, the voice analyzer 113 may analyze an emotional state corresponding to a corresponding word or phrase for each word and/or phrase in a sentence.

The analysis module 110 may further include a facial expression analyzer 115 to generate information about the user's emotion. The facial expression analysis unit 115 may receive, for example, a user's facial expression captured by a camera and recognize the user's emotion therefrom. Recognizing emotions from facial expressions is a technique known in the art, and is disclosed in, for example, Korean Patent Registration No. 10-1317047 (Emotion Recognition Apparatus Using Facial Expressions and Control Method Thereof).

The character data generation unit 114 is a functional unit that generates data (hereinafter also referred to as 'character data') representing characteristics to be given to a character (hereinafter also referred to as 'character characteristics'). In an embodiment, the character characteristics may include at least one of the character's emotion, gender, age, and appearance, and thus the character data may include at least one of emotion data, gender data, age data, and appearance data. .

The character data generation unit 114 receives the syntax data generated by the syntax analysis unit 112 and the emotion information determined by the voice analysis unit 113 and the expression analysis unit 115, and character data based on at least one of them. can create

As an example, punctuation marks or words and/or phrases of sentences included in the syntax data may contribute to the emotion determination. For example, if a specific word (for example, exclamation mark, onomatopoeia, mimetic word, etc.) representing an emotion is included in the sentence, it may be determined that the emotion is joy, sadness, or surprise in response thereto. Therefore, the character data generation unit 114 generates corresponding emotions in units of words or units of sentences based on the syntax data and the emotion information analyzed by the voice analysis unit 113 and the facial expression analysis unit 115. You can choose. At this time, the types of emotions are classified into joy, sadness, anger, surprise, fear, and the like, and one emotion may be matched for each word and/or each phrase of the sentence. In an alternative embodiment, one or more emotions may be complexly matched for each word and/or each phrase of a sentence, and in this case, a weight is applied to a plurality of emotions matched to one word and/or phrase. Complex emotions can be matched.

The character data generation unit 114 determines the gender based on at least one of the syntax data generated by the syntax analysis unit 112, the voice information determined by the voice analysis unit 113, and the facial expression obtained by the expression analysis unit 115. data, age data, and appearance data may be generated.

For example, the gender and age of the character can be determined by analyzing the user's voice and/or the user's face, and additional or auxiliary gender and/or age determination can be made by analyzing words or phrases used in sentences uttered by the user. can be used as a standard. The character's appearance is information about the character's appearance, such as whether the character has a fat body or a skinny body, and can be determined by, for example, analyzing the user's voice and/or the user's face.

In this way, the character data generation unit 114 determining the emotion, gender, age, and appearance of the character to generate the character data can be implemented using, for example, a deep learning algorithm. That is, by machine learning an artificial neural network using various voice signals and sentences as learning data, information on emotion, gender, age, and appearance can be output for an arbitrary voice signal. In addition, those skilled in the art will understand that each of the above-described components 111 , 112 , 113 , 114 , and 115 of the analysis module 110 may be implemented with one or more algorithms or software, or may be implemented with a combination of software, hardware, or firmware.

The animation archive 130 may store a plurality of animation videos. In one embodiment, the animation archive 130 includes a plurality of gesture videos and a plurality of face videos. Each gesture video is a short length video that expresses each different motion that a character can take. For example, each gesture video expresses basic motions such as running, sitting, and raising a hand, or motions according to emotions such as joy, sadness, and anger. For example, FIG. 2 shows a part of a gesture video expressing a character's motion according to the character's emotion classification according to an embodiment, as an example. In the illustrated embodiment, the types of emotions are classified into five types of surprise, happiness, anger, fear, and sadness, and each emotion includes a gesture video expressing one or more basic actions and a gesture video expressing one or more additional actions. can Although emotions are classified into five types in FIG. 2 , this is only exemplary and the number of emotion categories may vary according to specific implementation conditions of the invention.

Identification information (eg, identification number or meta data) or labels may be assigned to each gesture video, and one or more motions and/or emotions may be matched with identification information or labels of one or more gesture videos.

Each face video is a short video that expresses different facial expressions a character can take. Each face video expresses facial expressions according to basic emotional expressions such as laughing, crying, frowning, and getting angry. For example, a video of a face with a sad expression can express the tip of the eyebrows and the corners of the mouth going down, and a video of a happy expression can express the raising of the eyebrows and the corners of the mouth. Identification information or labels are assigned to each face video, and one or more emotions may be matched with identification information or labels of one or more face videos.

In one embodiment, the animation archive 130 may include gesture videos and/or face videos of various motions according to character characteristics of character data. For example, even for one motion of 'run', the running motion may be different depending on the character's gender, age, or appearance, and for emotional expressions such as 'laughing' or 'crying', depending on the character's gender, age, or appearance Each action may be different. Therefore, it is preferable to provide various and subdivided gesture videos and/or face videos according to character characteristics for one motion or emotion.

Identification information or labels assigned to each gesture video and/or face video may be assigned by a user, but may be automatically assigned by a machine learning algorithm such as artificial intelligence (AI). For example, a machine learning algorithm may display identification information or a label for an arbitrary gesture or face video after learning by using a gesture video or face video with identification information or a label as learning data.

As described above, each gesture video and/or face video is a short unit video representing a basic motion or emotion, and has a length of, for example, between one minute and several seconds.

In one embodiment, each gesture video and/or face video may be a video in which a specific character is not yet put on. For example, a gesture video may be a video defining movement of body feature points such as joints and bones, and a face video may be a video defining movement of facial feature points such as eyes, nose, and mouth. These gestures and/or faces It will be understood that the retargeting task of applying the appearance of a specific character to the video can be performed immediately before or after generating the character animation video according to the present invention.

The visual manager 120 is a functional unit that generates an animation image by applying motions and facial expressions according to a voice signal uttered by a user to a character. In one embodiment, the visual manager 120 receives the syntax data and character data generated by the analysis module 110, selects an appropriate gesture animation and facial animation from the animation archive 130 based on this, and selects the selected gesture animation and character data. Character animations are created by blending facial animations according to syntax data and character data.

In one embodiment, the visual manager 120 includes a gesture blending unit 121 and a face blending unit 122. The gesture blending unit 121 selects an appropriate gesture video from the animation archive 130 based on the syntax data and character data received from the analysis module 110 . For example, the gesture blending unit 121 may select one or more gesture videos based on motion information of words and/or phrases of phrase data and character characteristics (at least one of gender, age, emotion, and appearance) of character data. One temporally connected animation may be created by blending gesture videos corresponding to words and/or phrases in the chronological order of utterance.

For example, the face blending unit 122 may select one or more face videos based on emotion information of words and/or phrases of phrase data and characteristics (at least one of gender, age, emotion, and appearance) of character data, Changes in facial expressions corresponding to words and/or phrases in the utterance time sequence are blended with the gesture video. In addition, in one embodiment, the face blending unit 122 may also perform lip blending (lip sync) to match the shape of the character's lips to the sentence of the syntax data. For example, the face blending unit 122 may perform lip blending by matching the timing of the utterance of each vowel of each word according to the sentence (text information) of the syntax data and the mouth shape of the character.

Referring now to FIG. 3, exemplary operation of the character animation generation system 100 will be described. Referring to FIG. 3, in step S10, a voice signal uttered by a user is received through a voice receiving device such as a microphone (not shown). For example, in the embodiment of FIG. 3 , it is assumed that the user utters the sentence "Hello. Nice to meet you."

In step S20, the sentence is parsed by the STT analysis unit 111 and the syntax analysis unit 112 of the analysis module 110 to generate syntax data. For example, the STT analyzer 1110 converts the voice signal uttered by the user into text data of 'Hello, nice to meet you', and the syntax analyzer 112 parses the sentence to provide operation information according to each word or phrase ( Meta data including, for example, greeting motion, hand waving motion, etc.) and emotion information (eg joy, surprise, etc.) may be generated.

In addition, simultaneously or sequentially with step S20, in step S30, the voice analysis unit 113 of the analysis module 110 analyzes the user's voice signal (for example, when there is a photographing device such as a camera), and the expression analysis unit 115 analyzes the user's facial expression. Information according to each analysis is transmitted to the character data generator 114, and the character data generator 114 generates character data including at least one of character characteristics of gender, age, emotion, and appearance based on the information. can Alternatively, the character data generator 114 may generate character data by referring to sentences (text information) included in the syntax data, motion information, and emotion information.

Syntax data and character data are transferred to the visual manager 120, and in step S40, the visual manager 120 matches one or more gestures to the syntax data and character data in the animation archive 130 in which a plurality of gesture videos are stored. Choose a video. For example, the visual manager 120 may select one or more gesture videos based on one or more words or phrases included in sentences of phrase data and character features included in character data.

At this time, in one embodiment, the visual manager 120 may select a gesture video by additionally using the tense information of the syntax data. For example, if it is determined that the sentence uttered by the user is future tense or subjunctive, an action other than an action directly indicated by the corresponding word or phrase may be selected.

Simultaneously or sequentially with step S40, in step S50, the visual manager 120 selects one or more face videos matching the phrase data and character data from the animation archive 130 in which a plurality of face videos are stored. For example, the visual manager 120 may select one or more face videos based on one or more words or phrases included in sentences of phrase data and character features included in character data.

After that, the visual manager 120 blends the selected one or more gesture videos and one or more face videos (S60). In this step (S60), lip blending may also be performed in which the shape of the lips of the character is lip-synced according to the sentence uttered by the user. Next, in step S70, additional operations such as retargeting may be performed to finally create a character animation video in which gestures and faces are blended with the character.

4 shows another example of generating an animation expressing a motion and emotion of a character according to an embodiment. For convenience of description, some steps (S20, S30, S60, S70) of FIG. 3 are shown in FIG.

Referring to FIG. 4, when a user utters a specific sentence, the STT analysis unit 111 and the syntax analysis unit 112 of the analysis module 110 perform STT analysis and syntax analysis to generate syntax data (S20) and voice The analysis unit 112 and the facial expression analysis unit 115 analyze the voice signal and the facial expression, respectively, to generate character data (S30).

Syntax data and character data are transferred to the gesture blending unit 121 and the face blending unit 122 of the visual manager 120, respectively, and the visual manager 120 in the animation archive 130 based on the syntax data and character data. Home Select and load one or more gesture videos and face videos as appropriate.

Then, the visual manager 120 blends the selected one or more gesture videos and face videos (S60) and creates a character animation video (S70). At this time, for example, assuming that the user utters the sentence "Uh... Wait a minute! Do you think I'm an idiot?", the visual manager 120 calculates 'Uh... Each gesture video is created by matching the action of thinking in response to the phrase “Wait!”, matching the action of pointing to oneself for the word “I”, and matching the action of getting angry for the phrase “Do you think I’m an idiot?” select and call

In addition, the visual manager 120, based on the syntax data and the character data, 'uh... Each face video is selected and loaded by matching a facial expression thinking for the phrase 'Wait!' and matching an angry facial expression for the phrase 'I think I'm an idiot!'.

After that, the visual manager 120 blends the selected gesture video temporally (ie, according to the sentence utterance time), and also temporally (according to the sentence utterance time) and spatially (ie, synthesized with the gesture) the face video. Blend to create character animations. At this time, the visual manager 120 may perform lip sync by blending lip movements according to the utterance timing of each word in the sentence.

As described above, according to one embodiment of the present invention, character animation can be created by applying appropriate gestures and facial expressions corresponding to the voice signal and facial expression uttered by the user to the character, so that various emotional changes can be expressed temporally within a sentence. Gestures can also be changed accordingly.

Now, referring to FIGS. 5 to 7 , a system in which the character animation generating system 100 according to an embodiment of the present invention and the wireless care device are integrated will be described.

Referring to FIG. 5 , a character animation generating system 100 in which a wireless care device 200 according to an embodiment of the present invention is integrated is shown in a block diagram. Here, the configuration and operation of the animation generating system 100 have been described with reference to FIGS. 1 to 4, and hereinafter, differences will be mainly described.

Referring to FIG. 5 , video data generated by the animation creation system 100 may be stored in the server 300 via a local area network N2 and a wide area communication network N1. Here, the local area network N2 may be a wireless local area network, and may be, for example, a wireless local area network supporting Wi-Fi or Bluetooth communication. The wide area communication network N1 may be a communication network such as the Internet.

The wireless care device 200 may be installed within a predetermined distance (eg, 1 kilometer) from a sound or text recognition based video character automatic generation system. Here, the predetermined distance is a distance at which short-distance wireless communication with a sound or text recognition-based automatic video character generation system is possible.

The wireless care device 200 may detect a wireless signal of a wireless device and determine whether the detected wireless device is a fake device.

Specifically, the wireless care device 200 performs a type estimation operation of estimating the type of the wireless device and an operation of determining whether the wireless device is a fake device by attacking the wireless device with a vulnerability according to the type of the wireless device. .

Here, the type estimation operation is an operation of estimating the type of the wireless device according to the results of the port scanning operation and the protocol scanning operation.

The port scanning operation is an operation to find an open port of a wireless device, and the protocol scanning operation is an operation to find a protocol used in an open port found as a result of performing the port scanning operation.

The protocol scanning operation includes an operation of checking the type of open port, an operation of creating a packet for protocol confirmation according to the type of open port and transmitting it to a wireless device having an open port, and an operation of transmitting the packet to the wireless device having an open port This is an operation to check whether a response to the packet for protocol verification is received from Here, the response includes at least one of banner information and service information of a wireless device having an open port, and the banner information or service information includes information indicating a type of wireless device.

The type estimation operation includes a first estimation operation of estimating the type of service from the protocol found as a result of performing the protocol scanning operation, a second estimation operation of estimating the type of service from information indicating the type of wireless device, and the first estimation operation. An operation of comparing the type of service estimated by the second estimation operation and the type of service estimated by the second estimation operation, and a wireless device having an open port from information indicating the type of the wireless device when both comparison results are different. Performs an operation of estimating the type of .

The server 300 is communicatively connected to the automatic character motion generation system described with reference to FIG. 1 and physically includes one or more devices or storage devices, operating systems, firmware, application programs, communication units, and other resources, respectively, or some of them. Can be configured to share resources.

In FIG. 5, wireless devices 15 (15a, 15b, 15c, 15d) are additionally shown for explanation of the present system.

The wireless care device 200 may detect a dangerous device capable of hacking communication between the server 300 and the animation generating system 100 (hereinafter referred to as a 'fake device') and block the communication. Hereinafter, the operation of the wireless care device 200 will be described in more detail.

The wireless care device 200 may monitor communication of the wireless device 15 and may also perform short-range wireless communication. For example, the wireless device 15 may perform communication according to a short-range wireless communication technology standard such as Bluetooth or Wi-Fi. The wireless device 15 is a device that may adversely affect the system 200, and the wireless care device 200 may select a fake device from the wireless device 15. The wireless device 15 may be a device such as a smart phone or a laptop capable of wireless communication, but is not limited thereto.

The wireless care device 200 may perform a type estimation operation and a fake device selection and blocking operation.

As will be described later, the communication unit included in the wireless care device 200 (see FIG. 6 and its description) can detect wireless signals output by the wireless device 15. Here, the wireless signal may be, for example, a signal output for Bluetooth communication or a signal output for Wi-Fi communication. The wireless care device 200 may select one or more devices as a target device according to a predetermined criterion among devices detected by the communication unit. Here, the predetermined criterion may be signal strength.

As will be described later, the diagnosis module included in the wireless care device 200 can, for example, estimate the type of the target device 15 and also determine whether the target device 15 is a fake device.

In this embodiment, the number of wireless devices 15 is shown as four each, but this number is illustrative and may be less or more than this number. Hereinafter, assuming that the wireless device 15 is a target device, a type estimation operation, a fake device selection operation, and a blocking operation will be sequentially described.

type estimation behavior

This is an operation of estimating the type of the target device 15. In this embodiment, the diagnosis module may perform a port scanning operation, a protocol scanning operation, and a type assumption operation.

A port scanning operation is to find an open port of the target device 15 . That is, the port scanning operation is an operation of checking which ports are open for each of the target device 15a, the target device 15b, the target device 15c, and the target device 15d.

The port scanning operation may be performed by techniques called, for example, full scanning or stealth scanning. Full scanning is a technique that establishes a complete TCP session and checks open ports. Stealth scanning is a kind of half scan technology. A packet for port checking is transmitted and when a response to the port checking packet is received, the responded port is open. If no response is received, the port is closed. judge it to be Stealth scanning can be, for example, FIN, NULL, or XMASH.

A port is a logical unit that identifies a network service or a specific process, and a protocol using a port is, for example, a transport layer protocol. Examples of transport layer protocols may be Transmission Control Protocol (TCP) and User Datagram Protocol (UDT). Ports are identified by numbers, and these numbers are called port numbers. For example, port numbers are used with IP addresses.

Port numbers can be classified into three types, for example.

If the port number is from 0 to 1023, it is a well-known port, if the port number is from 1024 to 49151, it is a registered port, and if the port number is from 49152 to 65535 If it belongs to , it is a dynamic port. On the other hand, representative examples of well-known ports include: 20: FTP (data), 21: FTP (control), 22: SSH, 23: Telnet, 53: DNS, 80: World Wide Web HTTP, 119: NNTP, 443: TLS/SSL HTTP. Those skilled in the art will readily appreciate that these port numbers and port types are exemplary.

For purposes of explanation of the present invention, the open port of target device 15a is port 80, the off port of target device 15b is port 23, the open port of target device 15c is port 5555, , it is assumed that the open port of the target device 15d is port 5559.

A port scanning operation is an operation of finding an open port of the target device 15 . For example, the diagnostic module performs a port scanning operation, the open port of the target device 15a is port 80, the off port of the target device 15b is port 23, and the open port of the target device 15c is port 23. It finds out that it is port 5555 and that the open port of the target device 15d is port 5559.

A protocol scanning operation is an operation for knowing the type of protocol used in an open port. Here, the open port is found by port scanning operation. For example, the diagnosis module searches for a protocol used in port 80, which is an open port of the target device 15a.

In addition, the diagnosis module determines which protocol is used in the port 23 of the target device 15b, the protocol used in the port 5555 of the target device 15c, and the protocol used in the port 5559 of the target device 15d. Find each cognition.

The protocol scanning operation performed by the diagnosis module includes an operation to check the type of open port, an operation to create a packet to be sent to the open port according to the type of open port (hereinafter referred to as 'protocol checking packet'), and an operation to check the protocol. An operation of transmitting to the wireless device having the open port, and an operation of confirming whether a response is received from the wireless device that has transmitted the packet for protocol confirmation.

A packet for protocol confirmation may be, for example, a script.

The operation of checking the type of the open port is an operation of checking the type of the open port found through the port scanning operation. For example, the operation of checking the type of open port is an operation of checking whether the open port corresponds to a well-known port, a registered port, or a dynamic port. am. In order to perform the operation of checking the type of open port, data (eg, <Table 1>) in which the type of port is classified according to the port number (hereinafter referred to as 'port type data') must be prepared in advance. Such 'port type data' may be stored and managed by the wireless care device 200 .

According to one embodiment, the diagnosis module can know the type of the open port by referring to 'port type data'. For example, in the diagnosis module, port 80 of the target device 15a and port 23 of the target device 15b are well-known ports, and port 5555 of the target device 15c and the target device 15c are well-known ports. It can be seen that port 5559 in (15d) is a dynamic port.

The diagnosis module performs an operation of creating a packet for checking a protocol suitable for the type of open port.

For example, since port 80 of the target device 15a is a well-known port for using the World Wide Web HTTP protocol, a packet for protocol confirmation is created using the Web HTTP protocol, so that the target device ( 15a). If there is a response from the target device 15a to the protocol check packet written using the web HTTP protocol, the diagnosis module determines that port 80 of the target device 15a uses the web HTTP protocol.

On the other hand, if there is no response from the target device 15a to the packet for protocol confirmation written using the web HTTP protocol, the diagnosis module determines that port 80 of the target device 15a does not use the web HTTP protocol. In this case, the diagnosis module uses a protocol other than the World Wide Web HTTP protocol to create a packet for protocol confirmation and transmits the packet to the target device 15a.

If there is a response from the target device 15a to the packet for protocol confirmation written in a protocol other than the web HTTP protocol, the diagnosis module determines that port 80 of the target device 15a uses the other protocol. If there is no response from the target device 15a to the protocol confirmation packet written in the other protocol than the web HTTP protocol, the diagnosis module determines that port 80 of the target device 15a does not use the other protocol. . Thereafter, the target device 15a creates and transmits a protocol confirmation packet using another protocol until a response comes from the target device 15a. The diagnosis module finds out the type of protocol actually used in each open port of the target device 15 by the above-described method.

The diagnostic module performs a type estimation operation for estimating each type of the target device 15 by using at least one result of performing the above-described port scanning operation and performing the protocol scanning operation. (Type Assumption).

According to one embodiment, the type assumption performed by the diagnostic module includes an operation of estimating the type of service from the type of protocol found as a result of performing the protocol scan operation, and the type of service estimated as such. and estimating the type of wireless device.

The operation of estimating the type of service from the type of protocol is usually based on the experience that a mainly used protocol is determined for each service. For example, RTSP, RTP, or RTCP protocols mainly support streaming services. That is, when data defining services mainly supported for each protocol (hereinafter referred to as 'protocol-service mapping data') is prepared, the diagnostic module determines the type of service from the type of protocol by using the 'protocol-service mapping data'. 'Protocol-service mapping data' can be stored and managed by the wireless care device 200.

The operation of estimating the type of the target device 15 from the type of service is also based on experience that a service mainly used for each type of wireless devices is usually determined. For example, RTSP, RTP, or RTCP protocols mainly support streaming services, and these streaming services are provided by wireless devices such as IP TVs, for example. That is, when data defining the types of wireless devices mainly provided for each service (hereinafter referred to as 'service-type mapping data') is prepared, the diagnostic module uses the 'service-type mapping data' to determine the type of service The device type may be estimated, and 'service-type mapping data' may be stored and managed by the wireless care device 200.

Meanwhile, the response to the protocol confirmation packet may include at least one of banner information and service information of the target device 15 .

Banner information included in the response to the protocol confirmation packet typically includes data indicating what type of operating system is used in the target device 15 . The diagnostic module may know the type of service or estimate the type of the target device 15 if the type of operating system is known.

Service information included in the response to the protocol confirmation packet typically includes data indicating what type of target device is. For example, the service information may include data such as 'My iPhone', and this data is information directly indicating the type of the target device. According to an embodiment, the type assumption performed by the diagnosis module is performed using at least one of banner information and service information of the target device and a result of performing the port scanning operation described above. At least one of the performance results of the protocol scanning operation and the protocol scanning operation is used.

For example, the type assumption performed by the diagnostic module includes a first estimation operation, a second estimation operation, a comparison operation, and a type determination operation.

The first estimation operation is an operation of estimating the type of service from the type of protocol obtained through the protocol scanning operation. The second estimation operation is an operation of estimating the type of service by using at least one of banner information and service information of the target device 15 .

The comparison operation is an operation of comparing the service type estimated by the first estimation operation with the service type estimated by the second estimation operation.

The type determination operation determines the target device 15 from the service type estimated by the second estimation operation when the service type estimated by the first estimation operation and the service type estimated by the second estimation operation are different from each other. Determines the type of and, when the type of service estimated by the first estimation operation and the type of service estimated by the second estimation operation are the same, the type of service estimated by the first estimation operation or the second estimation operation This is an operation of estimating the type of target devices 15 from

On the other hand, when there is no banner information and service information of the target device 15, or there is no data for estimating the type of service in such banner information and service information, the type assumption operation , includes a first estimation operation and a type determination operation. That is, the type estimation operation may be performed only with the first estimation operation and the type determination operation without performing the second estimation operation and the comparison operation.

The fake device selection operation is an operation of selecting a fake device from among the target devices 15 . In this embodiment, the diagnosis module determines whether each of the target devices 15 are fake devices. For example, the diagnostic module determines whether the target device 15a is a fake device among the target devices 15, determines whether the target device 15b is a fake device, and determines whether the target device 15c is a fake device. It is possible to determine whether the device is a fake device for the target device 15d, and to determine whether the device is a fake device for the target device 15d. Here, the order is exemplary, and a person engaged in the technical field to which the present invention belongs (hereinafter, 'one skilled in the art') may determine the order to suit the situation when practicing the present invention.

The diagnosis module attacks vulnerabilities according to the type of target device to determine whether the target device is a fake device, and the diagnosis module determines whether the target device is a fake device according to whether the attack on the target device succeeds. judge Typically, vulnerability in a wireless device means a weakness that an attacker can use to lower the information assurance of a device.

For example, when the target device 15a is a 'target device', the diagnosis module determines whether the target device 15a is a fake device according to whether an attack on a vulnerability of the target device 15a succeeds. can That is, the diagnostic module determines that the target device 15a is not a fake device if the attack on the vulnerability of the target device 15a succeeds, and if the attack on the vulnerability of the target device 15a does not succeed, the target device 15a is not successful. It is determined that (15a) is a fake device.

The diagnostic module selects a vulnerability according to each type of target device 15, attacks each of the target devices 15 using the selected vulnerability, and fake devices among the target devices 15 based on the attack result. can be selected.

The diagnosis module according to another embodiment of the present invention estimates each type of the target device 15, attacks a vulnerability corresponding to each estimated type, and among the target devices 15 based on the attack result. A fake device can be determined.

In the above-described embodiments, the diagnostic module may generate a vulnerability attack message according to the type of the target device 15 by referring to data ('vulnerability data by type') to which vulnerabilities are mapped by type. Vulnerability data for each type may be stored and managed by a storage device (to be described later) provided in the wireless care device 200 . Alternatively, vulnerability data for each type may be stored and managed in an external storage device (not shown) connected communicatively so that the wireless care device 200 can access it.

In the above-described embodiments, the operation of selecting a vulnerability according to the type of the target device 15 may be an operation of finding a vulnerability corresponding to the type of the target device by referring to vulnerability data for each type.

In the above-described embodiments, the diagnostic module may generate a vulnerability attack message according to the type of the target device 15 by referring to data ('vulnerability data by type') to which vulnerabilities are mapped by type. Vulnerability data for each type may be stored and managed by a storage device (to be described later) provided in the wireless care device 200 . Alternatively, vulnerability data for each type may be stored and managed in an external storage device (not shown) connected communicatively so that the wireless care device 200 can access it.

In the above-described embodiments, the operation of selecting a vulnerability according to the type of the target device 15 may be an operation of finding a vulnerability corresponding to the type of the target device by referring to vulnerability data for each type.

In the above-described embodiments, the operation of attacking the vulnerability of the target device 15 may be, for example, an operation of the diagnostic module transmitting a message for attacking the vulnerability to the target device.

In the above-described embodiments, the diagnosis module determines that the target device is not a fake device when the target device responds to a vulnerability attack, and if the target device does not respond to a vulnerability attack, the target device is a fake device. It can be judged as a device.

Hereinafter, vulnerabilities possessed by wireless devices and messages for attacking vulnerabilities will be exemplarily described.

For example, the target device may be a router having the following types of vulnerabilities.

. Vulnerability: An unprivileged attacker could access the "/category_view.php" URL with the HTTP GET method

. Message for vulnerability attack: GET/category_view.php

For another example, the target device may be a router with the following types of vulnerabilities.

ㆍVulnerability: Unprivileged attacker can access "/mydlink/get_TriggeredEventHistory.asp" URL with HTTP GET method

. Message for vulnerability attack: GET/mydlink/get_TriggeredEventHistory.asp

As another example, the target device may be a router with the following types of vulnerabilities.

. Vulnerability: An unauthorized attacker can obtain information such as the PIN and MAC address of the target device by accessing the "/router_info.xml?section=wps" URL using the HTTP GET method.

. Message for vulnerability attack: GET/router_info.xml?section=wps

As another example, the target device may be a router with the following types of vulnerabilities.

. Vulnerability: An unprivileged attacker could inject script statements into data when accessing the "/wpsacts.php" URL with the HTTP POST method.

. Message for vulnerability attack: GET//wpsacts.php

According to embodiments of the present invention, target devices are treated as different types if they have different vulnerabilities even if they have the same kind of function. For example, the aforementioned routers have different vulnerabilities and thus are distinguished as different types of target devices. For another example, if the manufacturer of the target device is the same and the firmware ware is different even if the device is of the same type, it is treated as a different type.

According to an embodiment of the present invention, 'vulnerability data by type' may be data corresponding to vulnerabilities for each type of target device. When such vulnerability data for each type is stored and managed, the diagnostic module refers to the vulnerability data for each type to attack the vulnerability of the target device.

Alternatively, 'vulnerability data by type' may be data to which 'messages for attacking weaknesses' are mapped to each type of target device.

According to another embodiment of the present invention, vulnerability data for each type may not be prepared in advance. In this embodiment, the diagnostic module may directly generate a vulnerability attack message corresponding to the type of the target device (eg, if the vulnerability attack unit is configured to directly generate a vulnerability attack message for each type).

The blocking operation is an operation of blocking the communication of the fake device. The blocking operation is an operation of blocking communication of the fake device by performing a DDoS attack on the fake device.

For example, the diagnosis module transmits a large amount of messages to the fake device, and the fake device that receives the large number of messages falls into an inability to communicate with other devices while responding to such messages. In this way, the diagnosis module may block communication of a target device having a vulnerability.

For example, assuming that the wireless device 15a is a fake device, the diagnosis module blocks communication of the Bluetooth device by performing a DDoS attack on the fake device 15a. That is, the diagnosis module transmits a large number of messages to the fake device 15a, so that the fake device 15a receiving the large amount of messages cannot communicate with other Bluetooth devices while responding to such messages. As such, the diagnosis module may block communication only with the fake device 15a.

Hereinafter, an exemplary configuration of the wireless care device 200 according to an embodiment of the present invention will be described.

The wireless care device 200 of the present invention includes a port scanning unit, a protocol scanning unit, a type estimation unit, a determination unit, an operating system, a communication unit, a vulnerability attacking unit, a computer processor, a peripheral device, a storage device, and a memory. The components are operatively connected to each other. Meanwhile, a port scanning unit, a protocol scanning unit, a type estimation unit, a determination unit, and a vulnerability attack unit will be collectively referred to as diagnosis modules.

The diagnostic module may perform the above-described type estimation operation, fake device selection operation, and blocking operation. Specifically, the port scanning unit may perform a port scanning operation, the protocol scanning unit may perform a protocol scanning operation, the type estimation unit may estimate a device type, and the determination unit may perform a fake device selection operation and blocking operation. For each of these operations, please refer to the above-mentioned part.

The port scanning unit performs the port scanning operation described above. The port scanning unit creates a port checking packet, transmits the port checking packet to the wireless device 15 through the communication unit, and checks whether a response to the port checking packet is received to determine the port opened in the wireless device 15. Decide.

The protocol scanning unit performs the protocol scanning operation described above. The protocol scanning unit creates a protocol confirmation packet, transmits the protocol confirmation packet to the wireless device 15 through the communication unit, checks whether a response to the protocol confirmation packet is received, and actually uses the protocol in the opened port. decide

The vulnerability attacking unit transmits the vulnerability attacking message to the target device through the communication unit, and the judgment unit checks whether there is a response to the vulnerability attacking message. .

An operating system is software that not only manages hardware but also provides a hardware abstraction platform and common system services to execute application software. A computer processor is a central processing unit (CPU), and such a central processing unit is a control unit of a computer that controls a computer system and executes program operations, or a chip having the function.

The memory and/or storage unit provides a space in which programs are stored or executed, and may also store data necessary for the operation of the present invention, such as protocol-service mapping data or service-type mapping data.

The memory and/or storage device may also temporarily and/or permanently store various data. For example, memory and/or storage may store messages for vulnerability exploitation.

The type estimation unit performs the above-described type estimation operation. According to an embodiment, the type estimating unit includes at least one information of Banner information and Service information of the wireless device 15, and at least one operation result of the port scanning unit operation result and the protocol scanning unit operation result. Use

According to another embodiment, at least one result of an operation result of the port scanning unit and an operation result of the protocol scanning unit is used. Since the detailed description of these embodiments has been described above, it will be omitted.

According to one embodiment, the type estimation result of the type estimation unit is stored in the memory and/or the memory. The type estimation result is data corresponding to a type for each wireless device 15 . Meanwhile, vulnerability data for each type may also be stored in the storage device and/or memory. Vulnerability data for each type (data corresponding to vulnerabilities for each type) is used to find vulnerabilities of the target device type.

The vulnerability attacking unit may refer to vulnerability data for each type to find a vulnerability corresponding to the type of the target device and generate a message for attacking the vulnerability. Then, the vulnerability attack message is transmitted to the target device through the communication unit. After checking whether there is a response to the vulnerability attack message, the determination unit determines that the target device is not a fake device if there is a response, and determines that it is a fake device if there is no response.

6 is a diagram for explaining a wireless care method according to an embodiment of the present invention. A wireless care method according to an embodiment of the present invention is to determine a fake device among target devices (also referred to as 'target devices') close to the system 200 .

Referring to FIG. 6, a wireless care method according to an embodiment of the present invention includes estimating the type of a target device (S100); Selecting a vulnerability according to the type of target device (S300); Attacking the vulnerability selected in step S300 (S400); and determining whether the target device is a fake device according to whether the attack on the target device succeeds (S500).

The wireless care method according to an embodiment of the present invention may further include storing and managing vulnerability data for each wireless device type corresponding to a vulnerability for each type of wireless device (S200).

The step of selecting the aforementioned vulnerability (S300) includes an operation of finding a vulnerability corresponding to the type of the target device by referring to vulnerability data for each wireless device type.

The step of attacking the target device described above (S400) may be a step of generating a vulnerability attack message that attacks the vulnerability selected in the step of selecting a vulnerability according to the type of the target device (S300) and transmitting it to the target device.

In the above-described step of determining whether the target device is a fake device (S500), when the target device responds to the vulnerability attack message, it is determined that the target device is not a fake device, and the target device responds to the vulnerability attack message. If there is no response, it may be a step of determining that the target device is a fake device.

Hereinafter, it is assumed that the wireless care method according to an embodiment of the present invention is applied to the wireless care device 200 described with reference to FIG. 5 .

For the step of estimating the type of the target device ( S100 ), please refer to the description with reference to FIG. 5 .

The step of storing and managing vulnerability data for each type (S200) is a step in which the wireless care device 200 stores and manages the result of the step of estimating the type of the target device (S100) in a recording medium. Meanwhile, it is also possible for a computer (not shown) other than the device 100 to perform step S100.

The step of selecting a vulnerability (S300) is a step in which the wireless care device 200 selects a vulnerability according to the type of the target device. For example, the vulnerability attacker may select a vulnerability according to the type of the target device estimated by the type estimation unit. To this end, vulnerability data for each type is stored in the storage device, and the vulnerability attacking unit can select a vulnerability corresponding to the type of the target device by referring to the vulnerability data for each type.

Attacking the target device (S400) is a step in which the wireless care device 200 attacks the target device's vulnerability. For example, the vulnerability attacker transmits a vulnerability attack message to the target device. The vulnerability attack message is generated based on vulnerability data selected according to the type of the target device. The vulnerability attacking unit may directly generate a message for attacking the vulnerability.

In the step of determining whether the target device is a fake device (S500), for example, the determining unit of the wireless care device 200 determines that the target device is not a fake device when the target device responds to the vulnerability attack message. And, if the target device does not respond to the vulnerability attack message, it may be determined that the target device is a fake device.

7 is a diagram for explaining a method of estimating a type of a wireless device according to an embodiment of the present invention.

Referring to FIG. 7 , the method for estimating the type of a wireless device according to an embodiment of the present invention includes a port scanning step (S110) of finding an open port of a target device, and a port scanning step (S110). A protocol scanning step (S120) to find the protocol used in the open port found as a result of performing the Port Scanning step (S110), based on the results of the above-described scan steps (S110, S120) Thus, a type assumption step (S130) of estimating the type of the target device may be included.

Hereinafter, it is assumed that the method for estimating the type of wireless device according to an embodiment of the present invention is applied to the apparatus 100 described with reference to FIG. 5, and the type of wireless device is estimated according to an embodiment of the present invention. I will explain how to do it.

The port scanning step (S110) is to find an open port of the wireless device 15. That is, the port scanning step (S110) is a step of performing the port scanning operation described with reference to FIG. 2 .

In this embodiment as well, for the purpose of explanation of the present invention, the open port of wireless device 15a is port 80, the off port of wireless device 15b is port 23, and the open port of wireless device 15c is port 23. It is assumed that is port 5555 and the open port of the wireless device 15d is port 5559.

The port scanning step (S110) is to find an open port of the wireless device 15. For example, as a result of performing the port scanning step (S110), the open port of the wireless device 15a is port 80, the off port of the wireless device 15b is port 23, and the open port of the wireless device 15c is is port 5555, and the open port of the wireless device 15d is port 5559.

The protocol scanning step (S120) is a step for knowing the type of protocol actually used in the open port. That is, the protocol scanning step ( S120 ) is a step of performing the protocol scanning operation described with reference to FIG. 2 . Meanwhile, the open port is obtained as a result of performing the port scanning step (S110).

For example, in the protocol scanning step (S120), an operation of finding out the type of protocol used in port 80, which is an open port of the wireless device 15a, is performed. In addition, the protocol scanning step (S120) is a protocol used in port 23 of the wireless device 15b, a protocol used in port 5555 of the wireless device 15c, and a protocol used in port 5559 of the wireless device 5d. Performs an operation to find out which protocol is used. As such, the protocol scanning step ( S120 ) is a step of finding out the types of protocols actually used for all open ports of the wireless device 15 .

The protocol scanning step (S120) includes checking the type of open port, creating a protocol checking packet, transmitting the protocol checking packet to a target device having the open port, and transmitting the protocol checking packet. and checking whether there is a response from one target device. Meanwhile, the step of confirming the type of open port is a step of performing the above-described operation of confirming the type of open port, and the step of creating a packet for protocol confirmation is the step of performing the above-described operation of creating a packet for confirmation of protocol. , and the step of checking whether a response is received from the target device is a step of performing an operation of checking whether a response is received from the target device. Therefore, a more detailed description of these steps will be omitted.

In the step of checking the type of open port, it is checked whether the open port obtained by the port scanning operation corresponds to a well-known port, a registered port, or a dynamic port. perform the action

According to one embodiment, the step of checking the type of open port. By referring to the port type data, an operation of finding out the type of the open port acquired by the port scanning operation is performed.

According to one embodiment, the step of creating a packet for protocol confirmation performs an operation of creating a packet for protocol confirmation according to the type of open port.

For example, in the step of creating a packet for protocol confirmation, since port 80 of the wireless device 15a is a well-known port for using the World Wide Web HTTP protocol, Performs an operation to create a packet for protocol confirmation.

In addition, in the step of creating a protocol confirmation packet, if there is no response from the wireless device 15a to the protocol confirmation packet created using the web HTTP protocol, port 80 of the IoT device 15a uses the web HTTP protocol. Determines not to do so, and performs an operation of creating a packet for protocol confirmation using a protocol other than the World Wide Web HTTP protocol.

The step of transmitting the packet for protocol confirmation to the wireless device is the step of transmitting the packet for protocol confirmation to the target device. In the step of transmitting the protocol confirmation packet to the target device, an operation of transmitting the protocol confirmation packet is performed until a response comes from the target device.

The step of checking whether there is a response from the target device that has transmitted the packet for protocol confirmation is to monitor whether a response to the packet for protocol confirmation is coming from the target device, and if a response is received, the protocol used for such a response is changed from the corresponding open port. Performs an operation to determine the actual protocol used.

The type estimation step (S130) is a type of estimating each type of target devices using at least one result of the above-described photo scanning step (S110) and protocol scanning step (S120). This step is to perform an estimation operation. That is, the type estimation step (S130) is a step of performing the above-described type estimation operation.

According to an embodiment, the type estimating step (S130) includes estimating the type of service from the type of protocol found as a result of performing the protocol scanning operation, and estimating the type of the target device from the type of service. do. Here, the step of estimating the type of service from the type of protocol is a step of performing an operation of estimating the type of service from the type of protocol described above, and the step of estimating the type of target device from the type of service is the step of estimating the type of service described above. This is a step of performing an operation of estimating the type of the target device from the type. Therefore, please refer to the description of the embodiment of FIGS. 1 and 5 for a detailed description of these steps.

As described above, those skilled in the art to which the present invention pertains can understand that various modifications and variations are possible from the description of this specification. Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, but should be defined by not only the claims to be described later, but also those equivalent to these claims.

15, 15a, 15b, 15c, 15d: wireless device
100: Character animation creation system
110: voice and facial expression analysis module
120: visual manager
130: Animation archive
200: wireless care device

Claims (1)

As a method of generating animation expressing the motion and emotion of a character,
converting a voice signal including an arbitrary sentence into text data and parsing the text data to generate syntax data including the sentence;
generating character data including characteristics of a character from the voice signal;
selecting a first animation matching the syntax data from among a plurality of gesture animations;
selecting a second animation matching the character data from among a plurality of facial animations; and
Blending a gesture according to the syntax data and a facial expression according to the character data into first and second animations, respectively.

Images