KR102049838B1 - Apparatus and method for transferring emotion based on vibrotactile - Google Patents

Abstract

The present invention discloses an apparatus and method for transmitting vibration based emotion. According to an embodiment of the present invention, the emotion transmitting apparatus analyzes at least one of a user recognition unit that recognizes at least one of a user voice and a user image, and analyzes at least one of the recognized user voice and the recognized user image to obtain user emotion information. A generated emotion information generator, a vibration tactile information generator configured to generate vibration tactile information based on the generated user emotion information, and at least one of the recognized user voice and the recognized user image and the generated vibration tactile information It may include a data processing unit for generating the user emotion data based on the, and transmits the generated user emotion data to the other party device connected to the call.

Description

Vibration tactile based emotion transmission device and method {APPARATUS AND METHOD FOR TRANSFERRING EMOTION BASED ON VIBROTACTILE}

The present invention relates to a technical idea of transmitting a user's emotion using a vibration tactile device, and relates to an apparatus and a method for transmitting a vibrotactile generated based on an emotion analyzed to the other party together with a voice and an image.

As technology has advanced, voice-centric phone calls have evolved into video calls using cameras and monitors.

In particular, the emergence of smart phones has contributed greatly to the spread of video calls. As a result, people can simultaneously send and receive auditory informational visual information during a call.

 The smartphone may be equipped with a vibrator to provide a haptic function that also provides tactile information to the user. Therefore, the smart phone can provide an environment that can deliver tactile information to the other party.

The haptic function of a smartphone can play a role of interacting with a smartphone and a user.

In other words, the smartphone may provide various vibrations to the user only when the user touches or shakes his smartphone.

However, there is a problem in that a smart phone provides a vibration tactile touch to a user according to a touch and rotation operation, and cannot transmit the emotion and emotion of the other party to the other party using vibration during a video call.

Korea Patent Registration No. 10-1294214, "Mobile communication terminal and method for transmitting the emotion of the other party to the vibration" Korean Patent Publication No. 10-2010-0028764, "Mobile communication terminal and method for transmitting and receiving vibration information during a voice call"

The present invention is to provide an apparatus and method for transmitting vibration based emotion.

An object of the present invention is to provide an apparatus and method for analyzing user emotions based on video and audio obtained during a video call, and transmitting the analyzed user emotions to vibrotactile tactile senses.

An object of the present invention is to provide an apparatus and method for increasing the immersion of a video call through visual and auditory as well as touch.

An object of the present invention is to provide an apparatus and method for transmitting emotion information through voice, video, and vibration tactile in various entertainments such as voice call, video call, game, and conference.

 According to an embodiment of the present invention, the emotion transmitting apparatus analyzes at least one of a user recognition unit that recognizes at least one of a user voice and a user image, and analyzes at least one of the recognized user voice and the recognized user image to obtain user emotion information. A generated emotion information generator, a vibration tactile information generator configured to generate vibration tactile information based on the generated user emotion information, and at least one of the recognized user voice and the recognized user image and the generated vibration tactile information It may include a data processing unit for generating the user emotion data based on the, and transmits the generated user emotion data to the other party device connected to the call.

According to an embodiment of the present invention, the user recognizer may record the recognized user voice as much as a specific buffer capacity, and the emotion information generator may generate first user emotion information by analyzing the recorded user voice. .

According to an embodiment of the present invention, the user recognition unit may divide the recognized user image in units of frames and recognize a user face in each of the divided user images.

According to an embodiment of the present invention, the user recognizer determines a user image of a first frame and a user image of a second frame from the divided user image, and determines the user image of the determined first frame and the determined second frame. By comparing the user image of the user can recognize the facial expression change of the recognized user's face.

According to an embodiment of the present invention, the emotion information generation unit may generate second user emotion information by analyzing the user image of the determined first frame.

According to an embodiment of the present invention, the emotion information generator determines whether a user image of the second frame exists in the divided user image, and when the user image of the second frame exists, the recognized facial expression change. May be analyzed to update the generated second user emotion information.

According to an embodiment of the present invention, the emotion information generation unit may generate the user emotion information including the generated first user emotion information and the updated second user emotion information.

According to an embodiment of the present invention, the vibrotactile information generation unit generates first vibrotactile information based on the generated first user emotion information, and the data processor generates the recognized user voice and the generated agent. The first user emotion data may be generated based on the first vibration tactile information, and the generated first user emotion data may be transmitted to the counterpart device.

According to an embodiment of the present invention, the vibrotactile information generation unit generates second vibrotactile information based on the updated second user emotion information, and the data processor generates the recognized user image and the generated agent. The second user emotion data may be generated based on the vibration tactile information, and the generated second user emotion data may be transmitted to the counterpart device.

According to an embodiment of the present invention, the vibration tactile information generation unit generates third vibration tactile information based on the generated first user emotion information and the updated second user emotion information, and the data processor is configured to: The third user emotion data may be generated based on the recognized user voice, the recognized user image, and the generated second vibration tactile information, and the third user emotion data may be transmitted to the counterpart device.

According to an embodiment of the present invention, in the emotion transmitting method, at least one of a user voice and a user image is recognized by a user recognizer, and at the emotion information generator, at least one of the recognized user voice and the recognized user image. Generating any user emotion information by analyzing any one, generating the vibration tactile information based on the generated user emotion information, and generating, by the data processor, the recognized user voice and the recognized user The method may include generating user emotion data based on at least one of user images and the generated vibration tactile information, and transmitting the generated user emotion data to a counterpart connected device.

According to an embodiment of the present invention, in the emotion delivery method, recording the recognized user voice by a specific buffer capacity in the user recognition unit, in the user recognition unit, the recognized user image is divided in units of frames, Recognizing a user face in each of the divided user images, determining, by the user recognition unit, a user image of a first frame and a user image of a second frame in the divided user image, and in the user recognition unit, And comparing the user image of the determined first frame with the user image of the determined second frame to recognize a change in facial expression of the recognized user's face.

According to an embodiment of the present invention, in the emotion delivery method, generating the first user emotion information by analyzing the recorded user voice in the emotion information generator, and in the emotion information generator, Generating second user emotion information by analyzing a user image, determining, by the emotion information generator, whether a user image of the second frame exists in the divided user image, and a user image of the second frame If present, analyzing the recognized facial expression change to update the generated second user emotion information and the user emotion information including the generated first user emotion information and the updated second user emotion information It may comprise the step of generating.

According to an embodiment of the present invention, the user emotion may be analyzed based on the video and the voice acquired during the video call, and the analyzed user emotion may be transmitted to the vibrotactile.

According to an embodiment of the present invention, the immersion of the video call can be increased through not only visual and auditory but also tactile senses.

According to an embodiment of the present invention, emotional information may be transmitted through voice, video, and vibration tactile in various entertainments such as voice call, video call, game, and conference.

1 is a block diagram of an emotional delivery device according to an embodiment of the present invention.
2 to 5 show a flow diagram related to the emotional delivery method according to an embodiment of the present invention.

Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings.

The examples and terms used therein are not intended to limit the techniques described in this document to specific embodiments, but should be understood to include various modifications, equivalents, and / or alternatives to the examples.

In the following description of the various embodiments, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

In addition, terms to be described below are terms defined in consideration of functions in various embodiments, and may vary according to a user's or operator's intention or custom. Therefore, the definition should be made based on the contents throughout the specification.

In connection with the description of the drawings, similar reference numerals may be used for similar components.

Singular expressions may include plural expressions unless the context clearly indicates otherwise.

In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of items listed together.

Expressions such as "first," "second," "first," or "second," etc. may modify the components, regardless of order or importance, to distinguish one component from another. Used only and do not limit the components.

When any (eg first) component is said to be "connected (functionally or communicatively)" or "connected" to another (eg second) component, the other component is said other It may be directly connected to the component or may be connected through another component (for example, the third component).

In this specification, "configured to" is modified to have the ability to "suitable," "to," "to," depending on the context, for example, hardware or software. Can be used interchangeably with "made to", "doing", or "designed to".

In some situations, the expression “device configured to” may mean that the device “can” together with other devices or components.

For example, the phrase “processor configured (or configured to) perform A, B, and C” may be implemented by executing a dedicated processor (eg, an embedded processor) to perform its operation, or one or more software programs stored in a memory device. It may mean a general purpose processor (eg, a CPU or an application processor) capable of performing the corresponding operations.

In addition, the term 'or' means inclusive or 'inclusive or' rather than 'exclusive or'.

In other words, unless stated otherwise or unclear from the context, the expression 'x uses a or b' means any one of natural inclusive permutations.

1 is a block diagram of an emotional delivery device according to an embodiment of the present invention.

Specifically, Figure 1 illustrates the components of the emotional delivery device according to an embodiment of the present invention. Used below. Wealth, ' The term 'herein' refers to a unit for processing at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software.

Referring to FIG. 1, the emotion transmitting apparatus 100 may include a user recognizer 110, an emotion information generator 120, a vibration tactile information generator 130, and a data processor 140.

For example, the emotion transmitting device 100 may be a smartphone or a user terminal device, or may be an external device combined with a smartphone or a user terminal device.

According to an embodiment of the present invention, the user recognizer 110 recognizes at least one of a user voice and a user image.

According to an embodiment of the present invention, the user recognition unit 110 records the recognized user voice by a specific buffer capacity.

For example, the user recognition unit 110 may include a microphone and a camera, and may recognize a user voice through the microphone, and may recognize the user image through the camera.

According to an embodiment of the present invention, the user recognition unit 110 divides the user image recognized by the camera in units of frames, and recognizes a user face in each of the divided user images.

For example, the divided user image may include a user image of a first frame, a user image of a second frame, and a user image of an nth frame. Here, n may be changed to any number based on the user's setting.

For example, the user recognizer 110 determines the user image of the first frame and the user image of the second frame from the divided user image, and compares the user image of the determined first frame with the user image of the determined second frame. The facial expression change of the recognized user's face may be recognized.

For example, the user recognition unit 110 recognizes a user image using a frame that is a photo unit constituting an image. User video is a form that continuously outputs 24 to 60 pictures per second. The user recognition unit 110 divides the expression in the user face of the user image in units of one frame and performs emotion recognition through facial recognition.

According to an embodiment of the present invention, the emotion information generator 120 generates user emotion information by analyzing at least one of a user voice and a user image.

For example, the emotion information generator 120 analyzes the user voice recorded by a specific buffer capacity and generates first user emotion information.

For example, the specific buffer capacity may correspond to a reference value for generating the first user emotion information by analyzing the user voice by the emotion information generator 120.

According to an embodiment of the present invention, the emotion information generating unit 120 generates the second user emotion information by analyzing the determined user image of the first frame.

For example, the emotion information generator 120 may determine whether the user image of the second frame exists in the divided user image.

According to an embodiment of the present invention, when the user image of the second frame exists, the emotion information generator 120 updates the second user emotion information generated by analyzing the recognized facial expression change.

For example, the emotion information generator 120 divides the user image into one frame unit and determines whether a face of the user exists in the divided user image.

In addition, when the user's face is present, the emotion information generating unit 120 determines the facial expression of the user's face, compares the shape of the eye, the shape of the mouth, etc. in the user image of the frame to which the user's face is determined, Be aware of the change.

For example, the emotion information generator 120 may generate second user emotion information based on a change in facial expression of the user's face in the user image. In addition, the second user emotion information may be used for facial emotion recognition.

According to an embodiment of the present invention, the emotion information generator 120 generates user emotion information including the generated first user emotion information and the updated second user emotion information.

For example, the emotion information generator 120 generates first user emotion information based on a change in voice frequency in the user voice. In addition, the first user emotion information may be used for voice emotion recognition.

For example, the emotion information generator 120 generates user emotion information including first user emotion information based on voice analysis and second user emotion information based on image analysis.

According to an embodiment of the present invention, the vibration tactile information generation unit 130 generates vibration tactile information based on user emotion information.

For example, the vibration tactile information generation unit 130 generates vibration tactile information based on user emotion information analyzed based on a recognized user voice and a user image for communication with the counterpart device.

For example, the vibrotactile information is a haptic icon, a concept proposed by MacLean et al. In 2003, and includes a vibrotactile signal having a system made to convey an abstract meaning.

According to an embodiment of the present invention, the vibration tactile information generating unit 130 generates the first vibration tactile information based on the first user emotion information based on voice analysis.

For example, the vibration tactile information generator 130 generates second vibration tactile information based on the second user emotion information based on the image analysis.

According to an embodiment of the present invention, the vibration tactile information generating unit 130 generates third vibration tactile information based on the first user emotion information and the second user emotion information.

According to an embodiment of the present invention, the data processor 140 generates user emotion data based on at least one of user voice and user image and vibration tactile information.

For example, the user emotion data may include first user emotion data, second user emotion data, and third user emotion data.

According to an embodiment of the present invention, the data processor 140 transmits the user emotion data generated to the counterpart connected device.

For example, the data processor 140 may generate first user emotion data based on the user voice and the first vibration tactile information, and transmit the first user emotion data to the counterpart device.

According to an embodiment of the present invention, the data processor 140 may generate second user emotion data based on the user image and the second vibration tactile information, and transmit the second user emotion data to the counterpart device.

According to an embodiment of the present invention, the data processor 140 generates third user emotion data based on the user voice, the user image, and the second vibration tactile information, and transmits the third user emotion data to the counterpart device.

For example, the vibration tactile information may be transmitted to the counterpart device and used to generate a vibration based on the vibration tactile information in the counterpart device.

According to an embodiment of the present invention, the emotion transmitting apparatus may analyze the user emotion based on the image and the voice acquired during the video call, and transmit the analyzed user emotion as the vibrotactile.

According to an embodiment of the present invention, the emotional delivery device may increase the immersion of the video call through the sense of touch as well as visual and auditory.

According to an embodiment of the present invention, the data processing unit 140 may transmit user emotion data to the counterpart device through the communication unit 150.

According to another embodiment of the present invention, the emotion transmitting apparatus 100 may include a communication unit 150, a memory 160, and a controller 170.

According to another embodiment of the present invention, the communication unit 150 up-converts the baseband signal into an RF band signal and then transmits the signal through an antenna and downconverts the RF band signal received through the antenna into the baseband signal.

For example, the communication unit 150 may include a transmission filter, a reception filter, an amplifier, a mixer, an oscillator, a digital to analog convertor (DAC), an analog to digital convertor (ADC), and the like.

In addition, the communication unit 150 may include different communication modules to process signals of different frequency bands.

For example, different communication standards may include Bluetooth low energy (BLE), Wireless Fidelity (Wi-Fi), Wi-Fi Gigabyte (WiGig), cellular networks (e.g. Long Term Evolution), etc. Can be.

In addition, different frequency bands may include a super high frequency (SHF) (eg 2.5 GHz, 5 Ghz) band and a millimeter wave (eg 60 GHz) band.

According to another exemplary embodiment of the present invention, the memory 160 stores data such as a basic program, an application program, and setting information for the operation of the emotion transmitting apparatus 100.

In particular, the memory 160 may store at least one command set (eg, an application) for managing a file according to various embodiments.

For example, the memory 160 may provide stored data at the request of the controller 170, and the memory 160 may include volatile and / or nonvolatile memory and a buffer.

For example, the memory 160 may store instructions or data related to at least one other component of the emotion transmitting device 100.

The memory 160 is included in the emotion transmitting device 100 and may be referred to as an 'internal storage' or an 'internal storage device'.

For example, the memory 160 may store a user voice and a user image in a buffer.

According to another embodiment of the present invention, the control unit 170 may control operations of the user recognition unit 110, the emotion information generating unit 120, the vibration tactile information generating unit 130, and the data processing unit 140.

For example, the controller 170 may execute arithmetic or data processing related to control and / or communication of at least one other component of the emotion transmitting apparatus 100.

For example, the controller 170 may drive an operating system or an application program to control a plurality of hardware or software components connected to the controller 170, and may perform various data processing and operations.

For example, the controller 170 may be implemented as a system on chip (SOC). The controller 170 may load and process a command or data received from at least one of other components (eg, nonvolatile memory) into the volatile memory, and store the result data in the nonvolatile memory.

In addition, the controller 170 may generate a vibration by controlling the vibrator.

For example, the controller 170 may generate vibration tactile based on vibration tactile information generated based on user emotion information.

For example, the controller 170 may control the memory 160 to store a user voice as much as a specific buffer capacity.

For example, the controller 170 may control the emotional delivery apparatus 100 to perform the procedure illustrated in FIGS. 2 to 5.

According to an embodiment of the present invention, the emotion delivery device 100 may transmit emotion information through voice, video, and vibration tactile in various entertainments such as voice call, video call, game, and conference.

2 is a flowchart illustrating an emotional delivery method according to an embodiment of the present invention.

In detail, FIG. 2 illustrates a procedure of generating a vibration tactile information by analyzing a user voice and an image acquired during a video call with the counterpart device, and simultaneously transmitting the user voice and the image with the vibration tactile information.

Referring to FIG. 2, in step 201, the method for transmitting emotions recognizes a user voice and a user image. In other words, the emotional delivery method recognizes a user's voice using a microphone when a video call or a voice call is connected to the counterpart device, and recognizes the user's image through a camera.

In step 202, the emotion delivery method generates user emotion information. That is, the emotion delivery method generates user emotion information by analyzing at least one of a user voice and a user image.

In step 203, the emotional delivery method generates vibration tactile information. That is, the emotion delivery method generates vibration tactile information based on user emotion information.

In step 204, the emotion delivery method transmits user emotion data to the counterpart device. That is, the emotion transmission method generates user emotion data based on at least one of user voice and user image and vibration tactile information, and transmits the generated user emotion data to the counterpart device.

3 is a flowchart illustrating a method of transmitting emotion according to an embodiment of the present invention.

In detail, FIG. 3 illustrates a procedure in which the method of transmitting emotion generates vibration tactile information using user emotion information analyzed based on user voice and user image, and transmits user voice and user image and vibration tactile information to the counterpart device. To illustrate.

Referring to FIG. 3, in step 301, the method for transmitting emotions is connected with a counterpart device. That is, the emotional delivery method connects a voice call or a video call with the counterpart device, recognizes a user voice through a microphone, and recognizes a user image through a camera.

In step 302, the emotion delivery method performs voice based emotion analysis through the voice channel. That is, the emotion delivery method analyzes user emotion based on the user voice recognized through the microphone. Here, the emotion transmission method may analyze the user emotion based on the frequency change of the user voice.

In step 303, the emotion delivery method performs image-based emotion analysis through an image channel. That is, the emotion delivery method analyzes user emotions based on user images recognized through a camera. Here, the emotion delivery method may analyze the user emotion based on the face of the user image.

In step 304, the emotion delivery method generates vibration tactile information based on voice based emotion analysis and image based emotion analysis. That is, the emotion transmission method may generate vibration tactile information based on user emotion information generated based on analysis of user voice and analysis of user image, and generate vibration tactile information by combining the generated vibration tactile information. Can be.

In step 305, the emotional delivery method transmits voice, video and tactile information to the other party. That is, the emotion transmission method transmits the tactile information generated based on the analysis of the user voice, the user image, and the voice and the image to the counterpart connected device.

According to an embodiment of the present invention, the emotion delivery method may analyze the user emotion based on the image and the voice acquired during the video call, and transmit the analyzed user emotion as the vibrotactile.

4 is a flowchart illustrating an emotional delivery method according to an embodiment of the present invention.

Specifically, FIG. 4 illustrates a procedure in which the emotion transmission method generates vibration tactile information using the user emotion information analyzed based on the user voice and the user voice and transmits the emotion data.

Referring to FIG. 4, in step 401, the emotional delivery method recognizes a user voice. That is, the emotional delivery method recognizes a user's voice through a microphone when making a phone call with a counterpart device.

In step 402, the emotional delivery method determines whether the user voice has been recorded by a specific buffer capacity. That is, the emotional delivery method determines whether or not the user voice recognized through the microphone in step 401 is more than a reference value for determining the user emotion based on the user voice.

In one example, the emotional delivery method proceeds to step 403 if the user voice is recorded in the buffer by a specific buffer capacity, and returns to step 401 if the user voice is not recorded in the buffer by the specific buffer capacity. Write more to the buffer.

In step 403, the emotion delivery method analyzes the user voice stored in the buffer to generate first user emotion information. That is, the emotion transmission method analyzes the user's voice based on the change in the frequency band of the user's voice, and generates the user's emotion information based on the analyzed user's voice.

In step 404, the emotion delivery method generates vibration tactile information based on the first user emotion information. That is, the emotion transmission method generates first vibration tactile information based on the first user emotion information generated based on the user voice analysis.

In step 405, the emotional delivery method transmits user voice and vibration tactile information. That is, the emotion transmission method generates first user emotion data based on the user voice and the first vibration tactile information, and transmits the generated first user emotion data to the counterpart device.

5 is a flowchart illustrating an emotional delivery method according to an embodiment of the present invention.

In detail, FIG. 5 illustrates a procedure in which the method of transmitting emotion generates vibration tactile information by analyzing the user image and simultaneously transmits the vibration tactile information generated by the counterpart device with the user image.

Referring to FIG. 5, in step 501, the method for transmitting emotions recognizes and records a user image. That is, in the emotion delivery method, when the video call is started with the counterpart device, the user image is recognized and the recognized user image is recorded.

In step 502, the emotional delivery method divides the recognized user image into frames. That is, the emotion transmitting method divides a user image composed of consecutive scenes into frame units which are images of sheets.

In one example, the emotion transfer method divides a user image into a user image of a first frame, a user image of a second frame, and a user image of an nth frame.

In one example, the emotional delivery method may be divided into a user image of the first frame of the current frame and a user image of the second frame of the previous frame.

In step 503, the method of emotional delivery recognizes a user face in each of the divided user images. That is, the emotion transmission method recognizes a user's face in each of the images which are user images divided by frame.

In step 504, the emotion delivery method generates second user emotion information based on the recognized user face. That is, the emotion delivery method recognizes a user's expression based on a mouth, eyes, nose, etc. in the user's face, and generates second user emotion information based on the recognized user's expression.

In step 505, the method for transmitting emotion determines whether a user image of a second frame exists. That is, the emotion transfer method may recognize a plurality of user images by dividing the recognized user image into frame units.

That is, the method for transmitting emotions determines whether a second frame, which is the user image of the previous frame of the current image, exists, and if the user image of the second frame exists, proceeds to step 506, and If there is no user image, the flow proceeds to step 508.

In step 506, the emotional delivery method determines whether a facial expression change occurs. That is, the emotion delivery method determines whether the facial expression change of the user face occurs by comparing the user face based on the user image of the first frame and the user face based on the user image of the second frame.

In one example, the emotional delivery method proceeds to step 507 when a change in the expression of the user's face occurs, and proceeds to step 508 when a change in the expression of the user's face does not occur.

In step 507, the emotion delivery method analyzes the facial expression change to update the second user emotion information. That is, the emotion delivery method analyzes the facial expression change to update the second user emotion information generated in step 504.

In step 508, the emotion delivery method generates vibration tactile information based on the second user emotion information. That is, the emotion delivery method generates second vibration tactile information based on the updated second user emotion information.

In step 509, the emotional delivery method transmits the user image and the vibration tactile information. That is, the emotion transmission method generates second user emotion data based on the user image and the second vibration tactile information, and transmits the second user emotion data generated to the counterpart device.

As a result, according to the present invention, the user emotion may be analyzed based on the video and audio obtained during the video call, and the analyzed user emotion may be transmitted to the vibrotactile. In addition, visual and auditory as well as tactile sense can increase the immersion of the video call, and can be sent through the voice, video and vibration tactile information in a variety of entertainment, such as voice calls, video calls, games and meetings have.

Methods according to the embodiments described in the claims or the specification of the present invention may be implemented in the form of hardware, software, or a combination of hardware and software.

Such software may be stored on a computer readable storage medium. The computer readable storage medium includes at least one program (software module), at least one program including instructions for causing the electronic device to perform the method of the present invention when executed by the at least one processor in the electronic device. Save it.

Such software may be in the form of volatile or non-volatile storage, such as Read Only Memory (ROM), or random access memory (RAM), memory chips, devices or In the form of memory, such as integrated circuits, or in compact disc ROM (CD-ROM), digital versatile discs (DVDs), magnetic disks, or magnetic tapes tape, or the like, on an optical or magnetic readable medium.

Storage device and storage media are embodiments of machine-readable storage means suitable for storing a program or programs that include instructions that, when executed, implement embodiments.

Embodiments provide a program comprising code for implementing an apparatus or method as claimed in any of the claims herein, and a machine-readable storage medium storing such a program.

Furthermore, such programs may be delivered electronically by any medium, such as a communication signal carried over a wired or wireless connection, and embodiments include equivalents as appropriate.

In the above-described specific embodiments, the components included in the invention are expressed in the singular or plural according to the specific embodiments presented.

However, the singular or plural expressions are selected to suit the situations presented for convenience of description, and the above-described embodiments are not limited to the singular or plural elements, and the singular or plural elements may be composed of the singular or the plural. However, even a component expressed in the singular may be configured in plural.

Meanwhile, in the description of the present invention, specific embodiments have been described, but various modifications may be made without departing from the scope of the technical idea included in the various embodiments.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims below and equivalents thereof.

100: emotional delivery device 110: user recognition unit
120: emotion information generation unit 130: vibration tactile information generation unit
140: data processing unit

Claims (13)

A user recognizer configured to recognize at least one of a user voice and a user image;
An emotion information generator configured to analyze at least one of the recognized user voice and the recognized user image to generate user emotion information;
A vibration tactile information generating unit generating vibration tactile information based on the generated user emotion information; And
And a data processor configured to generate user emotion data based on at least one of the recognized user voice and the recognized user image and the generated vibration tactile information, and to transmit the generated user emotion data to a counterpart connected device. and,
The emotion information generation unit,
It is determined whether the user voice is recorded by the buffer capacity corresponding to the reference value for determining the user emotion,
As a result of the determination, the recognized user voice is analyzed only when the user voice is recorded in the buffer by the buffer capacity.
Analyzing the recognized user image by recognizing a change in facial expression by comparing the shape of the eye and the mouth of the user identified in the recognized user image with a comparison target frame, using a frame divided from the recognized user image Generate user emotion information, update the generated user emotion information using another frame after the divided frame,
The vibration tactile information generation unit generates the vibration tactile information by using the generated user emotion information and the updated user emotion information.
Vibration tactile based emotion transmission device.
The method of claim 1,
The user recognition unit records the recognized user voice by a specific buffer capacity,
The emotion information generation unit analyzes the recorded user voice to generate first user emotion information.
Vibration tactile based emotion transmission device.
The method of claim 2,
The user recognition unit divides the recognized user image on a frame basis and recognizes a user face in each of the divided user images.
Vibration tactile based emotion transmission device.
The method of claim 3,
The user recognition unit determines the user image of the first frame and the user image of the second frame from the divided user image, compares the user image of the determined first frame with the user image of the determined second frame, and recognizes the image. To recognize facial expression changes on the user's face
Vibration tactile based emotion transmission device.
The method of claim 4, wherein
The emotion information generator is configured to analyze the user image of the determined first frame to generate second user emotion information.
Vibration tactile based emotion transmission device.
The method of claim 5,
The emotion information generator determines whether a user image of the second frame exists in the divided user image, and analyzes the recognized facial expression change when the user image of the second frame exists. To update user sentiment information
Vibration tactile based emotion transmission device.
The method of claim 6,
The emotion information generation unit generates the user emotion information including the generated first user emotion information and the updated second user emotion information.
Vibration tactile based emotion transmission device.
The method of claim 7, wherein
The vibration tactile information generation unit generates first vibration tactile information based on the generated first user emotion information,
The data processor generates first user emotion data based on the recognized user voice and the generated first vibration tactile information, and transmits the generated first user emotion data to the counterpart device.
Vibration tactile based emotion transmission device.
The method of claim 7, wherein
The vibration tactile information generation unit generates second vibration tactile information based on the updated second user emotion information,
The data processor generates second user emotion data based on the recognized user image and the generated second vibration tactile information, and transmits the generated second user emotion data to the counterpart device.
Vibration tactile based emotion transmission device.
The method of claim 7, wherein
The vibration tactile information generation unit generates third vibration tactile information based on the generated first user emotion information and the updated second user emotion information,
The data processor generates third user emotion data based on the recognized user voice, the recognized user image, and the generated third vibration tactile information, and transmits the generated third user emotion data to the counterpart device. Transmitting
Vibration tactile based emotion transmission device.
Recognizing, by the user recognition unit, at least one of a user voice and a user image;
In the emotion information generator, determining whether a user voice is recorded by a buffer capacity corresponding to a reference value for determining a user emotion;
As a result of the determination, the recognized user voice is analyzed only when the user voice is recorded in the buffer as much as the buffer capacity, and the shape of the eyes and the mouth of the user identified in the recognized user image are compared with the comparison target frame. Recognizing facial expression changes to analyze the recognized user image to generate user emotion information, generating user emotion information from a frame divided from the recognized user image, and using another frame after the divided frame. Updating the generated user emotion information;
Generating, by the vibration tactile information generating unit, the vibration tactile information using the generated user emotion information and the updated user emotion information; And
In the data processing unit, generating user emotion data based on at least one of the recognized user voice and the recognized user image and the generated vibration tactile information, and transmitting the generated user emotion data to a counterpart device. Containing
Vibration tactile based emotion transmission method.
In claim 11,
Recognizing at least one of the user voice and the user image,
Recording, by the user recognition unit, the recognized user voice by a specific buffer capacity;
Dividing the recognized user image by frame unit and recognizing a user face in each of the divided user images;
Determining, by the user recognizer, a user image of a first frame and a user image of a second frame from the divided user image; And
In the user recognition unit, comparing the determined user image of the first frame with the determined user image of the second frame, and recognizing a change in facial expression of the recognized user face;
Vibration tactile based emotion transmission method.
In claim 12,
Generating user emotion information by analyzing at least one of the recognized user voice and the recognized user image,
Generating, by the emotion information generator, first user emotion information by analyzing the recorded user voice;
Generating, by the emotion information generator, second user emotion information by analyzing the determined user image of the first frame;
Determining, by the emotion information generator, whether a user image of the second frame exists in the divided user image;
Updating the generated second user emotion information by analyzing the recognized facial expression change when the user image of the second frame exists; And
Generating the user emotion information including the generated first user emotion information and the updated second user emotion information.
Vibration tactile based emotion transmission method.

Images