KR20210003480A - Artistic effects providing method and system during video call - Google Patents

Abstract

The present invention relates to a method for providing a decorative effect during a video call and a system thereof. According to an embodiment of the present invention, by analyzing facial expressions, gestures, or the like of a user during a video call, a decorative effect between users, and a more interesting and rich video call service can be provided, so as to provide an effect of enabling non-verbal communication between users.

Description

Method and system for providing decorative effects during video call {ARTISTIC EFFECTS PROVIDING METHOD AND SYSTEM DURING VIDEO CALL}

The present invention relates to a method and system for providing decorative effects during a video call.

In general, a video call device such as a mobile communication terminal transmits and receives a user's video and voice acquired through a camera and a microphone, and thus performs a video call.

Since such a video call device transmits the image acquired through the camera as it is, it has not been able to satisfy the needs of users who place importance on visual effects.

In order to solve this problem, in the related art, various emoticons can be inserted and transmitted in an image acquired through a camera, thereby satisfying the visual needs of people.

That is, if a user wants to insert an emoticon during a video call, the user selects a desired emoticon through key manipulation, determines a location to insert the selected emoticon using direction keys, and places the emoticon at that location.

When the emoticon is placed, the video call device inserts and transmits the evoticon to the video input through the camera.

However, in such a conventional technique, whenever an emoticon is inserted, the user has to select a desired emoticon through key manipulation and place it in a desired position.

Korean Patent Registration No. 10-1326651'Video call device and method using emoticons'

Accordingly, the present invention is conceived from the above-described background, and provides a decorative effect between users through analysis of the user's facial expressions and gestures during a video call, thereby providing a more interesting and rich video call service. Its purpose is to provide a method and system for providing decorative effects during video calls that can be expected to enable non-verbal communication.

The object of the present invention is not limited thereto, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve this object, an embodiment of the present invention is a step of (a) generating call information (CI) including image information (i1) photographed by the input unit 103 of the terminal 101 ; (b) the control unit 107 of the terminal 101 analyzes the generated call information (CI) and extracts the state information (i3) of the user 105; (c) the control unit 107 of the terminal 101, the AR decoration corresponding to the state information (i3) extracted from the basic item information (BI) previously stored in the storage unit 111 of the terminal 101 Selecting item information (ARI); And (d) displaying, by the display unit 113 of the terminal 101, the AR decoration item information (ARI) superimposed on the image information (i1). Provides a way to provide effects.

In addition, in a system that provides a decorative effect when performing a video call with another user using the terminal 101 used by the user 105, the terminal 101 comprises: An input unit 103 for generating call information CI including information i1; The generated call information (CI) is analyzed to extract the state information (i3) of the user 105, and the state information (i3) extracted from the basic item information (BI) previously stored in the storage unit 111 A control unit 107 for selecting a matching AR decorative item information (ARI); And a display unit 113 that superimposes and displays the AR decorative item information ARI on the image information i1. It provides a system for providing a decorative effect during a video call.

According to an embodiment of the present invention, it is possible to provide a more interesting and rich video call service by providing a decorative effect between users through analysis of the user's facial expressions, gestures, etc. during a video call, and non-verbal communication between users. There is an effect that becomes possible.

1 is a block diagram showing the configuration of a method and system for providing a decorative effect during a video call according to an embodiment of the present invention.
2 is a flow chart showing each step of a method for providing a decorative effect during a video call according to an embodiment of the present invention.
3 is a diagram showing steps (a) and (b) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.
4 is a diagram showing step (c) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.
5 is a view showing step (d) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.
6 is a diagram showing steps (e) and (f) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a), (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected or connected to that other component, but another component between each component It should be understood that elements may be “connected”, “coupled” or “connected”.

1 is a block diagram showing the configuration of a method and system for providing a decorative effect during a video call according to an embodiment of the present invention.

2 is a flow chart showing each step of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

3 is a diagram showing steps (a) and (b) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

4 is a diagram showing step (c) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

5 is a view showing step (d) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

6 is a diagram showing steps (e) and (f) of a method for providing a decorative effect during a video call according to an embodiment of the present invention.

As shown in these drawings, the method of providing a decorative effect during a video call according to an embodiment of the present invention includes: (a) the input unit 103 of the terminal 101 includes image information photographing the user 105 ( generating call information (CI) including i1); (b) the control unit 107 of the terminal 101, analyzing the generated call information (CI) and extracting the state information (i3) of the user 105; (c) The control unit 107 of the terminal 101 is authorized from the server 109 and stored in the storage unit 111 of the terminal 101 from the basic item information (BI), extracted from the state information (i3). Selecting a matching AR decorative item information (ARI); And (d) displaying, by the display unit 113 of the terminal 101, the AR decoration item information ARI overlaid on the image information i1.

Hereinafter, each step will be described in detail.

First, step (a) is a step in which the input unit 103 of the terminal 101 generates call information CI including image information i1 photographing the user 105.

Here, the terminal 101 may be, for example, a smartphone, and the input unit 103 includes a camera, a microphone, and the like.

In addition, the call information (CI) includes image information (i1) photographing the user 105, and the image information (i1) is the expression information (i1-1) of the user 105 or the gesture of the user 105 It includes information (i1-2), and the call information (CI) may include the voice information (i2) of the user 105.

On the other hand, the gesture information (i1-2) includes hand gesture information of the user 105 (gesture using a finger, etc.), and gesture information of the user 105 (raising head, winking, showing palm, etc.) And the like.

Subsequently, step (b) is a step in which the control unit 107 of the terminal 101 analyzes the generated call information CI to extract the user's state information i3.

More specifically, the control unit 107 of the terminal 101 includes facial expression information (i1-1) of the user 105 included in the image information (i1), gesture information (i1-2) of the user 105, By analyzing the voice information (i2) of the user 105, etc., the state information (i3) for the current state of the user 105 is extracted.

Here, the current state of the user 105 refers to the current state of the user 105 including, for example,'joy','depression','excitement','sad','irritation','likeness','tiredness', etc. It means various states.

Subsequently, in step (c), the control unit 107 of the terminal 101 corresponds to the extracted state information (i3) from the basic item information (BI) previously stored in the storage unit 111 of the terminal 101. This is the step of selecting AR decoration item information (ARI).

Here, the AR decoration item information (ARI) may include, for example, emoticon information, sticker information, and the like.

Meanwhile, the control unit 107 may update the basic item information BI at a predetermined time interval through communication with the server 109.

For example, the control unit 107 may update the basic item information (BI) while communicating with the server 109 at intervals of 2 hours.

Through this, the control unit 107 can keep the basic item information (BI) up to date.

On the other hand, the control unit 107 selects the AR decorative item information ARI corresponding to the state information i3 from the basic item information BI.

For example, when the state information (i3) of the user 105 is extracted as'joy', the control unit 107 is an AR decoration item information (ARI) corresponding to the keyword'joy' among the basic item information (BI). Choose

In addition, the control unit 107, when the state information (i3) of the user 105 is extracted as'tired', the AR decoration item information (ARI) corresponding to the keyword'tired' among the basic item information (BI) Choose.

On the other hand, the control unit 107 may select the AR decorative item information (ARI) using the order of selection frequency of other users or the order of recommendations of other users authorized through communication with the server 109.

More specifically, when there are a plurality of AR decorative item information (ARI) corresponding to the keyword'joy' (for example, ARI1, ARI2, ARI3, ARI4), the control unit 107 communicates with the server 109 Through communication, you can select AR decoration item information (ARI2) using the order of selection frequency of other users or the order of recommendations of other users (① ARI2 → ② ARI1 → ③ ARI4 → ④ ARI3).

In addition, the control unit 107 uses the order of selection frequency of other users or the order of recommendations of other users authorized through communication with the server 109, and uses the order of the highest similarity, respectively, to obtain AR decoration item information (ARI). You can also choose.

More specifically, there are a plurality of AR decorative item information (ARI) matching the keyword'joy' (for example, ARI1, ARI2, ARI3, AIR4), and each AR decorative item information (ARI) has the highest similarity. When there is a plurality of AR decorative item information (for example, ARI1', ARI2', ARI3', ARI4'), the control unit 107 communicates with the server 109 in order of selection frequency of other users or in order of recommendation of other users. (① ARI2 → ② ARI1 → ③ ARI4 → ④ ARI3) to set the order of the most similar AR decoration item information (① ARI2' → ② ARI1' → ③ ARI4' → ④ ARI3'), and AR decoration item You can also select information (ARI2').

Next, step (d) is a step in which the display unit 113 of the terminal 101 superimposes the AR decorative item information ARI on the image information i1 and displays it.

Through this, it is possible to provide a more interesting and rich video call service by providing a decorative effect between users through analysis of the user's 105's facial expressions and gestures during a video call, and to enable non-verbal communication between users. You can expect an effect.

Meanwhile, according to an embodiment of the present invention, (e) the control unit 107 of the terminal 101 selects similar AR ornament item information (SARI) similar to the AR ornament item information (ARI) from the basic item information (BI). It may include;

Of course, the control unit 107 may select similar AR ornament item information (SARI) similar to the AR ornament item information (ARI) from the basic item information (BI) using machine learning or deep learning.

And according to the embodiment of the present invention, (f) the display unit 113 of the terminal 101 deletes the AR ornament item information (ARI), and transfers the similar AR ornament item information (SARI) to the image information (i1). It may include; displaying by overlapping.

That is, the display unit 113 of the terminal 101 deletes the AR decoration item information ARI that was superimposed on the image information i1 and displayed after a certain period of time, and the similar AR decoration item information SARI. May be newly superimposed on the image information i1 and displayed.

Through this, it is possible to experience variable decorative effects between users during a video call.

On the other hand, the system for providing a decorative effect during a video call according to another embodiment of the present invention is a system for providing a decorative effect when performing a video call with another user using the terminal 101 used by the user 105. , The terminal 101 includes an input unit 103 for generating call information CI including image information i1 photographing the user 105; By analyzing the generated call information (CI), the state information (i3) of the user 105 is extracted, and the AR corresponding to the state information (i3) extracted from the basic item information (BI) previously stored in the storage unit 111 A control unit 107 for selecting decorative item information (ARI); And a display unit 113 for superimposing and displaying the AR decorative item information ARI on the image information i1.

In addition, the image information i1 may include facial expression information i1-1 of the user 105 or gesture information i1-2 of the user 105.

In addition, the control unit 107 may update the basic item information BI at a predetermined time interval through communication with the server 109.

In addition, the control unit 107 may select the AR decoration item information (ARI) using the order of selection frequency of other users or the order of recommendations of other users authorized through communication with the server 109.

In addition, the control unit 107 selects the similar AR ornament item information (SARI) similar to the AR ornament item information (ARI) from the basic item information (BI), and the display unit 113, the AR ornament item information (ARI) After deleting, the similar AR decoration item information (SARI) can be superimposed on the image information (i1) and displayed.

Since the functions, actions, and effects of each component of the system for providing decorative effects in the video call described above are the same as those described in the method for providing decorative effects during the video call, detailed descriptions thereof will be omitted.

As described above, according to an embodiment of the present invention, by providing a decorative effect between users through analysis of the user's facial expressions and gestures during a video call, it is possible to provide a more interesting and rich video call service, The effect of enabling non-verbal communication between users can be expected.

In the above, even if all the constituent elements constituting the embodiments of the present invention have been described as being combined into one or operating in combination, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all of the constituent elements may be selectively combined and operated in one or more.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

101: terminal
103: input unit
105: user
107: control unit
109: server
111: storage
113: display unit
i1: image information
i1-1: Facial expression information
i1-2: Gesture information
i2: Voice information
i3: Status information
CI: Currency information
BI: Basic item
ARI: AR decoration item information
SARI: Similar AR decoration item information

Claims (11)

(a) generating, by the input unit 103 of the terminal 101, call information (CI) including image information (i1) photographing the user 105;
(b) the control unit 107 of the terminal 101 analyzes the generated call information (CI) and extracts the state information (i3) of the user 105;
(c) the control unit 107 of the terminal 101, the AR decoration corresponding to the state information (i3) extracted from the basic item information (BI) previously stored in the storage unit 111 of the terminal 101 Selecting item information (ARI); And
(d) displaying, by the display unit 113 of the terminal 101, the AR decorative item information ARI overlaid on the image information i1;
A method for providing a decorative effect during a video call, comprising: a. The method of claim 1,
The image information (i1) of the step (a),
A method for providing a decorative effect during a video call, comprising the expression information (i1-1) of the user 105 or the gesture information (i1-2) of the user 105. The method of claim 1,
The control unit 107 of the step (c),
The method of providing a decorative effect during a video call, comprising updating the basic item information (BI) at a predetermined time interval through communication with the server (109). The method of claim 1,
The control unit 107 of the step (c),
The method of providing a decorative effect during a video call, characterized in that the AR decorative item information (ARI) is selected using the order of selection frequency of other users or the order of recommendations of other users approved through communication with the server 109. The method of claim 1,
(e) selecting, by the control unit 107 of the terminal 101, similar AR decoration item information (SARI) similar to the AR decoration item information (ARI) from the basic item information (BI);
A method for providing a decorative effect during a video call, comprising: a. The method of claim 5,
(f) the display unit 113 of the terminal 101 deletes the AR ornament item information (ARI), and displays the similar AR ornament item information (SARI) superimposed on the image information (i1). ;
A method for providing a decorative effect during a video call, comprising: a. In a system that provides a decorative effect when performing a video call with another user using the terminal 101 used by the user 105,
The terminal 101,
An input unit 103 for generating call information (CI) including image information (i1) photographing the user 105;
The generated call information (CI) is analyzed to extract the state information (i3) of the user 105, and the state information (i3) extracted from the basic item information (BI) previously stored in the storage unit 111 A control unit 107 for selecting a matching AR decorative item information (ARI); And
A display unit 113 that superimposes and displays the AR decorative item information ARI on the image information i1;
A system for providing decorative effects during a video call, comprising: a. The method of claim 7,
The image information (i1) includes display information (i1-1) of the user 105 or gesture information (i1-2) of the user (105). The method of claim 7,
The control unit 107,
The system for providing a decorative effect during a video call, characterized in that the basic item information (BI) is updated at a predetermined time interval through communication with the server (109). The method of claim 7,
The control unit 107,
A system for providing a decoration effect during a video call, characterized in that the AR decoration item information (ARI) is selected using the order of selection frequency of other users or the order of recommendations of other users approved through communication with the server 109. The method of claim 7,
The control unit 107,
Selecting similar AR decorative item information (SARI) similar to AR decorative item information (ARI) from the basic item information (BI),
The display unit 113,
After deleting the AR decorative item information (ARI), the similar AR decorative item information (SARI) is superimposed on the image information (i1) to display the decorative effect during a video call.

Images