KR102120871B1 - Terminal and server providing a video call service - Google Patents

Abstract

Disclosed is a terminal providing a video call service by generating evaluation information corresponding to a gesture of a user of a video call service, and a server providing a video call service based on a reputation index of a user based on the evaluation information received from the terminal.

Description

TERMINAL AND SERVER PROVIDING A VIDEO CALL SERVICE}

It relates to a terminal providing a video call service and a server providing a video call service.

With the development of science and technology, most people have terminals such as smart phones and tablet PCs. Such a terminal can take a video, play it, and communicate with other terminals to exchange information or support a video call.

On the other hand, there are various types of video call services based on video calls. For example, the random video chat service is a service that connects a terminal of a user who has made a request to use the random video chat service with a terminal of a randomly selected user among users using the random video chat service.

It is to provide a terminal for providing a video call service by generating evaluation information corresponding to a gesture of a user of a video call service, and a server for providing a video call service based on a reputation index of a user based on the evaluation information received from the terminal.

The application stored in the computer-readable storage medium according to the first aspect includes the steps of: establishing a video call session between the first terminal of a first user and a second terminal of a second user as a counterpart of a video call in a first terminal; Detecting a gesture of the first user or the second user in a first video stream photographing the first user or a second video stream photographing the second user received from the second terminal; Generating evaluation information on a counterpart of the user who took the detected gesture based on the detected gesture; And transmitting the generated evaluation information to a server that provides a video call service. A method of providing a video call service may be performed.

A server providing a video call service according to the second aspect includes: a communication interface for communicating with a terminal supporting video call; storage; Processor; And a memory for storing instructions executable by the processor, wherein the processor, by executing the instructions, receives evaluation information for a user using a video call service from the terminal, and the received evaluation information Based on this, the cumulative evaluation information and reputation index corresponding to the user's account stored in the storage are updated, and the video call service is managed based on the updated reputation index.

The non-transitory computer-readable storage medium storing instructions executable by the processor according to the third aspect establishes a video call session between the first terminal of the first user and the second terminal of the second user, the other party of the video call. Instructions; Instructions for detecting a gesture of the first user or the second user in a first image stream photographing the first user or a second image stream photographing the second user, received from the second terminal; Instructions for generating evaluation information on a counterpart of the user who took the detected gesture based on the detected gesture; And instructions for transmitting the generated evaluation information to a server providing a video call service.

1 is a view for explaining an environment in which a video call service is provided.
2 is a block diagram showing the configuration of a terminal according to an embodiment.
3 is a block diagram showing a configuration of a server providing a video call service according to an embodiment.
4 is a view for explaining a state in which a server providing a video call service according to another embodiment is implemented with a plurality of distributed servers.
5 is a diagram for explaining a process of generating evaluation information for a second user based on a gesture of a first user according to an embodiment.
6 is a diagram for explaining a process of generating evaluation information for a first user based on a gesture of a second user according to an embodiment.
7 is a view for explaining the type of information stored in the storage of the server providing a video call service according to an embodiment.
8 is a diagram for explaining a process of managing a video call service based on a reputation index of a user based on evaluation information received from a terminal according to an embodiment.
9 is a flowchart for explaining a method of providing a video call service by generating evaluation information corresponding to a gesture of a user of a video call service according to an embodiment.
10 is a flowchart illustrating a method of providing a video call service based on a reputation index of a user according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the drawings. The embodiments described below may be implemented in various different forms. In order to more clearly describe the features of the embodiments, detailed descriptions of the matters well known to those skilled in the art to which the following embodiments pertain will be omitted.

Meanwhile, in the present specification, when a component is "connected" with another component, this includes not only a case of'directly connecting', but also a case of'connecting with other components in between'. In addition, when it is said that a configuration "includes" another configuration, this means that unless otherwise stated, other configurations may be excluded and other configurations may be included.

In addition, terms including an ordinal number such as'first' or'second' used in the present specification can be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

The embodiments of the present invention relate to a terminal providing a video call service and a server providing a video call service, and detailed descriptions are omitted for matters well known to those skilled in the art to which the following embodiments pertain. do.

1 is a view for explaining an environment in which a video call service is provided.

The video call service refers to a service in which a plurality of users can transmit and receive their intentions by transmitting their video to the other party and receiving the other party's video using their respective terminals. Users using the video call service can send and receive video and audio through their terminals, and send and receive text through the chat function. A user who wants to use the video call service can use the video call service with the other party by directly designating the other party or by designating the other party according to a random or predetermined method in a server providing the video call service.

Referring to FIG. 1, the terminal 100 is connected to a server 200 that provides a video call service through a communication network. The server 200 may store various programs or applications and data to help a plurality of users use a video call service using their respective terminals 100. The server 200 that provides a video call service can perform both short-range communication and remote communication. The server 200 that provides a video call service may be connected to a plurality of terminals 100 through a communication network. The terminal 100 may be various types of user terminals that can be used to connect to the server 200 that provides a video call service. For example, the terminal 100 is a device capable of communicating with the server 200 that provides a video call service, a wearable device such as a smart watch, a mobile device such as a smart phone, tablet PC, laptop computer, desktop It can be a stationary device such as a computer. In addition, the terminal 100 may be a video call device that supports video calls, capable of shooting and playing video, so that video calls can be made between users connected through a video call service.

2 is a block diagram showing the configuration of a terminal 100 according to an embodiment.

Referring to FIG. 2, the terminal 100 may include a memory 110, a processor 120, a user interface 130, a communication interface 140, and a voice and video input unit 150. Those of ordinary skill in the art related to this embodiment can see that other general-purpose components may be further included in addition to the components shown in FIG. 2.

The memory 110 may store software or programs. For example, the memory 110 may store programs such as applications, application programming interfaces (APIs), and various types of data. The memory 110 may store instructions executable by the processor 120.

The processor 120 may execute instructions stored in the memory 110. The processor 120 may use various programs, contents, and data stored in the memory 110, or may store new programs, contents, and data in the memory 110.

The processor 120 may access the memory 110 and boot using the O/S stored in the memory 110. The processor 120 may perform various operations using various programs, contents, and data stored in the memory 110. For example, the processor 120 may display a predetermined screen on the display 136 using various programs, contents, and data stored in the memory 110. When a user manipulation of a region of the display 136 is performed, the processor 120 may perform a control operation corresponding to the manipulation of the user.

The processor 120 may include a graphic processing unit (GPU) specialized for graphic processing. When the booting of the terminal 100 is completed, the GPU displays a user interface screen in an area of the display 136. Specifically, the GPU may generate a screen displaying a video call layout including various objects such as content, icons, menus, and the like. The GPU may calculate attribute values such as coordinate values, shapes, sizes, colors, etc. to be displayed according to the video call layout of the screen. In addition, the GPU may generate screens of various layouts including objects based on the calculated attribute values. The screen generated by the GPU is provided to the display 136, and may be displayed on each area of the display 136, respectively.

Meanwhile, the processor 120 may include a video processor and an audio processor. The processor 120 may control the video processor and the audio processor to process video data or audio data included in the video stream received through the communication interface 140 or the video stream stored in the memory 110.

The user interface 130 may include an input unit 131 and an output unit 135.

The input unit 131 may receive various commands from a user. The input unit 131 may include at least one of a keypad 132, a touch panel 133, and a pen recognition panel 134.

The keypad 132 may include various types of keys such as mechanical buttons, wheels, etc. formed in various areas such as a front portion, a side portion, and a rear portion of the body exterior of the terminal 100.

The touch panel 133 may detect a user's touch input and output a touch event value corresponding to the sensed touch signal. When the touch panel 133 forms a touch screen in combination with a display panel, the touch screen may be implemented with various types of touch sensors such as capacitive, pressure-sensitive, and piezoelectric types.

The pen recognition panel 134 may detect the proximity input or touch input of the pen according to the operation of the user's touch pen (for example, a stylus pen) and output the detected pen proximity event or pen touch event. . The pen recognition panel 134 may be implemented by, for example, an EMR method, and sense a touch or proximity input according to a change in the intensity of the electromagnetic field due to proximity or touch of the pen. The pen recognition panel 134 may include an electromagnetic induction coil sensor having a grid structure and an electronic signal processing unit that sequentially provides an AC signal having a predetermined frequency to each loop coil of the electromagnetic induction coil sensor.

The output unit 135 may include a display 136 and a speaker 137.

The display 136 may include a display panel and a controller that controls the display panel. The display panel may be implemented in various ways such as a liquid crystal display (LCD), organic light emitting diodes (OLED) display, active-matrix organic light-emitting diode (AM-OLED), plasma display panel (PDP), and the like. The display panel can be implemented to be flexible or wearable. The display 136 may be combined with the touch panel 133 of the input unit 131 and provided as a touch screen.

The speaker 137 may output sound based on audio data. For example, the speaker 137 may output a user's voice according to audio data included in the video stream.

The communication interface 140 may communicate with various types of external devices according to various types of communication methods. The communication interface 140 may include at least one of a Wi-Fi chip, a Bluetooth chip, an NFC chip, and a wireless communication chip. The processor 120 may communicate with various external devices using the communication interface 140.

The Wi-Fi chip and the Bluetooth chip can perform communication using a WiFi method and a Bluetooth method, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various connection information such as an SSID and a session key may be first transmitted and received, and communication information may be used to transmit and receive various information. The NFC chip means a chip operating in a Near Field Communication (NFC) method using a 13.56 MHz band among various RF-ID frequency bands. The wireless communication chip communicates according to various communication standards such as the Institute of Electrical and Electronics Engineers (IEEE), Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), 5th Generation (5G), etc. Means a chip to do it.

The audio and video input unit 150 may include a microphone 151 and a camera 153. The microphone 151 may receive a user voice or other sound and convert it into audio data. The processor 120 may use a user voice input through the microphone 151 for a video call or convert it into audio data and store it in the memory 110. The camera 153 may capture a still image or a video according to the user's control. The camera 153 may be a camera module located at the front or rear of the terminal 100. The processor 120 may generate a video stream for a video call by using the voice input through the microphone 151 and the image captured by the camera 153.

Meanwhile, the terminal 100 may operate in a motion control mode or a voice control mode. When operating in the motion control mode, the processor 120 may activate the camera 153 to photograph a user, track a user's motion change, and perform a corresponding control operation. When operating in the voice control mode, the processor 120 may analyze the user voice input through the microphone 151 and perform a control operation according to the analyzed user voice.

The names of the components of the above-described terminal 100 may be different. In addition, the terminal 100 according to the present disclosure may be configured to include at least one of the above-described components, and some components may be omitted or further include other components. The terminal 100 may perform the following operation using at least one of the above-described components.

3 is a block diagram showing the configuration of a server 200 that provides a video call service according to an embodiment.

Referring to FIG. 3, a server 200 that provides a video call service includes a memory 210, a processor 220, storage 230, and a communication interface 240. Those of ordinary skill in the art related to this embodiment can see that other general-purpose components may be further included in addition to the components shown in FIG. 3. Each component of the block diagram of FIG. 3 may be separated, added, or omitted depending on the implementation method of the server 200 that provides a video call service. That is, depending on the implementation method, one component may be subdivided into two or more components, or two or more components may be combined into one component, and some components may be further added or removed.

The memory 210 may store instructions executable by the processor 220. The memory 210 may store software or programs.

The processor 220 may execute instructions stored in the memory 210. The processor 220 may perform overall control of the server 200 that provides a video call service. The processor 220 may obtain information and requests received through the communication interface 240 and store the received information in the storage 230. Also, the processor 220 may process the received information. For example, the processor 220 may generate information used for a video call service from information received from the terminal 100 or perform processing to manage the received information and store it in the storage 230. . In addition, the processor 220 in response to the request obtained from the terminal 100, using the information stored in the storage 230, to provide a video call service to the terminal 100 through the communication interface 240 Information.

The storage 230 may store various software and information necessary for the server 200 providing the video call service to provide the video call service. For example, the storage 230 may store programs, applications, and various data used in the video call service, which are executed in the server 200 that provides the video call service.

The storage 230 may store and manage personal information for each user using a video call service as a database. The storage 230 may store personal information of a user and various information used for a video call service for each account for accessing the server 200 that provides a video call service.

The communication interface 240 may communicate with external devices including the terminal 100. For example, the server 200 that provides a video call service receives a request for initiating a video call service from the terminal 100, a request for setting information to prepare a video call service environment, and responds to the request of the terminal 100. In response, it is possible to provide all matters related to the video call service.

4 is a view for explaining a state in which the server 200 providing a video call service is implemented as a plurality of distributed servers according to another embodiment. The contents described in connection with the server 200 that provides a video call service as described above may be applied as it is even if omitted.

Referring to FIG. 4, distributed servers providing video call services may be composed of a load balancing server 200-1 and function servers 200200, 200-5, and 200-7 providing video call services. have. The load balancing server 200-1 is a functional server (200-3, 200-5, 200-7) that provides a video call service when there is a request for a video call service of an external device such as the terminal 100 Select any server among them to connect to the terminal 100, or monitor the status of function servers (200-3, 200-5, 200-7) that provide video call services to select the optimal server It can be connected to (100).

5 is a diagram for explaining a process of generating evaluation information for a second user based on a gesture of a first user according to an embodiment.

Referring to FIG. 5, the first user and the second user who is the other party of the video call can make a video call through the first terminal 100-1 and the second terminal 100-2, respectively. The first terminal 100-1 and the second terminal 100-2 may be installed with an application for providing a video call service provided by a video call service provider. The first terminal 100-1 and the second terminal 100-2 may be connected to a server 200 that provides a video call service by executing an application for providing a video call service, and the server 200 You can receive the video call service provided.

The first terminal 100-1 may transmit a request for using the video call service to the server 200 that provides the video call service. The server 200 that provides a video call service, in response to a request for using the video call service of the first terminal 100-1, makes a video call with the second terminal 100-2 of the second user who is the other party of the video call. In order to be made, information required for a video call may be provided to the first terminal 100-1. The server 200 that provides the video call service may randomly select the other party of the video call according to a request for use of the video call service of the first terminal 100-1, or select according to a predetermined rule, or the first user It can be selected as the opponent designated by. The first user's first terminal 100-1 and the second user's second terminal 100-2 may establish a video call session with each other for a video call.

When a video call session is established between the first terminal 100-1 and the second terminal 100-2, the first video stream photographing the first user is the second terminal 100 in the first terminal 100-1. -2), and the second video stream photographing the second user may be transmitted from the second terminal 100-2 to the first terminal 100-1. The first terminal 100-1 receives the second video stream from the second terminal 100-2, and the second terminal 100-2 receives the first video stream from the first terminal 100-1. can do.

The first terminal 100-1 may acquire a first video stream photographing the first user or a second video stream photographing the second user received from the second terminal. The second terminal 100-2 may acquire a second image stream photographing the second user or a first image stream photographing the first user received from the first terminal 100-1.

The first terminal 100-1 may display the image of the second user on the screen of the first terminal 100-1 based on the second image stream, and the image of the first user based on the first image stream May be displayed in a part of the screen area where the second user's image is displayed. The second terminal 100-2 may display the image of the first user on the screen of the second terminal 100-2 based on the first image stream, and the image of the second user based on the second image stream May be displayed in a part of the screen area where the first user's image is displayed. As illustrated in FIG. 5, the first terminal 100-1 displays the image of the second user on the screen of the first terminal 100-1 and simultaneously displays the image of the first user (100- A smaller display size can be displayed on a part of the screen of 1). The second terminal 100-2 displays the image of the first user on the screen of the second terminal 100-2 and simultaneously displays the image of the second user in a partial area of the screen of the second terminal 100-2. You can display with a smaller display size.

In the first terminal 100-1 and the second terminal 100-2, when the same application for providing a video call service is executed, and a video call is made between the first user and the second user, the first terminal ( Since the operations and functions on the applications of the 100-1) and the second terminal 100-2 are the same, hereinafter, the description will focus on the first terminal 100-1, and the second terminal 100-2 will be described. Description is omitted.

When a video call service is executed between the first terminal 100-1 and the second terminal 100-2, the first terminal 100-1 is the first user in the first video stream that captured the first user. Can detect the gesture. The first terminal 100-1 may detect a gesture of at least one of a facial expression, a gesture, and an action of the first user in the first video stream photographed by the first terminal 100-1.

Gesture means, for example, a non-verbal communication that a user delivers using a face, hand, or body, and may be an expression, hand gesture, gesture, or the like. By making a gesture, the user can convey his or her intention to the other party or induce a reaction to the other party. An expression is, for example, a user expressing his or her thoughts by moving an eye, an eyebrow, a nose, a mouth, a forehead, or a ball through the face. For example, facial expressions can be blinking eyes, making the eyes wider or cold, raising the mouth, opening the mouth, biting the lips, frowning the forehead, and the like. Gestures are, for example, a user's hand taking a predetermined shape or action, and a gesture is a user's body expressing his or her thoughts by taking a predetermined shape or action. Hand gestures can be the OK sign, raising or lowering the thumb, making the'V' shape, shaking hands back and forth, and shaking hands from side to side. Gestures can include twisting your body, shaking your shoulders, and raising your arms.

Referring to FIG. 5, the first terminal 100-1 displays an image of the second user smiling on the screen of the first terminal 100-1, and the image of the first user laughing with the image of the second user It is displayed in the lower left area of the displayed screen area. The first terminal 100-1 may detect the smiley expression of the first user in the first video stream photographing the first user. For example, when the first user makes a gesture of raising the thumb, the first terminal 100-1 may detect the gesture of the first user raising the thumb in the first image stream. The first terminal 100-1 may detect the gesture of the first user in the first video stream based on the feature points on the face, hand, and body of the first user in the first video stream. The first terminal 100-1 may continuously detect the gesture of the first user in real time or at a predetermined interval.

The first terminal 100-1 may generate evaluation information for the second user that is the counterpart of the first user who took the detected gesture based on the detected gesture of the first user. The first terminal 100-1 may generate emotion information or icon information corresponding to the detected gesture as evaluation information for the second user.

Since the gesture is a user's conscious or unconscious expression of his or her intention using the face, hands, or body, each of the gestures may have corresponding emotion information. Emotion information may mean a user's emotion according to a gesture or a feeling, mood, or expression that arouses emotion. For example, Emotion information includes joy, happiness, anger, joy, suffering, sadness, depression, drowsiness, boredom, wonder, displeasure, satisfaction, smile, laugh, wink, cry, positive, negative, good (accept), dislike (Reject).

In addition, each gesture may have corresponding icon information. The icon information may be an intuitive depiction of the gesture in a shape resembling a gesture, or may be a picture or a symbol representing the emotion information corresponding to the gesture. For example, the icon information may include a figure of a shape or shape corresponding to each of the user's facial expression, hand gesture, and gesture, or may include a symbol or character expressing the emotion information.

Referring to FIG. 5, the first terminal 100-1 may generate evaluation information for the second user based on the detected smiley expression of the first user. For example, the first terminal 100-1 is an opponent of the first user who has the expression of laughing the emotion information of the'laugh' or the icon of the'smiley face icon' corresponding to the smiley expression of the first user. It can be generated as evaluation information for the second user. For another example, when the first user gestures to raise the thumb upward, the first terminal 100-1 corresponds to the gesture of raising the thumb of the first user based on the gesture of raising the thumb of the first user Emotion information called'satisfaction' or icon information called'thumbs up icon' can also be generated as a rating for the second user, the other user's opponent who took the gesture of raising the thumb. Since the gesture taken by the first user is a response to the attitude or behavior of the second user, when the first user's gesture is detected in the first video stream, the second user that is the counterpart of the first user causing the detected gesture It becomes evaluation information.

The first terminal 100-1 may transmit evaluation information for the second user generated by the first terminal 100-1 to the server 200 that provides a video call service. The first terminal 100-1 may transmit the evaluation information for the second user to the server 200 that provides a video call service in correspondence with the account of the second user. As illustrated in FIG. 5, the first terminal 100-1 is a server 200 that provides a video call service with emotion information or icon information generated based on the gesture of the first user detected in the first video stream. Can be transferred to. Similarly, the second terminal 100-2 may transmit the emotion information or icon information generated based on the gesture of the second user detected in the second video stream to the server 200 that provides a video call service. have.

Meanwhile, the first terminal 100-1 may transmit evaluation information for the second user to the second terminal 100-2. Since the evaluation information for the second user is generated in the first terminal 100-1 based on the gesture of the first user detected in the first video stream, feedback to the second user about what evaluation the second user will receive This is to induce positive conversation and positive gestures between users making video calls. Referring to FIG. 5, icon information of a “smiley face icon” corresponding to a smiley expression of the first user may be transmitted from the first terminal 100-1 to the second terminal 100-2. According to the icon information transmitted to the second terminal 100-2, the'smiley face icon' may be displayed overlapping the area where the first video stream is displayed in the second terminal 100-2. Accordingly, the second user can know that the first user, the other party of the video call, has a smiley expression, and has obtained the emotion information or icon information corresponding to the smiley expression as his evaluation information.

Conversely, the first terminal 100-1 receives evaluation information for the first user from the second terminal 100-2, and the received evaluation information for the first user is displayed in an area where the second video stream is displayed. It can be superimposed and displayed. Since the evaluation information for the first user is generated in the second terminal 100-2 based on the gesture of the second user detected in the second video stream, feedback to the first user about what evaluation the first user will receive To give. Referring to FIG. 5, when the second user smiles, the second terminal 100-2 detects the smiling expression of the second user in the second video stream photographing the second user and corresponds to the smiling expression Emotion information called'smile' or icon information called'smile icon' may be generated as evaluation information for the first user who is the other user's counterpart. The icon information called'smile icon' corresponding to the smiley expression of the second user may be transmitted from the second terminal 100-2 to the first terminal 100-1. According to the icon information transmitted to the first terminal 100-1, a'smile icon' may be displayed overlapping the area where the second video stream is displayed in the first terminal 100-1. Accordingly, the first user can know that the second user, the other party of the video call, has a smile expression, and has obtained the emotion information or icon information corresponding to the smile expression as his evaluation information.

6 is a diagram for explaining a process of generating evaluation information for a first user based on a gesture of a second user according to an embodiment.

In response to the request for the use of the video call service of the first terminal 100-1, the process of making a video call with the second terminal 100-2 of the second user who is the other party of the video call is the same as described in FIG. So it is omitted.

Referring to FIG. 6, when a video call service is executed between the first terminal 100-1 and the second terminal 100-2, the first terminal 100-1 is the second terminal 100-2. The gesture of the second user may be detected from the second video stream captured from the second user. The first terminal 100-1 may detect at least one gesture of a second user's facial expression, gesture, and action in the second image stream captured by the second terminal 100-2.

Referring to FIG. 6, the first terminal 100-1 displays an image of the second user smiling on the screen of the first terminal 100-1, and the image of the first user laughing on the image of the second user It is displayed in the lower left area of the displayed screen area. The first terminal 100-1 may detect a smiley expression of the second user from the second video stream that photographed the second user. For example, when the second user makes an OK sign gesture, the first terminal 100-1 may detect the gesture of the second user who makes an OK sign gesture in the second video stream. The first terminal 100-1 may detect the gesture of the second user in the second video stream based on the feature points on the face, hand, and body of the second user in the second video stream. The first terminal 100-1 may continuously detect a second user's gesture in real time or at a predetermined interval.

Based on the detected second user's gesture, the first terminal 100-1 may generate evaluation information for the first user that is the counterpart of the second user who took the detected gesture. The first terminal 100-1 may generate emotion information or icon information corresponding to the detected gesture as evaluation information for the first user.

Referring to FIG. 6, the first terminal 100-1 may generate evaluation information for the first user based on the detected smiley expression of the second user. The first terminal 100-1 for the first user who is the counterpart of the second user who has the expression of smiling smile information or'smile icon' corresponding to the smile expression of the second user Can be generated as evaluation information. When the second user makes an OK sign gesture, the first terminal 100-1, based on the OK sign gesture of the second user, responds to the gesture of raising the thumb of the second user to the good gesture (acceptance). Information or icon information called'OK icon' may also be generated as evaluation information for the first user, which is the counterpart of the second user who took the OK sign gesture. Since the gesture taken by the second user is a response to the attitude or behavior of the first user, when a gesture of the second user is detected in the second video stream, the first user that is the counterpart of the second user causing the detected gesture It becomes evaluation information.

The first terminal 100-1 may transmit evaluation information for the first user generated in the first terminal 100-1 to the server 200 that provides a video call service. The first terminal 100-1 may transmit the evaluation information for the first user to the server 200 that provides a video call service in correspondence with the account of the first user. As shown in FIG. 6, the first terminal 100-1 is a server 200 that provides a video call service with the emotion information or icon information generated based on the gesture of the second user detected in the second video stream. Can be transferred to. Similarly, the second terminal 100-2 may transmit the emotion information or icon information generated based on the gesture of the first user detected in the first video stream to the server 200 that provides a video call service. have.

The first terminal 100-1 may display the evaluation information for the first user generated in the first terminal 100-1 overlapping the area where the second image stream is displayed. Since the evaluation information for the first user is generated in the first terminal 100-1 based on the gesture of the second user detected in the second video stream, feedback to the first user about what evaluation the first user will receive This is to induce positive conversation and positive gestures between users making video calls. Referring to FIG. 6, as the icon information'Smile Icon' corresponding to the smiley expression of the second user is generated in the first terminal 100-1, the'Smile Icon' is the first terminal 100-1. In the second image stream may be displayed overlapping the displayed area. In addition, as the first terminal 100-1 generates'emotion' emotion information corresponding to the smiley expression of the second user in the first terminal 100-1, the sound of "the other party smiles" You can also output Accordingly, the first user can know that the second user, the other party of the video call, has a smile expression, and has obtained the emotion information or icon information corresponding to the smile expression as his evaluation information.

Referring to FIG. 6, when the first user laughs, the second terminal 100-2 detects the smiling expression of the first user in the first video stream photographing the first user and corresponds to the smiling expression Emotion information called'laugh' or icon information called'laugh face icon' may be generated as evaluation information for the second user who is the first user's counterpart with a smiley expression. The icon information of the “smiley face icon” corresponding to the smiley expression of the first user may be displayed overlapping the area where the first video stream is displayed in the second terminal 100-2. In addition, as the second terminal 100-2 generates the emotion information of'laugh' corresponding to the first user's smiley expression in the second terminal 100-2, the "other party laughs" sound. You can also print Accordingly, the second user can know that the first user, the other party of the video call, has a smiley expression, and has obtained the emotion information or icon information corresponding to the smiley expression as his evaluation information.

7 is a view for explaining the type of information stored in the storage 230 of the server 200 providing a video call service according to an embodiment.

As described above with reference to FIGS. 5 and 6, the server 200 that provides the video call service is the evaluation information for the user who uses the video call service from the first terminal 100-1 or the second terminal 100-2. Can receive. The storage 230 of the server 200 that provides a video call service can store various information required by the server 200 that provides a video call service as well as evaluation information for the user. have.

Referring to FIG. 7, the storage 230 of the server 200 that provides a video call service includes a plurality of types of cumulative evaluation information and each cumulative evaluation information for each user's account of a plurality of users using the video call service. A comprehensive reputation index reflecting the corresponding reputation index and all kinds of reputation indexes can be stored.

For example, the cumulative evaluation information may include cumulative emotion information, cumulative icon information, cumulative preference/avoidance information, and the like, depending on the type. The accumulated emotion information is, for example, information obtained by accumulating the emotion information received from the terminal 100 using a video call service, and the accumulated icon information includes, for example, icon information received from the terminal 100. It may be a cumulative form of information. Emotion information or icon information may be evaluation information for a user corresponding to a gesture taken by a counterpart of a user using a video call service in a first video stream photographing a first user or a second video stream photographing a second user. . The accumulated preference/avoidance information may be, for example, information obtained by accumulating preference/avoidance information received from the terminal 100. The preference/avoidance information may be that a user who uses a video call service directly inputs a favorable or unfavorable feeling to a counterpart of a video call during or after a video call.

The reputation index may be a measure of how a user using a video call service is evaluated by other users. The comprehensive reputation index may reflect a reputation index corresponding to each cumulative evaluation information and all kinds of reputation indices. For example, the Emotion reputation index may be obtained based on cumulative Emotion information, the Icon reputation index may be obtained based on cumulative icon information, and the Preference/Repellency index may be obtained based on Preference/Repellency information. The comprehensive reputation index may be a reputation index that comprehensively reflects the Emotion reputation index, icon reputation index, and preference/avoidance reputation index. The overall reputation index can be calculated by varying the weight according to the type of reputation index.

The personal information may include personal information such as the nationality, name, gender, and interests of the user who uses the video call service.

The cyber money information is a virtual money for a user using a video call service to use an additional service provided for a fee. For example, in order to make a video call match with a counterpart of a user's desired condition, the user must pay a predetermined cyber money.

The server 200 that provides a video call service receives evaluation information for a user who uses the video call service, and based on the received evaluation information, is stored in the storage 230 and cumulative evaluation information corresponding to the user's account. You can update your reputation index. The server 200 that provides a video call service may receive the emotion information or icon information corresponding to the gesture taken by the other party of the user using the video call service as evaluation information for the user. The server 200 that provides a video call service may obtain an updated reputation index from the updated cumulative evaluation information by varying the weight according to the type of the e-motion information or the type of the icon information.

The server 200 providing the video call service may manage the video call service based on the updated reputation index. For example, the server 200 that provides a video call service may manage the video call service by providing a video call matching supplementary service that matches a plurality of users using the video call service based on the updated reputation index. have. For another example, the server 200 that provides the video call service categorizes a plurality of users using the video call service based on the updated reputation index, and manages users corresponding to each category to which the user belongs By providing an additional service, a video call service can be managed.

Meanwhile, the server 200 providing the video call service may transmit the updated cumulative evaluation information or the updated reputation index to the terminal 100 through the communication interface 240.

8 is a diagram for explaining a process of managing a video call service based on a reputation index of a user based on evaluation information received from a terminal according to an embodiment.

The server 200 that provides a video call service may receive evaluation information for a user using the video call service, and may update cumulative evaluation information stored in the storage 230 and corresponding to the user's account. The server 200 providing the video call service may update the reputation index corresponding to the user's account based on the updated cumulative evaluation information corresponding to the user's account.

Referring to FIG. 8, the server 200 that provides a video call service is first user evaluation information, such as'smile','good (accept)','motion icon', and'OK icon'. Icon information can be received. The server 200 that provides a video call service may update the received emotion information and icon information to the accumulated emotion information and the accumulated icon information. Accordingly, in the space corresponding to the first user account in the storage 230,'smile','good (accept)','sadness' and'anger' are stored as cumulative emotion information, and a'smile icon' as cumulative icon information ','OK icon','Angry icon','Tear icon' can be stored. Meanwhile,'down', which means avoidance or dislike, may be stored twice in the cumulative preference/avoidance information. The server 200 providing the video call service may update the reputation index based on the updated cumulative evaluation information corresponding to the first user account. The server 200 that provides a video call service varies the weight for each of the'smile','good (accept)','sadness', and'anger', which are cumulative emotion information corresponding to the first user account. Emotion reputation index corresponding to the user account may be obtained. The server 200 that provides a video call service is provided with different weights for each of the'smile icon','OK icon','angry icon', and'tear icon', which are cumulative icon information corresponding to the first user account. 1 The icon reputation index corresponding to the user account can be obtained. The server 200 that provides a video call service may acquire a preference/avoidance index corresponding to the first user account based on the two'downs' of the cumulative preference/avoidance information corresponding to the first user account. . The server 200 that provides a video call service may obtain a comprehensive reputation index corresponding to the first user account by considering all of the Emotion reputation index, icon reputation index, and preference/avoidance index corresponding to the first user account. . The server 200 that provides the video call service may calculate the overall reputation index by varying the weight according to the type of each reputation index corresponding to the first user account.

Referring to FIG. 8, the server 200 that provides a video call service is second user evaluation information, such as'smiling','satisfied', emotion information,'smiling face icon', and'thumbs up icon' Information can be received. The server 200 that provides a video call service may update the received emotion information and icon information to the accumulated emotion information and the accumulated icon information. Accordingly, in the space corresponding to the second user account in the storage 230,'laughs','satisfaction','sleepiness' and'unpleasance' are stored as cumulative emotion information, and a'smiley face icon' as cumulative icon information, 'Thumbs up icon','Yawn icon', and'Upset icon' can be stored. Meanwhile,'up', which means preference or favor, may be stored twice in the cumulative preference/avoidance information. The server 200 providing the video call service may update the reputation index based on the updated cumulative evaluation information corresponding to the second user account. The server 200 that provides a video call service has different weights for the accumulated laugh information corresponding to the second user account,'laugh','satisfaction','sleepiness', and'uncomfortable', to the second user account. The corresponding Emotion reputation index can be obtained. The server 200 that provides a video call service has different weights for each of the'smiley face icon','thumbs up icon','yawn icon', and'upset icon', which are cumulative icon information corresponding to the second user account. Thus, an icon reputation index corresponding to the second user account can be obtained. The server 200 providing the video call service may acquire the preference/avoidance index corresponding to the second user account based on the two'ups', which are cumulative preference/avoidance information corresponding to the second user account. . The server 200 that provides a video call service may obtain a comprehensive reputation index corresponding to the second user account by considering all of the Emotion reputation index, icon reputation index, and preference/avoidance index corresponding to the second user account. . The server 200 that provides a video call service may calculate the overall reputation index by varying the weight according to the type of each reputation index corresponding to the second user account.

The server 200 providing the video call service may manage the video call service based on the updated reputation index. The server 200 that provides a video call service may manage the video call service based on at least one of an Emotion reputation index, an icon reputation index, a preference/avoidance index, and a comprehensive reputation index.

Referring to FIG. 8, the server 200 that provides a video call service manages a video call service by providing a video call matching additional service that matches a plurality of users using the video call service based on the updated reputation index can do. For example, the server 200 that provides a video call service performs video call matching among people who have the same reputation index based on the updated reputation index of each of a plurality of users using the video call service, It is possible to provide a video call matching additional service such as making a video call match with a reputable user with a high reputation index for a paid user, or making a video call match with a reputable user with a high reputation index for a new video call service.

In addition, the server 200 providing the video call service categorizes a plurality of users using the video call service based on the updated reputation index, and provides a user management supplementary service corresponding to each category to which the user belongs, You can manage the video call service. For example, the server 200 providing the video call service categorizes the plurality of users using the video call service based on the updated reputation index of each of the plurality of users using the video call service, and is different for each category. For example, to provide benefits, to warn users in categories with a low reputation score to not blacklist, or to temporarily or permanently restrict the use of video calling services to users in a category with a reputation score below the reputation score threshold. A user management supplementary service corresponding to each category to which the user belongs may be provided.

9 is a flowchart for explaining a method for providing a video call service by generating evaluation information corresponding to a gesture of a user of a video call service according to an embodiment.

In block 910, the first terminal 100-1 establishes a video call session between the first terminal 100-1 of the first user and the second terminal 100-2 of the second user that is the other party of the video call. Can be.

In block 920, the first terminal 100-1 is the first user in the first video stream photographing the first user or the second video stream photographing the second user received from the second terminal 100-2. Alternatively, the gesture of the second user can be detected.

When the gesture of the first user is detected by the first terminal 100-1, the first terminal 100-1 expresses the first user's facial expression in the first video stream captured by the first terminal 100-1 , Gestures, and gestures.

When the gesture of the second user is detected by the first terminal 100-1, the first terminal 100-1 is the facial expression of the second user in the second video stream received from the second terminal 100-2 , Gestures, and gestures.

In block 930, the first terminal 100-1 may generate evaluation information on the counterpart of the user who took the detected gesture based on the detected gesture. The first terminal 100-1 may generate emotion information or icon information corresponding to the detected gesture as evaluation information for the other party.

When the first terminal 100-1 detects the gesture of the first user, the first terminal 100-1 may generate evaluation information for the second user based on the detected gesture.

When the gesture of the second user is detected by the first terminal 100-1, the first terminal 100-1 may generate evaluation information for the first user based on the detected gesture.

In block 940, the first terminal 100-1 may transmit the generated evaluation information to the server 200 that provides a video call service.

When the gesture of the first user is detected by the first terminal 100-1, the first terminal 100-1 associates the generated evaluation information for the second user with the account of the second user, and then the server 200. Can be transferred to.

When the gesture of the second user is detected by the first terminal 100-1, the first terminal 100-1 associates the generated evaluation information for the first user with the account of the first user, so that the server 200 Can be transferred to.

When the gesture of the first user is detected by the first terminal 100-1, the first terminal 100-1 may transmit evaluation information for the generated second user to the second terminal 100-2. . In addition, the first terminal 100-1 receives evaluation information for the first user from the second terminal 100-2, and the received evaluation information for the first user is displayed in an area where the second video stream is displayed. It can be superimposed and displayed.

When the gesture of the second user is detected by the first terminal 100-1, the first terminal 100-1 displays the generated evaluation information for the first user overlapping the area where the second video stream is displayed. can do.

10 is a flowchart illustrating a method of providing a video call service based on a reputation index of a user according to an embodiment.

In block 1010, the server 200 that provides a video call service may receive evaluation information for a user using the video call service from the terminal 100 that supports the video call. The server 200 that provides a video call service may receive the emotion information or icon information corresponding to the gesture taken by the other party of the user using the video call service as evaluation information for the user.

In block 1020, the server 200 providing the video call service may update the accumulated evaluation information and reputation index corresponding to the user's account, stored in the storage 230, based on the received evaluation information. The server 200 that provides a video call service may obtain an updated reputation index from the updated cumulative evaluation information by varying the weight according to the type of the e-motion information or the type of the icon information.

In block 1030, the server 200 that provides the video call service may manage the video call service based on the updated reputation index. For example, the server 200 that provides a video call service may manage the video call service by providing a video call matching supplementary service that matches a plurality of users using the video call service based on the updated reputation index. have. For another example, the server 200 providing the video call service categorizes a plurality of users using the video call service based on the updated reputation index, and provides a user management supplementary service corresponding to each category to which the user belongs. By providing, it is possible to manage the video call service.

The server 200 providing the video call service may transmit the updated cumulative evaluation information or the updated reputation index to the terminal using the video call service through the communication interface 240.

Embodiments of a method for providing a video call service by generating evaluation information corresponding to a gesture of a user of the video call service described above correspond to a user's gesture of the video call service in the terminal 100 that provides the video call service It can be provided in the form of an application stored in a computer-readable storage medium to perform a method of providing a video call service by generating evaluation information. In other words, the terminal 100 may be provided in the form of an application or computer program stored in a computer-readable storage medium for executing each step of the method for providing the video call service described above.

Embodiments of a method for providing a video call service based on a user's reputation index are provided in a computer readable storage medium so that the server 200 can perform a method of providing a video call service based on the user's reputation index. It may be provided in the form of a stored computer program. In other words, the server 200 may be provided in the form of a computer program or application stored in a computer-readable storage medium for executing each step of the method for providing the video call service described above.

The above-described embodiments may be implemented in the form of a computer-readable storage medium that stores instructions or data executable by a computer or processor. At least one of the instructions and data may be stored in the form of program code, and when executed by a processor, a predetermined program module may be generated to perform a predetermined operation. Computer-readable storage media such as read-only memory (ROM), random-access memory (RAM), flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs , DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetic tapes, floppy disks, magneto-optical data storage devices, An optical data storage device, a hard disk, a solid-state disk (SSD), and instructions or software, associated data, data files, and data structures, and instructions or instructions to the processor or computer to enable the processor or computer to execute the instructions or It can be any device capable of providing software, related data, data files, and data structures.

So far, we have focused on the embodiments. Those of ordinary skill in the art to which the disclosed embodiments belong will understand that the disclosed embodiments may be implemented in a modified form without departing from essential characteristics. Therefore, the disclosed embodiments should be considered in terms of explanation, not limitation. The scope of the invention is shown in the claims rather than the description of the above-described embodiments, and all differences within the equivalent scope should be interpreted as being included in the scope of the invention.

Claims (15)

In the first terminal,
Establishing a video call session between the first user's first terminal and a second user's second terminal that is a video call counterpart;
Detecting a gesture of the first user or the second user in a first video stream photographing the first user or a second video stream photographing the second user received from the second terminal;
Generating evaluation information on the counterpart of the user who took the detected gesture, wherein the evaluation information is generated based on the detected gesture taken by the user by the counterpart; And
Transmitting the generated evaluation information to a server providing a video call service;
An application stored in a computer-readable storage medium to perform a method of providing a video call service comprising a. According to claim 1,
The step of generating evaluation information for the other party may include:
An application stored in a computer-readable storage medium that generates emotion information or icon information corresponding to the detected gesture as evaluation information for the other party. According to claim 1,
When detecting the gesture of the first user,
Step of detecting the gesture,
Detecting at least one gesture of a facial expression, a gesture, and an action of the first user in the first video stream photographed by the first terminal,
The step of generating evaluation information for the counterpart may include:
Based on the detected gesture, evaluation information for the second user is generated,
The step of transmitting to the server,
An application stored in a computer-readable storage medium that transmits the evaluation information for the generated second user to the server in association with the account of the second user. The method of claim 3,
And transmitting evaluation information for the generated second user to the second terminal. The method of claim 3,
Receiving evaluation information for a first user from the second terminal; And
Displaying the evaluation information for the first user superimposed on an area where the second video stream is displayed;
An application stored in a computer-readable storage medium, including. According to claim 1,
When detecting the gesture of the second user,
Step of detecting the gesture,
Detecting at least one gesture of the expression, gesture, and behavior of the second user in the second video stream received from the second terminal,
The step of generating evaluation information for the other party may include:
Based on the detected gesture, evaluation information for the first user is generated,
The step of transmitting to the server,
An application stored in a computer-readable storage medium that transmits the generated evaluation information for the first user to the server in association with the account of the first user. The method of claim 6,
And superimposing and displaying evaluation information for the generated first user on an area in which the second video stream is displayed, an application stored in a computer-readable storage medium. A communication interface performing communication with a terminal supporting video calling;
storage;
Processor; And
It includes; a memory for storing instructions executable by the processor;
The processor, by executing the instructions, receives evaluation information about a user who uses a video call service from the terminal, wherein the evaluation information is generated based on a gesture taken by a counterpart of the video call service triggered by the user. , Update the cumulative evaluation information and reputation index corresponding to the user's account, stored in the storage, based on the received evaluation information, and manage the video call service based on the updated reputation index. A server that provides call services. The method of claim 8,
The processor,
A server for providing a video call service, which receives, as evaluation information for the user, emoji information or icon information corresponding to a gesture taken by the other party of the user using the video call service. The method of claim 9,
The processor,
A server that provides a video call service by obtaining the updated reputation index from the updated cumulative evaluation information by varying weights according to the type of the emotion information or the type of the icon information. The method of claim 8,
The processor,
A server providing a video call service, managing the video call service by providing a video call matching supplementary service that matches a plurality of users using the video call service based on the updated reputation index. The method of claim 8,
The processor,
Video call service that manages the video call service by categorizing a plurality of users using the video call service and providing a user management supplementary service corresponding to each category to which each user belongs, based on the updated reputation index Server that provides. The method of claim 8,
The processor,
A server providing a video call service, transmitting the updated cumulative evaluation information or the updated reputation index to the terminal through the communication interface. The method of claim 8,
The storage,
For each account of each of a plurality of users using the video call service, a video call storing a plurality of types of cumulative evaluation information, a reputation index corresponding to each cumulative evaluation information, and a comprehensive reputation index reflecting all kinds of reputation indices Server providing the service. Instructions for establishing a video call session between a first terminal of a first user and a second terminal of a second user that is a counterpart of a video call;
Instructions for detecting a gesture of the first user or the second user in a first image stream photographing the first user or a second image stream photographing the second user, received from the second terminal;
Instructions for generating evaluation information on the counterpart of the user who took the detected gesture, wherein the evaluation information is generated based on the detected gesture taken by the user by the counterpart; And
Instructions for transmitting the generated evaluation information to a server providing a video call service;
A non-transitory computer readable storage medium storing instructions executable by a processor comprising a.

Images