KR20180123998A - Image displaying method, apparatus and computer readable recording medium - Google Patents

Abstract

Disclosed are a method and an apparatus for displaying an image which can induce a sound video call between users, and a computer readable recording medium. According to an embodiment of the present invention, the method for displaying an image comprises the steps of: receiving, from a first terminal, a stream including a plurality of images photographed by the first terminal; detecting a face from each of one or more sample images selected among the plurality of images included in the stream; detecting a skin color from each of the sample images; processing the image in an area where the skin color is detected in response to a result of detecting the face and the skin color; and displaying the plurality of images included in the stream.

Description

[0001] IMAGE DISPLAYING METHOD, APPARATUS AND COMPUTER READABLE RECORDING MEDIUM [0002]

The present invention relates to an image display method, an apparatus, and a computer-readable recording medium for preventing a user performing a video call from being exposed to an inappropriate image by a video communication party.

With the development of communication technology and the miniaturization of electronic devices, personal terminals have become widespread to general consumers. In recent years, portable personal terminals such as a smart phone or a smart tablet have become widespread. Most of the terminals include a communication function. The user can perform a search on the Internet or exchange messages with other users using the terminal.

In addition, with the development of small camera technology, small microphone technology, small display technology and small speaker technology, most terminals such as smart phones include cameras, microphones, displays and speakers. A user can record a voice using the terminal or a video including voice. The user can check the recorded voice through a speaker included in the terminal or confirm the photographed moving picture through a display.

Also, the user can share the recorded voice or the photographed moving picture with other users by using the communication function of the terminal. The user can transmit a previously recorded voice or a previously photographed video to another user. In addition, the user can transmit the voice currently being recorded by the terminal or the moving picture currently being photographed to another user in real time.

Also, at the same time, the other user can transmit the voice currently being recorded by the terminal or the moving picture currently being photographed to the user in real time. The display included in the user terminal can simultaneously display the moving image currently being photographed by the user's terminal and the moving image currently being photographed by another user's terminal. In addition, the speaker included in the user terminal can simultaneously reproduce the voice currently being recorded by the terminal of another user. In other words, users and other users can make video calls with each other using their terminals.

The user and the other user may have an already known relationship with each other or an unknown relationship with each other. A plurality of terminals including the terminal of the user and the terminal of the other user may be connected to the server. The server can mediate between the terminal of the user and the terminal of the other user. Therefore, even though the user and the other users have no relationship with each other, they can make video calls with each other through intermediation of the server.

When the user makes a video call with the other party, the user can be exposed to the obscene video from the other party. In particular, when a user makes a video call with a partner having an unknown relationship with each other, there is a high possibility that the user is exposed to obscene images from the other party.

For example, a naked body image of the other party photographed by the other party's terminal can be received at the user's terminal through a video call session. The user's terminal displays the received image so that the user can be exposed to the sexually explicit image. If the user is exposed to an obscene video that he does not want, the user may feel sexual shame or discomfort.

An object of the present invention is to provide an image display method, an apparatus, and a computer readable recording medium for preventing a user who performs a video call from being exposed to an inappropriate image by a video communication party.

It is another object of the present invention to provide an image display method, an apparatus, and a computer readable recording medium capable of preventing a user who performs a video call from feeling sexual bullies or uncomfortable images by video provided by a video caller do.

It is another object of the present invention to provide an image display method, apparatus, and computer readable recording medium that can induce a sound video call between users.

A video display method according to an embodiment of the present invention includes receiving a stream including a plurality of videos photographed by the first terminal from a first terminal, receiving at least one video selected from the plurality of videos included in the stream Detecting a face in each of the sample images, detecting a skin color in each of the sample images, performing image processing in an area in which the skin color is detected corresponding to the face and the skin color detection result, And displaying the plurality of images.

If the face is not detected in the step of detecting the face and the skin color is detected in the step of detecting the skin color, the image processing may be performed in the region where the skin color is detected in the step of performing the image processing .

The step of performing the image processing may include selecting at least one target image among the plurality of images included in the stream, selecting at least a part of target areas in each target image, And performing the image processing on the selected target area of the target image, wherein the target area may correspond to the area where the skin color is detected.

In addition, the step of detecting the face may include the steps of: selecting at least one sample stream corresponding to a recent specific time among the plurality of images included in the stream; and detecting the face in the selected at least one sample image Step < / RTI >

In addition, the step of detecting the face includes selecting a new sample stream among the newly received images in the stream for each particular period, and detecting the face in the new sample stream each time the new sample stream is selected can do.

In addition, the image processing may correspond to an operation of applying a blur effect or a mosaic effect, or adjusting an hue, brightness, contrast, or saturation.

According to another aspect of the present invention, there is provided an image display method including: receiving a video stream including a plurality of consecutive images from a second terminal, in which a first terminal establishes a video call session with the first terminal; Determining whether a facial image exists in each of the plurality of images by the first terminal, detecting a skin color in an image in which the facial image does not exist, performing image processing on the image in which the facial image does not exist And displaying the video stream on which the image processing has been performed through the output unit of the first terminal.

In addition, the image processing may correspond to an operation of applying a blur effect or a mosaic effect, or adjusting an hue, brightness, contrast, or saturation.

Meanwhile, a computer-readable recording medium on which a program for performing an image display method according to the present invention is recorded can be provided.

According to another aspect of the present invention, there is provided an image display apparatus including a communication unit that receives a stream including a plurality of images taken by the first terminal from a first terminal, a communication unit that displays the plurality of images included in the stream And a controller for detecting a face in each of at least one or more sample images selected from among the plurality of images included in the stream, wherein the controller is configured to determine whether the face is detected in at least some sample images of the sample images The image processing unit performs image processing on at least one image among the plurality of images included in the stream, and the output unit displays the at least one image on which the image processing is performed.

Also, the controller may perform the image processing on a region where the skin color is detected when a skin color is detected from a sample image in which the face is not detected.

In addition, the image processing may correspond to an operation of applying a blur effect or a mosaic effect, or adjusting an hue, brightness, contrast, or saturation.

The present invention can provide an image display method, an apparatus, and a computer readable recording medium for preventing a user who performs a video call from being exposed to an inappropriate image by a video communication party.

In addition, the present invention can provide an image display method, an apparatus, and a computer-readable recording medium that can prevent a user who performs a video call from feeling a sexual shame or an unpleasantness by an image provided by a video caller.

In addition, the present invention can provide an image display method, an apparatus, and a computer readable recording medium that can induce a sound video call between users.

1 is a network configuration diagram illustrating an environment in which an image display apparatus according to an embodiment operates.
2 is a block diagram showing a configuration of a video display device according to an embodiment.
3 is a flowchart illustrating a process of performing an image display method according to an embodiment.
4 is a sample image selected from a plurality of images included in the stream according to the embodiment.
5 is another sample image selected from a plurality of images included in the stream according to the embodiment.
6 is a flowchart illustrating a process of performing an image process according to an embodiment.
FIG. 7 is a flowchart illustrating a process of detecting a face and selecting a target image according to an exemplary embodiment of the present invention.
8 is a reference diagram for explaining a method of selecting a sample image according to an embodiment.
FIG. 9 is a flowchart illustrating a process of detecting a face and selecting a target image according to another embodiment.
FIG. 10 is a flowchart illustrating a process of detecting a facial image and selecting a target image according to another embodiment of the present invention.
11 is a reference diagram for explaining a method of selecting a sample image according to another embodiment.
FIG. 12 is a flowchart illustrating a process of detecting a facial face and selecting a target image according to another embodiment.
13 is a reference view showing a selected object area according to an embodiment.
14 is a reference view showing a selected object area according to another embodiment.
15 is a reference view showing a selected object area according to another embodiment.
16 is an image in which the image processing according to the embodiment is performed.
17 is an image in which image processing according to another embodiment is performed.
18 is an image in which image processing according to another embodiment is performed.
FIG. 19 is an image in which image processing according to another embodiment is performed.
20 is a flowchart illustrating a process of performing an image display method according to another embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Although "first" or "second" and the like are used to describe various components, such components are not limited by such terms. Such terms may be used to distinguish one element from another. Therefore, the first component mentioned below may be the second component within the technical spirit of the present invention.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. &Quot; comprises "or" comprising "as used herein mean that the stated element or step does not exclude the presence or addition of one or more other elements or steps.

Unless defined otherwise, all terms used herein are to be construed in a sense that is commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

Hereinafter, the video display devices 100 and 150 and the video display method according to the embodiment will be described in detail with reference to FIGS. 1 to 20. FIG.

1 is a network configuration diagram illustrating an environment in which the video display devices 100 and 150 according to the embodiment operate. 1, an environment in which the image display devices 100 and 150 operate includes a first image display device 100 and a second image display device 150 connected to the first image display device 100 .

According to another embodiment, the environment in which the video display devices 100 and 150 operate may further include a server 200 connected to the first video display device 100 or the second video display device 150 . The environment in which the image display apparatuses 100 and 150 operate may not include the server 200.

The first image display apparatus 100 may be a terminal. Hereinafter, for convenience of description, the first video display device 100 and the first terminal 100 are used as a combination of the first video display device 100 and the first video display device 100. [

The first terminal 100 may be, for example, a desktop computer, a laptop computer, a smart phone, a smart tablet, a smart watch, a mobile terminal, a digital camera, a wearable device, or a portable electronic device. The first terminal 100 may execute a program or an application.

The first terminal 100 may be connected to a communication network. The first terminal 100 may be connected to another external device through the communication network. The first terminal 100 may transmit data to or receive data from other devices connected to each other.

The communication network connected to the first terminal 100 may include a wired communication network, a wireless communication network, or a hybrid communication network. The communication network may include a mobile communication network such as 3G, LTE, or LTE-A. The communication network may include wired or wireless communication networks such as Wi-Fi, UMTS / GPRS, or Ethernet. The communication network can be classified into MST (Secure Transmission), RFID (Radio Frequency IDentification), NFC (Near Field Communication), ZigBee, Z-Wave, Bluetooth, Bluetooth low energy (BLE) , Or a local area network such as infrared (IR) communication. The communication network may include a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN).

2 is a block diagram showing the configuration of the video display devices 100 and 150 according to the embodiment. 2, the image display apparatuses 100 and 150 according to the embodiment may include an input unit 101, a control unit 102, a storage unit 103, a communication unit 104, and an output unit 105 have.

The input unit 101 can receive a signal from the outside. The input unit 101 may receive a signal from a user of the video display devices 100 and 150. [ Further, the input unit 101 can receive a signal from an external device. The input unit 101 may include, for example, a microphone, a camera, a keyboard, a mouse, a trackball, a touch screen, a button, a switch, a sensor, a network interface or other input device.

The input unit 101 can receive light from the outside through a camera included in the input unit 101. [ The input unit 101 can receive light emitted from an external object or light reflected from an external object. The input unit 101 may photograph an image of a user of the image display apparatuses 100 and 150 through the camera.

The input unit 101 can receive a voice from outside through a microphone included in the input unit 101. [ The input unit 101 can receive a voice from a user of the video display devices 100 and 150. [ Further, the input unit 101 can receive voice from an external audio reproduction apparatus. The input unit 101 can record the user's voice of the video display devices 100 and 150 through the microphone.

The control unit 102 can control the operation of the video display devices 100 and 150. [ The control unit 102 may be connected to each of the components included in the image display apparatuses 100 and 150. The control unit 102 may control the operation of each component included in the video display devices 100 and 150.

The control unit 102 can process the signal. The control unit 102 can process the signal received by the input unit 101. [ For example, the control unit 102 may process the user's request received through the input unit 101. [ The control unit 102 can process the user's image captured by the camera included in the input unit 101. [ The control unit 102 can process the voice of the user recorded by the microphone included in the input unit 101. [

The control unit 102 can process the signal received by the communication unit 104. [ For example, the control unit 102 may process the user's request received through the communication unit 104. [ The control unit 102 can process video or audio of another user received through the communication unit 104. [

The control unit 102 can control the operation of the video display devices 100 and 150 in response to the signal received by the input unit 101. [ The control unit 102 can control the operation of the video display devices 100 and 150 in response to the user's request received through the input unit 101. [

The control unit 102 may perform an operation. The control unit 102 may perform an operation according to a signal received by the input unit 101. [ The control unit 102 may perform an operation using the signal received by the input unit 101 or the data stored in the storage unit 103. For example, the control unit 102 may encode an image captured by a camera included in the input unit 101. [ The control unit 102 can encode the voice recorded by the microphone included in the input unit 101. [

The control unit 102 can decode the video or audio of another user received by the communication unit 104. [ The control unit 102 may perform postprocessing on the decoded image or voice. For example, the control unit 102 may perform image processing on an image of another user received by the communication unit 104. [

The control unit 102 can control the calculation result to be stored in the storage unit 103. [ The control unit 102 can control so that the calculation result is outputted by the output unit 105. [

The control unit 102 includes, for example, a central processing unit (CPU), a graphic processing unit (GPU), a microcontroller unit (MCU), or a microprocessor can do.

The storage unit 103 may store data. The storage unit 103 may store data to be used by the control unit 102 to perform the operation. The storage unit 103 may store the operation result performed by the control unit 102. For example, the storage unit 103 may store an image or a voice encoded by the control unit 102. [ The storage unit 103 may store data to be transmitted to the outside through the communication unit 104 or may store data received from the outside through the communication unit 104. [

The storage unit 103 may include a volatile memory or a non-volatile memory. The storage unit 103 may include, for example, a flash memory, a ROM, a RAM, an EEROM, an EPROM, an EEPROM, a hard disk drive (HDD) The storage unit 103 may include, for example, a file system, a database, or an embedded database.

The communication unit 104 can transmit data to the outside or receive data from the outside. The communication unit 104 can transmit data to the other video display devices 100 and 150 or the server 200. [ The communication unit 104 can receive data from the other video display devices 100 and 150 or the server 200. [ The communication unit 104 can transmit the calculation result performed by the control unit 102 to the outside. Also, the communication unit 104 can transmit the data stored in the storage unit 103 to the outside. The communication unit 104 can receive data from the outside for use by the control unit 102 to perform the calculation.

The data to be transmitted by the communication unit 104 or the data received by the communication unit 104 may be stored in the storage unit 103. [ For example, the communication unit 104 may transmit the encoded video or audio stored in the storage unit 103 to the server 200 or other video display devices 100 and 150. In addition, the communication unit 104 can receive images or voices encoded by the other video display devices 100 and 150 from the other video display devices 100 and 150.

The communication unit 104 may include a remote network interface such as a 3G module, an LTE module, an LTE-A module, a Wi-Fi module, a WiGig module, an UWB (Ultra Wide Band) module, . The communication unit 104 may include a short-range network interface such as a magnetic secure transmission (MST) module, a Bluetooth module, an NFC module, an RFID module, a ZigBee module, a Z-Wave module, can do. In addition, the communication unit 104 may include other network interfaces.

The output unit 105 can output a signal to the outside. The output unit 105 can display a screen, reproduce sound, or output a vibration, for example. Output 105 may include a display, a speaker, a vibrator, an oscillator, or other output device.

The output unit 105 can display a screen. The control unit 102 can control the output unit 105 so that the output unit 105 displays a screen. The output unit 105 may display a user interface. The output unit 105 can display another screen in response to the input from the user.

The output unit 105 can display data. The output unit 105 may display the result of the operation performed by the control unit 102. The output unit 105 may display the data stored in the storage unit 103. [ The output unit 105 can display the data received by the communication unit 104. [

The output unit 105 may include a flat panel display device such as a liquid crystal display (LCD), an organic light emitting diode (OLED), or a plasma display panel (PDP). The output unit 105 may include a curved display or a flexible display. The output unit 105 may include a touch screen.

The output unit 105 may display an image of a user photographed by the first image display device 100 or an image of another user photographed by the second image display device 150. [ The output unit 105 may reproduce the voice of the user recorded by the first video display device 100 or the voice of another user recorded by the second video display device 150. [

The second image display device 150 may be a terminal. Hereinafter, for the sake of convenience, the second video display device 150 and the second terminal 150 are used in combination as the terms of the second video display device 150.

The second terminal 150 may be a terminal of the same type as the first terminal 100, or may be a terminal of a different type. The second terminal 150 may execute a program or an application.

The second terminal 150 may be connected to a communication network. The second terminal 150 may be connected to another external device through the communication network. The second terminal 150 may transmit data to or receive data from other devices connected to each other.

And the second terminal 150 may be connected to the first terminal 100. [ The second terminal 150 may be connected to the first terminal 100 through the communication network. The second terminal 150 may transmit data to the first terminal 100 or receive data from the first terminal 100.

The first terminal 100 and the second terminal 150 connected to each other can exchange messages, files, voice data, images, moving images, and the like. The second terminal 150 can establish a voice call session or a video call session with the first terminal 100. [ The video call session may be established using, for example, Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Web Real-Time Communication (WebRTC). The user of the first terminal 100 and the user of the second terminal 150 can make a voice call or a video call with each other.

The video call session may be established directly between the first terminal 100 and the second terminal 150. According to another embodiment, a video call session may be established between the first terminal 100 and the second terminal 150 via at least one or more other devices. For example, the video call session may include a session established between the first terminal 100 and the server 200, and a session established between the server 200 and the second terminal 150.

The first terminal 100 and the second terminal 150 can directly exchange data with each other. Also, the first terminal 100 and the second terminal 150 can exchange data with each other via the server 200 or another intermediary device.

The server 200 may be connected to the first terminal 100 or the second terminal 150. When the server 200 is connected to the first terminal 100, the server 200 can exchange data with the first terminal 100 through a communication network. When the server 200 is connected to the second terminal 150, the server 200 can exchange data with the second terminal 150 through a communication network.

The server 200 can receive data from the first terminal 100. [ The server 200 may perform an operation using data received from the first terminal 100. The server 200 may transmit the result of the calculation to the first terminal 100 or the second terminal 150. The server 200 may receive data from the second terminal 150. The server 200 may perform an operation using data received from the second terminal 150. The server 200 may transmit the result of the calculation to the first terminal 100 or the second terminal 150.

Hereinafter, more detailed operations of the video display devices 100 and 150 in the video display method according to the embodiment will be described in detail with reference to FIGS. 3 to 20. FIG.

3 is a flowchart illustrating a process of performing an image display method according to an embodiment. Referring to FIG. 3, in the video display method according to the embodiment, first, a step S100 of receiving a stream including a plurality of videos photographed by the first terminal 100 from the first terminal 100 .

The communication unit 104 of the second terminal 150 can establish a video call session with the first terminal 100. [ The communication unit 104 of the second terminal 150 may receive the stream from the first terminal 100 through the video call session. The stream may include a plurality of images. For example, the stream may include a video or at least two still images.

The plurality of images included in the stream may be images captured by a camera included in the input unit 101 of the first terminal 100. The second terminal 150 can successively receive a plurality of images continuously captured by the first terminal 100 through the stream.

The plurality of images may be images of a user of the first terminal 100. For example, the plurality of images may include an image of a face of the user of the first terminal 100. The plurality of images may include an image of a user's body of the first terminal 100.

The second terminal 150 may receive the stream directly from the first terminal 100. [ According to another embodiment, the second terminal 150 may receive the stream from the first terminal 100 via the server 200. [

Next, a step of attempting to detect a face in each of at least one or more sample images selected from the plurality of images included in the stream (S110) may be performed. The control unit 102 of the second terminal 150 can select at least one of the plurality of images included in the stream. The control unit 102 of the second terminal 150 may select a sample image randomly or according to a specific rule among the plurality of images.

For example, the control unit 102 of the second terminal 150 may select a new sample image among the newly received images in the stream for a specific period. The controller 102 of the second terminal 150 may select a new sample image from among all the images received later than the most recently selected sample image in the stream. The control unit 102 of the second terminal 150 may select the new sample image at random or according to a specific rule.

The control unit 102 of the second terminal 150 may attempt to detect the face in each of the at least one or more selected sample images. The control unit 102 of the second terminal 150 can display a portion corresponding to the face of a person in one image using an open source library such as OpenCV (Open Source Computer Vision) It is possible to detect whether or not it exists. The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face of each sample image exists.

4 is a sample image selected from a plurality of images included in the stream according to the embodiment. 5 is another sample image selected from a plurality of images included in the stream according to the embodiment. The control unit 102 of the second terminal 150 may attempt to detect the face in each of the sample image of Fig. 4 and the sample image of Fig.

Referring again to FIG. 3, a step (S120) may be performed in which it is determined that the face is not detected in at least some sample images among the sample images. The control unit 102 of the second terminal 150 can determine whether there is a portion corresponding to the face for each sample image. As a result of the determination, the control unit 102 of the second terminal 150 can determine that the face is not detected from at least one of the sample images.

For example, the control unit 102 of the second terminal 150 may determine that the face is detected in the sample image corresponding to the upper half of the user of the first terminal 100 as in the image of FIG. However, the control unit 102 of the second terminal 150 may determine that the face is not detected in the sample image corresponding to the lower half of the user of the first terminal 100 as in the image of Fig.

Referring again to FIG. 3, step S 130 of performing image processing on at least one or more images among the plurality of images included in the stream may be performed. If it is determined that the face is not detected in at least some sample images among the sample images as described above, it may be determined that the plurality of images received from the first terminal 100 include an obscene image. For example, when a plurality of images received from the first terminal 100 include an image as shown in FIG. 5, it can be determined that the plurality of images include an image that is sexually explicit.

The control unit 102 of the second terminal 150 may perform image processing on at least one of the plurality of images to prevent the user of the second terminal 150 from being exposed to the lewd image. The control unit 102 of the second terminal 150 may perform image processing on all the plurality of images or may perform image processing on a part of the images selected from the plurality of images.

6 is a flowchart illustrating a process of performing image processing (S130) according to an embodiment. Referring to FIG. 6, at least one target image among the plurality of images included in the stream may be selected (S131).

The control unit 102 of the second terminal 150 may select at least one or more target images to be subjected to image processing among the plurality of images. For example, the control unit 102 of the second terminal 150 may select all the plurality of images included in the stream as the target image.

According to another embodiment, the control unit 102 of the second terminal 150 may perform image processing on at least one or more images corresponding to the sample images determined not to be detected from the plurality of images. The image corresponding to the sample image may be all images received within a specific time range from the time when the sample image is received from the plurality of images.

For example, it is assumed that 20 images are received consecutively per second through the stream. It is also assumed that the most recently received image is selected as a new sample image every 4 seconds cycle. It is also assumed that the face is not detected in the two sample images. The time at which the two sample images are received may be 24 seconds and 32 seconds, respectively. The image corresponding to the two sample images may be about 80 images received within the time range from the 22 second time point to the 26 second time point and about 80 images received within the time range from the 30 second time point to the 34 second time point .

The method for the controller 102 of the second terminal 150 to select the sample image and the method for selecting the target image may differ according to the embodiment. FIG. 7 is a flowchart illustrating a process of detecting a facial face (S110) to selecting a subject image (S131) according to an embodiment.

Referring to FIG. 7, a step S210 of selecting at least one sample image corresponding to a specific recent time among the plurality of images included in the stream may be performed. The control unit 102 of the second terminal 150 may select at least one or more sample images among the images received during the recent specific time in the stream.

For example, the control unit 102 of the second terminal 150 may select two sample images from the received images in the last one second. 8 is a reference diagram for explaining a method of selecting a sample image according to an embodiment. Referring to FIG. 8, the first image 301 in the stream may be already selected as a sample image in the past. The control unit 102 of the second terminal 150 additionally adds the second image 302 and the third image 303 as the sample images among the images received after the first time t1 which is a time before the specific time from the present You can choose.

Referring again to FIG. 7, a step S211 of attempting to detect the face in the selected at least one sample image may be performed. The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face is present in each of the selected sample images. For example, the control unit 102 of the second terminal 150 may attempt to detect the face in the second image 302 and the third image 303.

Next, it can be determined that the face is not detected in the selected at least one sample image (S220). The control unit 102 of the second terminal 150 may determine that there is no portion corresponding to the face in the selected sample image.

The controller 102 of the second terminal 150 may determine that the face is not detected when there is no portion corresponding to the face in all of the selected sample images. According to another embodiment, the control unit 102 of the second terminal 150 may determine that the face is not detected when there is no portion corresponding to the face in at least one or more sample images among the selected sample images. For example, the control unit 102 of the second terminal 150 may determine that the face is not detected in the second image 302 and the third image 303.

Next, a step S231 of selecting all images corresponding to the latest specific time among the plurality of images included in the stream as the target image may be performed. For example, the control unit 102 of the second terminal 150 may select all of the images received in the last one second as the target images. If 20 images are received continuously per second through the stream, the control unit 102 of the second terminal 150 can select about 20 images received in the last 1 second as the target images. Referring to FIG. 8, the control unit 102 of the second terminal 150 includes all of the images received after the time t1 before the specific time including the second image 302 and the third image 303 Can be selected as the target image.

FIG. 9 is a flowchart illustrating a process of detecting a facial face (S110) to selecting a target image (S131) according to another embodiment. Referring to FIG. 9, a new sample image among the newly received images in the stream may be selected (S310) at a specific period. The control unit 102 of the second terminal 150 may select at least one new sample image among all the images received later than the most recently selected sample image for a specific period.

For example, the control unit 102 of the second terminal 150 can select two new sample images every second. 8, the controller 102 of the second terminal 150 selects the second image 302 and the third image 303 from among all images received later than the first image 301 as a new sample image You can choose.

Referring again to FIG. 9, a step S311 of attempting to detect the face in the new sample image may be performed each time the new sample image is selected. The control unit 102 of the second terminal 150 may attempt to detect the face from the newly selected sample image at the specific period.

Next, a step S320 may be performed in which it is determined that the face is not detected in all of the sample images selected within the latest specific time among the sample images. For example, the control unit 102 of the second terminal 150 can determine that the face is not detected in the two sample images selected within the last one second. In other words, the control unit 102 of the second terminal 150 can determine that the face is not detected in the second image 302 and the third image 303.

Next, the step S331 of selecting all the images corresponding to the latest specific time among the plurality of images included in the stream as the target image may be performed. For example, the control unit 102 of the second terminal 150 may select all of the images received in the last one second as the target images. If 20 images are received continuously per second through the stream, the control unit 102 of the second terminal 150 can select about 20 images received in the last 1 second as the target images. Referring to FIG. 8, the control unit 102 of the second terminal 150 includes all of the images received after the time t1 before the specific time including the second image 302 and the third image 303 Can be selected as the target image.

FIG. 10 is a flowchart illustrating a process of detecting a facial face according to another embodiment (S110) to selecting a target image (S131). Referring to FIG. 10, a step S410 of attempting to detect the face in a first sample image selected from the plurality of images included in the stream may be performed. The control unit 102 of the second terminal 150 can select the first sample image from among the plurality of images. The control unit 102 of the second terminal 150 may detect whether or not a portion corresponding to the face in the first sample image exists.

11 is a reference diagram for explaining a method of selecting a sample image according to another embodiment. Referring to FIG. 11, the fourth image 311 in the stream may be already selected as a sample image in the past. The control unit 102 of the second terminal 150 may select the fifth image 312 among the images received later than the fourth image 311 as the first sample image.

Next, a step S411 of attempting to detect the face in each of at least one second sample image selected from all the images received later than the first sample image in the stream may be performed. The controller 102 of the second terminal 150 may select at least one second sample image from the images received later than the first sample image. The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face is present in each second sample image.

For example, referring to FIG. 11, the controller 102 of the second terminal 150 may select the sixth image 313 from the images received later than the fifth image 312 as the second sample image have. The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face in the sixth image 313 exists.

As another example, the control unit 102 of the second terminal 150 can select two images as the second sample image. The two images may include a sixth image 313 and other images received later than the fifth image 312 and received earlier than the sixth image 313. [ The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face is present in each of the two images.

Referring again to FIG. 10, a step S420 may be performed in which it is determined that the face is not detected in the first sample image and all the at least one second sample image. The control unit 102 of the second terminal 150 may determine that the face is not detected in the first sample image and the second sample image. For example, the control unit 102 of the second terminal 150 may determine that the face is not detected in the fifth image 312 and the sixth image 313.

Next, a step S431 of selecting, as the target image, an image corresponding to the first sample image and the second sample image among the plurality of images included in the stream may be performed. The control unit 102 of the second terminal 150 may select an image corresponding to the first sample image and the second sample image as the target image.

For example, an image corresponding to the first sample image and the second sample image may include all the images received later than the first sample image and the first sample image among the plurality of images. Referring to FIG. 11, the controller 102 of the second terminal 150 may select all images received later than the fifth image 312 and the fifth image 312 as the target image.

According to another embodiment, the selected object image may include the first sample image, the at least one second sample image, and the surrounding image. The peripheral image may be received later than the first sample image and may be all images received earlier than the last received image among the at least one second sample image. 11, the control unit 102 of the second terminal 150 receives the fifth image 312, the sixth image 313, and the fifth image 312, All of the received images can be selected as the target images.

FIG. 12 is a flowchart illustrating a process of performing detection of a facial face (S110) to selecting a target image (S131) according to still another embodiment. 12, first, a first sample image corresponding to a first time among the plurality of images included in the stream and a second sample image corresponding to a second time having a time difference of a predetermined time or longer from the first time (S510) may be performed to select at least two or more sample images.

The control unit 102 of the second terminal 150 can select the first sample image from among the plurality of images included in the stream. The time at which the first sample image is received may be the first time. The control unit 102 of the second terminal 150 may select the received image after a predetermined time or more from the time when the first sample image was received as the second sample image. In other words, the time difference between the time when the first sample image is received and the time when the second sample image is received may be longer than the predetermined time. The specific time may be, for example, 1 second.

In addition, the control unit 102 of the second terminal 150 can additionally select a sample image among the images received earlier than the first sample image. In addition, the control unit 102 of the second terminal 150 can additionally select a sample image from an image received later than the first sample image and received earlier than the second sample image. In addition, the control unit 102 of the second terminal 150 can additionally select a sample image among the images received later than the second sample image.

For example, referring to FIG. 11, the control unit 102 of the second terminal 150 may select an image received at an arbitrary second time point t2 in the stream as a first sample image. The control unit 102 of the second terminal 150 may select the image received at the third time point t3 among the images received after one second or more of the second time point t2 as the second sample image.

The control unit 102 of the second terminal 150 can additionally select the fourth image 311 as a sample image from the images received earlier than the second time point t2. The control unit 102 of the second terminal 150 transmits the fifth image 312 and the sixth image 313 received later than the second time point t2 and received before the third time point t3 Additional sample images can be selected. The controller 102 of the second terminal 150 can additionally select the seventh image 314 from the images received later than the third time point t3 as a sample image.

Referring again to FIG. 12, next step S511 of attempting to detect the face in each of the at least two selected sample images may be performed. The control unit 102 of the second terminal 150 may attempt to detect the face in the first sample image, the second sample image, and all the selected sample images.

Next, it is determined that the face is not detected in all of the sample images corresponding to the time from the first sample image, the second sample image, and the first time to the second time among the selected at least two sample images (S520) may be performed. The control unit 102 of the second terminal 150 may determine that the portion corresponding to the face in the first sample image received at the second time point t2 is not detected. In addition, the control unit 102 of the second terminal 150 can determine that the portion corresponding to the face in the second sample image received at the third time point t3 is not detected. The control unit 102 of the second terminal 150 may further include a fifth image 312 selected as a sample image among the images received later than the second time point t2 and received earlier than the third time point t3, 6 image 313 can not be detected. In other words, the control unit 102 of the second terminal 150 determines that the face is not detected in the sample image received during the period from the second time point t2 to the third time point t3 among all the selected sample images .

Next, the step S531 of selecting all the images corresponding to the time from the first time to the second time among the plurality of images included in the stream may be performed as the subject image. For example, the control unit 102 of the second terminal 150 may select all the images received during the time from the second time point t2 to the third time point t3 as the target images. The selected target image may include the first sample image, the second sample image, and a sample image received later than the second point of time t2 and received earlier than the third point of time t3. Also, the selected target image may include all remaining images that are received later than the second time point t2 in the stream and not selected as sample images among all the images received earlier than the third time point t3 .

Referring again to FIG. 6, after the step S131 of selecting the target image is performed, a step S132 of selecting at least a part of the target regions in each of the target images may be performed. 13 is a reference view showing a selected object area according to an embodiment. As shown in FIG. 13, the control unit 102 of the second terminal 150 can select the entire area of each of the target images as the target area.

According to another embodiment, the control unit 102 of the second terminal 150 can select an area corresponding to the skin color in each of the target images as the target area. The control unit 102 of the second terminal 150 can detect an area corresponding to the skin color in each of the target images. The skin color may indicate at least one color. Since the skin color of each person may be different from each other, the control unit 102 of the second terminal 150 can detect an area corresponding to the skin color for various skin colors.

14 is a reference view showing a selected object area according to another embodiment. As shown in Fig. 14, the control unit 102 of the second terminal 150 can select an area having a skin color in the target image as the target area. In other words, each pixel included in the selected object region may have a skin color.

The control unit 102 of the second terminal 150 may select at least one closed curve in each target image as the target area. For example, unlike the one shown in Fig. 14, regions having a skin color may include a plurality of closed lines that are adjacent to or spaced from each other. The control unit 102 of the second terminal 150 may select all of the plurality of closed lines as the target area. According to another embodiment, the control unit 102 of the second terminal 150 may select one or more closed curves having the largest area among the plurality of closed curves as the target area.

15 is a reference view showing a selected object area according to another embodiment. As shown in FIG. 15, the control unit 102 of the second terminal 150 can select a rectangular area including a skin color area in the target image as the target image. According to another embodiment, the control unit 102 of the second terminal 150 may select a circular, elliptical, or polygonal region including the skin color region as the target image, instead of the rectangular region.

The control unit 102 of the second terminal 150 may select at least one polygonal area within each object image as the object area. For example, unlike the one shown in Fig. 15, regions having a skin color may include a plurality of closed lines that are adjacent to or spaced from each other. The control unit 102 of the second terminal 150 may select one polygonal area including all of the plurality of closed lines as the target area. According to another embodiment, the control unit 102 of the second terminal 150 can select a plurality of polygonal areas as the target area. The area of each polygon may include at least one closed curve among the plurality of closed curves. According to another embodiment, the control unit 102 of the second terminal 150 may select one or more polygonal regions that include one or more closed curves having the largest area among the plurality of closed curves as the target region.

Referring again to FIG. 6, the step S133 of performing the image processing on the selected target area of each of the target images may be performed. 16 is an image in which the image processing according to the embodiment is performed. As shown in FIG. 16, the control unit 102 of the second terminal 150 may delete the selected target region from the target image, as shown in FIG.

17 is an image in which image processing according to another embodiment is performed. As another example, as shown in FIG. 17, the control unit 102 of the second terminal 150 may apply a blur effect to the target area. 18 is an image in which image processing according to another embodiment is performed. As another example, as shown in FIG. 18, the control unit 102 of the second terminal 150 may apply a mosaic effect to the target area. FIG. 19 is an image in which image processing according to another embodiment is performed. As another example, as shown in FIG. 19, the control unit 102 of the second terminal 150 may replace the target area with another image.

As another example, the control unit 102 of the second terminal 150 may adjust the hue, brightness, contrast, or saturation of the target area. In addition, the control unit 102 of the second terminal 150 can perform the combination of the image processing listed above. For example, the control unit 102 of the second terminal 150 may apply a blur effect to the target region, reduce saturation, and reduce contrast.

Referring again to FIG. 3, a step of displaying the plurality of images included in the stream (S140) may be performed. The output unit 105 of the second terminal 150 can display the plurality of images. For example, the display included in the output unit 105 of the second terminal 150 can continuously display 20 images per second. The plurality of images may include at least one image processed by the control unit 102 of the second terminal 150.

Thus, the user of the second terminal 150, which is in video communication with the user of the first terminal 100, can be prevented from being exposed to obscene images from the other terminal. In addition, a sexual shame or discomfort that the user of the second terminal 150 can feel by the image from the other can be prevented. In addition, a sound video call between the user of the first terminal 100 and the user of the second terminal 150 can be induced.

Next, a step of receiving a request for releasing the image processing (S150) may be performed. For example, the user of the second terminal 150 may wish to view the original of the received image, assuming that it may be exposed to an inappropriate image. The display included in the output unit 105 of the second terminal 150 may output a message asking whether to cancel the image processing. For example, the output unit 105 of the second terminal 150 may output a message such as "No face detected, please press the button if you want to view the original." The user of the second terminal 150 may press a button included in the input unit 101 of the second terminal 150 in response to the message. In other words, the input unit 101 of the second terminal 150 may receive a request for the cancellation of the image processing from the user of the second terminal 150.

Next, a step of displaying the received image after the request is received among the plurality of images included in the stream, without performing the image processing, may be performed (S160). The control unit 102 of the second terminal 150 may determine that the face is not detected in the received image after the request is received. The control unit 102 of the second terminal 150 may not perform image processing on the received image after the request is received, despite the above determination. The output unit 105 of the second terminal 150 may display at least one image received after the request is received without the image processing.

20 is a flowchart illustrating a process of performing an image display method according to another embodiment. Referring to FIG. 20, a step S600 of establishing a video call session with the first terminal 100 may be performed. The communication unit 104 of the second terminal 150 can establish a video call session with the first terminal 100. [

Next, a step S610 of receiving a stream including a plurality of images photographed by the first terminal 100 through the video call session may be performed. The communication unit 104 of the second terminal 150 may receive the stream from the first terminal 100 through the video call session. The stream may include a plurality of images.

Next, the step of displaying the plurality of images included in the stream (S620) may be performed. The output unit 105 of the second terminal 150 may display the received plurality of images.

Next, a step S630 of attempting to detect a face in each of at least one or more sample images selected from the plurality of images included in the stream may be performed. The control unit 102 of the second terminal 150 may select at least one or more sample images among the plurality of images. The control unit 102 of the second terminal 150 can detect whether or not a portion corresponding to the face is present in each of the selected sample images.

Next, a step S640 may be performed in which it is determined that the face is not detected in at least some sample images among the sample images. The control unit 102 of the second terminal 150 may determine that the face is not detected in at least one or more sample images. If the face is not detected in the sample image, the plurality of images may include an inappropriate image.

Next, ending the video call session (S650) may be performed. The communication unit 104 of the second terminal 150 may terminate the video call session with the first terminal 100. [ Thus, the user of the second terminal 150, which is in video communication with the user of the first terminal 100, can be prevented from being exposed to inappropriate images from the other terminal.

According to the embodiment described above, it is possible to prevent a user who makes a video call with a partner from being exposed to an obscene image from the other party. In addition, a sexual shame or uncomfortable feeling that a user who makes a video call with the other party can feel by the image from the other party can be prevented. In addition, a sound video call between users can be induced.

The embodiments described above may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer, and can include both volatile and nonvolatile media, removable and non-removable media.

The computer-readable medium may also include computer storage media or communication media. Computer storage media may include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and may include any information delivery media.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: a first video display device, a first terminal
101: Input unit
102:
103:
104:
105: Output section
150: second video display device, second terminal
200: Server
301: First video
302: Second image
303: Third video
311: fourth image
312: fifth image
313: the sixth image
314: Seventh image

Claims (12)

The method comprising: receiving a stream including a plurality of images taken by the first terminal from a first terminal;
Detecting a face in each of at least one or more sample images selected from the plurality of images included in the stream;
Detecting skin color in each of the sample images;
Performing image processing on an area where the skin color is detected corresponding to the face and the skin color detection result; And
Displaying the plurality of images included in the stream;
And displaying the image. The method according to claim 1,
If the face is not detected in the step of detecting the face and the skin color is detected in the step of detecting the skin color,
Wherein the image processing is performed in a region where the skin color is detected. The method according to claim 1,
The step of performing the image processing includes:
Selecting at least one target image from the plurality of images included in the stream;
Selecting at least some object regions within each of the object images; And
Performing the image processing on the selected target area of each of the target images;
Lt; / RTI >
Wherein the object area corresponds to the area where the skin color is detected. The method according to claim 1,
The step of detecting the face includes:
Selecting at least one sample stream corresponding to a specific recent time among the plurality of images included in the stream; And
Detecting the face in the selected at least one sample image;
And displaying the image. 5. The method of claim 4,
The step of detecting the face includes:
Selecting a new sample stream from the newly received images in the stream for each particular period; And
Detecting a face in the new sample stream each time the new sample stream is selected;
And displaying the image. The method according to claim 1,
Wherein the image processing corresponds to an operation of applying a blurring effect or a mosaic effect, or adjusting an hue, brightness, contrast, or saturation. Receiving a video stream including a plurality of consecutive video images from a second terminal in which a first terminal has established a video call session with the first terminal;
Determining whether a facial image exists in each of the plurality of images by the first terminal;
Detecting a skin color in an image in which the face does not exist;
Performing image processing on an image in which the face does not exist corresponding to the skin color detection result; And
Displaying the video stream on which the image processing is performed through the output unit of the first terminal;
And displaying the image. 8. The method of claim 7,
Wherein the image processing corresponds to an operation of applying a blurring effect or a mosaic effect, or adjusting an hue, brightness, contrast, or saturation. 9. A computer-readable recording medium on which a program for performing the method according to any one of claims 1 to 8 is recorded. A communication unit for receiving a stream including a plurality of images taken by the first terminal from a first terminal;
An output unit displaying the plurality of images included in the stream; And
A controller for detecting a face in each of at least one or more sample images selected from the plurality of images included in the stream;
Lt; / RTI >
Wherein the control unit performs image processing on at least one or more images among the plurality of images included in the stream when the face is not detected in at least some sample images among the sample images,
Wherein the output unit displays the at least one image subjected to the image processing. 11. The method of claim 10,
Wherein the control unit performs the image processing on a region where the skin color is detected when a skin color is detected from a sample image in which the face is not detected. 11. The method of claim 10,
Wherein the image processing corresponds to an operation of applying a blur effect or a mosaic effect or adjusting a hue, brightness, contrast or saturation.

Images