KR101392917B1 - Portable terminal and its image transfer and screen display method - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a method for controlling transmission of an image and a video call display screen according to the state of an image input through a camera during a video call, and a portable terminal having the function.

A portable terminal of the present invention includes: a communication unit for transmitting and receiving signals for video or voice communication with a terminal or a server; An output unit for outputting a call image or audio during a video call, and a display unit for prohibiting transmission or display of the video image according to a change in the amount of light input through the camera during a video call And a controller for controlling the motor.

The mobile terminal of the present invention can block image transmission or screen display when an image input through a camera during a video call is less than a reference image quality or a camera is hidden and can also automatically transmit a predetermined alternative image have.

Mobile terminal, video call, video transmission

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a portable terminal,

The present invention relates to a method for determining a state of an image input through a camera during a video call, controlling transmission of the image and a video call display screen according to the state, and a portable terminal having the function.

The portable terminal for video call according to the related art transmits all the images inputted through the camera in the same state. For example, even when the camera is blocked during a video call or the image quality is deteriorated to such an extent that the input image of the camera can not be distinguished because the calling place is very dark, the image input through the camera is directly transmitted to the other party or displayed on the screen.

It is an object of the present invention to provide a method for automatically shutting down a video transmission or a camera in accordance with the state of an image input through a camera during a video call, and a portable terminal.

It is another object of the present invention to provide a method for detecting whether or not a camera is blocked by a specific object so as to automatically shut off image transmission or camera driving, and a portable terminal thereof.

In addition, the present invention provides a method for automatically determining a quality of an image input through a camera during a video call or whether a camera is hidden, and automatically transmitting a predetermined alternative video according to the status, and a portable terminal There is a purpose.

The present invention relates to a communication terminal for transmitting / receiving a signal for video or voice communication with a terminal or a server, At least one camera for video call or photo taking; An output unit for outputting a call image or audio during a video call; And a controller for blocking transmission or display of the image in accordance with a change in the amount of light input through the camera during a video call.

According to another aspect of the present invention, there is provided a method of detecting a blindness of a camera in accordance with a change in an amount of light or luminance input through a camera during a video call; And a process of automatically transmitting an image input through the camera or screen display when the amount of change in the amount of light or luminance is equal to or greater than a specific set value.

The portable terminal of the present invention determines the state of an image according to a color tone, a light amount, a luminance value, or a variation amount of each of the color tones, the light amount, and the luminance amount inputted through a camera during a video call, It is effective.

In addition, the portable terminal of the present invention detects whether the camera is hidden or not, thereby enabling the image transmission, the image display, or the camera operation to be automatically interrupted.

In addition, the portable terminal of the present invention has an effect of automatically determining whether a video image input through a camera during a video call or a camera is blinded, and automatically transmitting a preset video or a user selected video.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail for the sake of clarity and conciseness. In describing the present invention with reference to the drawings, the same reference numerals are used to denote components performing the same function.

1 is a block diagram showing a configuration of a mobile terminal according to the present invention.

Referring to FIG. 1, a portable terminal 100 according to the present invention includes a communication unit 110 for performing communication with an external terminal or server in a wired or wireless manner, a camera unit 140 for video call or photographing, An output unit 130 for outputting audio or video data, an input unit 120 for receiving an input for controlling the terminal, a memory 150 for storing programs or input / output data to be executed in the terminal, And a controller 160 for controlling the respective components to perform the functions.

The communication unit 110 may communicate with an external terminal or a server providing a predetermined service through a wired / wireless communication network. The communication includes voice, image, and data communication, and can be connected to an external server under the control of the controller 160 to transmit and receive data.

The input unit 120 may include external audio input through a microphone (MIC), sensing information input through various sensors (not shown) included in the terminal, information input through a keypad or a touch pad, And outputs the converted signal to the controller 160.

In this case, the input unit 120 may include a sensor capable of detecting the obscuration of the camera. For example, the input unit 120 may include a touch sensor, an optical sensor, a light amount sensor, a capacitance sensor, a pressure sensor, The present invention is not limited to the above sensors, and it may be attached to a camera or attached to a receiver according to characteristics of the sensors.

The camera unit 140 performs photographing or video call under the control of the controller 160. The camera unit 140 may include at least one camera that can be commonly used for photographing or video call, and may include two or more cameras for photographing or video call, respectively.

The memory 150 may store a program necessary for controlling various operations of the terminal including the video call, or data input / output to / from the terminal. The input / output data includes a picture photographed through the camera unit 140, audio or video data input through the user's own terminal during video communication or transmitted through the other terminal.

The output unit 130 may include at least one speaker for outputting audio transmitted from another external terminal under control of the controller 160 or audio generated when various functions of the portable terminal are performed, Information, or a display for displaying an image generated when various functions are performed.

Here, the display may be a liquid crystal display (LCD) or an organic light emitting diode (OLDE), and the display may be a touch screen combining a touch input function with a video output function It is possible.

In particular, the output unit 130 displays at least one of an image input through the camera unit 140 and an image transmitted from a counterpart terminal during a video call. To this end, the output unit 130, under the control of the controller 160, divides the screen into a plurality of display areas, outputs the image transmitted from the counterpart terminal in a part of the display area, And outputs the image input through the camera unit 140 of the terminal. If there is no image transmitted from the other terminal, the camera input image of the user's portable terminal can be converted into a full screen and output.

The controller 160 detects whether the camera is hidden by a user during a video call and if the camera is hidden, the controller 160 determines that the user does not intend to make a video call and can stop the video output or the display for a predetermined time . For example, the camera may occur when a user mistakenly thinks that a user is habitually making a voice call and puts a receiver in his / her ear, or the user deliberately covers the camera lens using a finger or the like.

At this time, whether or not the camera is blocked can be determined by determining the state of the image input through the video communication camera of the input unit 120, or by determining whether there is an input of a specific sensor attached to the receiver. For example, the state of the image can be determined when the input image is less than the reference image quality or the color tone is high, so that the camera lens is hidden when the black image is output. Even if the camera is not covered, The image quality may be deteriorated even when it is incident. Alternatively, it may be determined whether or not the camera is blocked by the amount of light input through the camera, the luminance, or the amount of each change.

At this time, an objective evaluation method of the image quality is a PSNR (Peak Signal-to-Noise Ratio), and it can be analyzed by pixel scanning, frame rate, or illumination. Since recently improved image quality evaluation methods have been continuously developed, appropriate algorithms can be selectively used.

In addition, in the present invention, it is possible to determine whether or not the camera is covered by the color tone of the image. That is, when the terminal user hides the camera lens portion by hand during a video call, the user's image is changed to a specific color tone (for example, black). Accordingly, the controller determines that it desires not to transmit the user's own image, suspends the image transmission temporarily, and transmits the substitute image.

As described above, when the state of the image is determined by the color tone, there is no need of a separate sensor.

In the case where the input image of the camera can not be transmitted, the controller 160 may transmit the substitute image, and may rotate the at least one substitute image sequentially in order to prevent the other person from being bored. At this time, the substitute image may be automatically circulated by the controller 160, repeatedly hiding the camera by the user, repeatedly inputting a specific button, repeated input through a specific sensor provided in the camera or the receiver It can be changed or rotated to another alternative image every time.

Hereinafter, a process of transmitting an image input through the camera of the portable terminal and controlling the screen display of the image according to the present invention will be described in detail with reference to FIG.

FIG. 2 is a flowchart illustrating an exemplary process of controlling transmission and display of an image according to the state of an input image during a video call in the wireless terminal according to the present invention. As shown in FIG. 2, Connects the video call with the other party and enters the video call state (S101).

Here, the video call state refers to a state in which a call is performed using a voice and an image with a counterpart terminal. In the video call state, it is possible to output the video transmitted from the partner terminal and the video input from the user terminal on one screen, and only one video can be outputted on the screen by the user's selection, May be replaced and transmitted.

Typically, during the video call, a user hides a lens of a camera using a finger or the like in order to allow a user to habitually receive a receiver of the terminal for voice communication, to prevent the occurrence of an unexpected situation, Can occur.

At this time, the controller 160 can determine the state of the image or the shading of the camera based on the color tone, the amount of light, the luminance, or the variation amount of each of the color tone, the light amount, and the luminance amount inputted through the camera. The sensing information input through the sensor 121 attached to the periphery of the receiver 141 or the receiver 131 is analyzed (S102), and the state of the image input through the camera can be determined. In operation S103, it is determined whether or not transmission is possible according to the determined state of the image.

Here, the image in a state in which transmission is possible means an image having a state such that the other party to which the image is transmitted can identify the image. In the image in a state in which transmission is difficult, It is an image with a state that the other party can not identify. In this case, when the camera is hidden, the amount of light is very large or small so that the background and the user (the subject) can not be distinguished, or the color tone (especially black) If it is higher than the reference value, it can be judged that image transmission is difficult.

For example, whether or not the image can be transmitted can be determined to be a state in which image transmission is possible if the color tone of the image is equal to or less than a specific set value. If the color tone of the image is equal to or greater than a specific set value, can do. In addition, if the amount of light, color tone, and luminance value inputted through the camera is less than a specific set value, it can be determined that the image is difficult to transmit. Further, when the amount of change in the light amount, color tone, and luminance value is equal to or greater than a specific value, it can be determined that the transmission is difficult. At this time, the specific set value may be a variable value, and may be set to a default value when designing a terminal, or may be set by a user using a menu.

As described above, when the image input through the camera is a black screen (e.g., when the color tone is high or the amount of light, the brightness is low or low), the white screen (when the amount of light is high or high) The controller 160 determines that the image transmission is difficult and controls the communication unit 110 to transmit the image input through the camera to the control unit 160. [ (S104), and transmits a predetermined alternative image, moving image, guidance message or the like (S105).

For example, the guide message may be a message indicating that the communication is possible but the camera input of the terminal is abnormal and the screen is not transmitted, such as 'Wait for a moment'.

When the camera lens is hidden as described above, the controller 160 controls the camera unit to shut off the camera (S106), or controls the output unit so as not to output the camera input image to the display or stop the display itself (S107).

At this time, the controller 160 periodically determines the state of the image even if the camera lens is not blocked or the image inputted through the camera is not displayed on the screen, and if the image is restored to a predetermined reference image quality or more (S108, S103) , And automatically resumes transmission of the image input through the camera (S111 to S113).

At this time, the reference picture quality abnormality can be determined as an image quality in which a color tone of an image is lower than a reference value, an image quality having a light quantity within a certain range, and an image quality in which a variation amount of the light quantity is less than a predetermined value.

If a specific menu related to camera control or image transmission (eg, forced transmission, forced screen display, forced camera operation, etc.), button, or sensor input is detected and forced video transmission (S108). In operation S109, the controller 160 controls the output unit to display a message asking the user to resume transmission of the image (S109). The controller 160 can resume the transmission of the selected message (S110 to S113).

Hereinafter, a process of determining the state of the camera input image during the video call and a process of controlling the video transmission or the screen display according to the determined state of the video will be separately described.

4A is a diagram illustrating an example in which a user performs a telephone conversation as if the received call is a video call but a user as if the call was a voice call. Fig. 5 is a view showing an example in which the input of the camera is interrupted by intentionally covering the camera lens.

4B, the handset 131 of the portable terminal is in contact with the user's ear. In this case, the camera 141 may be hidden from the user's face, as shown in FIG. 4B, Input is interrupted.

Accordingly, the controller 160 detects the closure of the camera lens through the specific sensor 121 attached to the handset 131, analyzes the state of the image input through the camera, and if the image is blackened It is determined that the video call is difficult. Meanwhile, in addition to the above-described embodiments, when the camera is exposed to strong light, the input image may suddenly become whitened. In this case, the controller 160 can determine that the video call is difficult.

2, when the state of the image input through the camera is lower than or equal to the specific reference image quality due to the reason that the camera lens 141 is blocked as described above, the controller 160 controls the communication unit, A transmission of a predetermined alternative image, a moving image, a guidance message, and the like are transmitted. Or the camera unit may be controlled to block the driving of the camera, or the output unit may be controlled so as not to display the input image of the camera, or the driving of the display may be stopped.

Hereinafter, a method of controlling image transmission or screen display when camera input is interrupted or the video communication is difficult due to the surrounding environment of the user or the terminal, will be described with reference to the accompanying drawings.

FIG. 5A is a diagram illustrating an example of transmitting a predetermined specific substitute image when transmission of an image input through the camera or operation of the camera is interrupted.

As described above, when the image quality of an image blocked by a camera or inputted through a camera becomes lower than a reference image quality, the controller 160 can automatically block the transmission of images or the driving of the camera, After output, the user's choice can be blocked.

If no user is selected for a predetermined period of time after the guidance message 210 is output, the controller 160 may automatically block transmission of a camera input image, screen display of a camera input image, or driving of the camera.

When the driving of the camera is interrupted or the display or transmission of the image input through the camera is interrupted as described above, the predetermined alternative image can be transmitted, and the alternative image may be displayed in the input image display area 220 of the camera instead .

Or a file list 230 of alternative images stored in the memory 150 and uses the camera input image display area 220 as a preview display area to display a file list A preview image can be output.

5B is a view illustrating an example in which the size of the other party image is adjusted to be larger when the transmission of the image input through the camera or the driving of the camera is interrupted.

The controller 160 may reduce the size of the image input through the camera and may reduce the size of the image 240 transmitted from the counterpart terminal, Can be displayed in a larger size. That is, it displays a non-important substitute image or a black screen.

Alternatively, the replacement image may be changed in response to a user's selection after outputting the file list as described above, or may be sequentially changed when a specific button is input or the camera is hidden by the user. Alternatively, You can also set up context-sensitive options in the Preferences menu.

For example, the context-based options may include a case in which the camera is not driven, a case in which the camera is hidden, a case in which a substitute image is forcibly transmitted, and the like. The time for starting transmission of the substitute image, Make it possible to set different images. At this time, help for each item of the menu may be displayed.

An example of a method for setting the configuration menu is shown in Fig.

Hereinafter, a method for resuming transmission of a video or camera driving that has been interrupted as described above will be described.

FIG. 7A is a diagram illustrating an example of automatically resuming image transmission or camera operation when the camera shake is released or the image quality is improved.

For example, when the user who is talking on the voice communication method recognizes that the user is in a video communication and disconnects the handset 131 from the ear as shown in FIG. 4A, the user may be able to access the receiver 121 through the sensor 121 attached to the camera or the receiver The state of the inputted information is changed so that the controller 160 can detect that the cover of the camera 141 is released.

Accordingly, the controller 160 determines that the image transmission can be resumed and automatically starts transmission. At this time, a guide message 311 indicating that the image transmission is possible is output, and when the user selects to resume the transmission with reference to the guidance message 311, the transmission can be started. Of course, when the camera is restarted, the driving can be resumed in the same manner as described above.

More specifically, when the cover of the camera lens 141 is released, the controller 160 determines whether the image input through the camera is the reference image quality or more, and transmits the image when the state is equal to or higher than the reference image quality Resume automatically.

FIG. 7B is a view for explaining an example of forcibly resuming the image transmission or the camera driving by the user even if the state of the image inputted through the camera is not good.

For example, if the user deliberately does not select the camera, simply because the surrounding environment is too dark or too bright, and the state of the image input through the camera is not good, the user may want to transmit the image even if the image quality is poor.

At this time, the portable terminal of the present invention can resume video transmission or camera operation by inputting a specific menu (e.g., forced transmission) or button from the user.

Of course, in the case where the controller 160 forcibly resumes the image transmission by the user as described above, the controller 160 outputs a guidance message 312 confirming whether or not to perform image transmission, resumes transmission according to the user's selection, Resume camera operation for video transmission.

In the foregoing, preferred embodiments of the present invention have been described with reference to the accompanying drawings. Here, the terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings, but should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It should be understood that various equivalents and modifications may be present.

1 is a block diagram showing a configuration of a mobile terminal according to the present invention;

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a mobile terminal, and more particularly,

3 is a view illustrating an example in which a sensor and a camera are attached to a portable terminal according to the present invention.

4A is a diagram illustrating a video communication method using an exemplary mobile terminal according to the present invention.

4B is a view illustrating an example of covering a camera during a video call using the portable terminal according to the present invention.

FIG. 5A illustrates an example of transmitting a substitute image in a portable terminal according to the present invention. FIG.

FIG. 5B is a view illustrating an example in which a size of an image is adjusted and displayed in a portable terminal according to the present invention. FIG.

6 is a diagram illustrating an example of a configuration menu for setting a substitute image in the portable terminal according to the present invention.

FIG. 7A is a diagram illustrating an example of automatically restarting image transmission or camera operation in the portable terminal according to the present invention; FIG.

FIG. 7B is a view illustrating an example of forcibly resuming image transmission or camera driving in the portable terminal according to the present invention; FIG.

Description of the Related Art [0002]

110: communication unit 120: input unit

130: output unit 140: camera unit

150: memory 160: controller

Claims (16)

A communication unit for transmitting and receiving a signal for video or voice communication with a terminal or a server; At least one camera for video call or photo taking; An output unit for outputting a call image or audio during a video call; When the amount of change in the amount of light or brightness input through the camera during a video call is greater than or equal to a specific set value, transmission of an image input through the camera or screen display is automatically interrupted, and when the amount of change in the amount of light or luminance is less than And a controller for automatically restarting transmission of an image input through the camera or screen display. The mobile terminal of claim 1, And a sensor for determining whether the camera is covered by the camera or the handset. 2. The apparatus of claim 1, Wherein when the amount of light or brightness input through the camera during a video call is less than or equal to a specific set value, transmission of the video or screen display is blocked. 2. The apparatus of claim 1, Wherein the portable terminal compares the luminance of the current image input through the camera with the luminance of the previous image and intercepts transmission or display of the image when there is a sudden change. delete 2. The apparatus of claim 1, Wherein when the color tone of the image input through the camera during the video communication is equal to or greater than a specific set value, the transmission of the image or the screen display is blocked. The mobile terminal of claim 1, And at least one of a menu, a button, and a sensor capable of forcibly controlling an image of which transmission or screen output is blocked by a user. 2. The apparatus of claim 1, And sequentially transmits predetermined specific substitute images to the portable terminal whenever a transmission of an image input through the camera is interrupted. 9. The apparatus of claim 8, Wherein when the substitute image is transmitted, a list for selecting the substitute image and a preview screen of the substitute image selected from the list are displayed on the output unit. Determining whether the camera is blinded according to a change in the amount of light or luminance input through the camera during a video call; The control unit automatically compares the amount of change in the amount of light or brightness with a specific set value and automatically blocks transmission or screen display of the image input through the camera when the amount of change in the amount of light or brightness is greater than or equal to a specific set value, And automatically restarting transmission or display of an image input through the camera when the predetermined setting value is less than the predetermined setting value. delete 11. The method of claim 10, Further comprising the step of automatically shutting off transmission of the image or screen display when the amount of light or luminance incident through the camera during the video call is less than a specific set value. 11. The method of claim 10, And a step of automatically shutting off transmission or display of the image when the color tone of the image input through the camera during the video call is equal to or greater than a specific set value. . The method as claimed in claim 10, further comprising the step of, when the transmission of the image is interrupted, sequentially transmitting predetermined predetermined alternate images and transmitting the images. The method as claimed in claim 10, wherein, when the transmission of the image input through the camera is interrupted, And displaying the image transmitted from the counterpart terminal in a full screen mode. 11. The method of claim 10, wherein, when resuming transmission of the image input through the camera, And outputting a message asking whether to resume the image transmission and resuming the selection according to the selection of the user.

Images