KR102241840B1 - Apparatus and method for processing a image in device - Google Patents

Abstract

An image processing method using an electronic device includes displaying at least a part of a first image obtained through a first image sensor and at least a part of a second image obtained through a second image sensor through a display functionally connected to the electronic device. , Storing at least a part of the first image, storing at least a part of the second image in response to detection of a set signal, and storing at least a part of the first image and at least a part of the second image in at least one It may include an operation of storing an image so that it can be reproduced at the same time.

Description

Image capture device and method {APPARATUS AND METHOD FOR PROCESSING A IMAGE IN DEVICE}

Various embodiments of the present invention relate to a method and apparatus for capturing an image, and more particularly, to an apparatus and method for capturing a plurality of images using an electronic device, such as a smartphone, for an image captured through at least one image sensor. .

In general, a method of photographing in a camera device displays image data obtained from an image sensor of each camera on a display, and when a user gives a command to control photographing, the obtained image data is compressed and encoded and stored in a memory.

Currently, a camera device or a portable terminal equipped with a camera may include a high-pixel camera or a plurality of low-pixel cameras. In this case, the plurality of cameras are mounted on the front and rear sides of the device, respectively. In addition, the apparatus including the plurality of dual cameras may independently select front and rear cameras to take a front subject or a rear subject. That is, a device including a plurality of cameras may separately photograph different images, and may separately store and manage the photographed images.

In addition, a device including a plurality of cameras may drive a plurality of cameras to obtain a plurality of images, and synthesize and store the obtained images as a picture in picture (PIP). At this time, when a plurality of cameras are simultaneously driven, it was difficult to acquire a plurality of natural images. That is, in the photographing mode, the user performs a photographing operation by checking an image output from the front or rear camera, and thus, it is difficult to simultaneously check and photograph images output from a plurality of cameras. Therefore, by controlling the image sensors in a device having a plurality of image sensors while taking a picture, it is possible to take an image captured from each of the image sensors in a desired shape.

The present invention proposes an apparatus and method capable of enhancing user convenience when capturing a plurality of images using at least one image sensor. To this end, a camera device or an electronic device including a camera according to an embodiment of the present invention proposes an apparatus and method capable of capturing a plurality of images while controlling a plurality of image sensors. To this end, an apparatus and method capable of capturing an image while continuously controlling image sensors using a timer when a photographing is requested in a device having a plurality of image sensors are proposed. In addition, the present invention is a device and method capable of capturing an image while controlling the image sensors by performing a first photographing when a photographing is requested in an electronic device having a plurality of image sensors, and by driving a timer to shoot when a set action is detected. Suggest.

According to various embodiments, the method displays at least a part of a first image obtained through a first image sensor and at least a part of a second image obtained through a second image sensor through a display functionally connected to the electronic device, Store at least a portion of the first image, store at least a portion of the second image in response to detection of a set signal, and simultaneously convert at least a portion of the first image and at least a portion of the second image as at least one image It may include operations of saving so that it can be reproduced.

According to various embodiments, an image processing method using an electronic device includes at least a part of a first image obtained through a first image sensor and at least a part of a second image obtained through a second image sensor. Displaying through, storing at least a part of the first image, storing at least a part of the second image in response to detection of a set signal, and storing at least a part of the first image and the second It may include an operation of storing at least a part of the image so that it can be simultaneously reproduced as at least one image.

According to various embodiments, an image processing method using an electronic device includes at least a part of a first image obtained through a first image sensor and at least a part of a second image obtained through a second image sensor. From the operation of displaying at, storing at least a part of the first image, storing at least a part of the second image after a second predetermined time has elapsed when a set signal is detected, and storing at least a part of the first image If the set signal is not detected within a first predetermined time, storing at least a part of the second image and at least a part of the first image and at least a part of the second image may be simultaneously reproduced as at least one image. .

According to various embodiments, the electronic device includes a storage unit for storing image information and a control unit for processing image information, wherein the control unit includes at least a part of a first image and a second image obtained through the first image sensor. Displaying at least a portion of the second image obtained through a sensor through a display functionally connected to the electronic device, storing at least a portion of the first image, and storing at least a portion of the second image in response to detection of a set signal And, it may be set to store in the storage unit so that at least a part of the first image and at least a part of the second image can be simultaneously reproduced as at least one image.

Various embodiments of the present invention may capture images captured through at least one image sensor, and may store the captured images as a main screen and a sub screen. After the main screen is taken, a timer is driven to automatically take a sub screen, so that multiple images can be acquired by a single shooting command, and the user can acquire natural multiple images desired by the user when shooting by the timer. have. In addition, for example, when the sequential shooting mode is set, the main screen image can be taken at the time when the set action is detected after the main screen image is taken. This allows the sub screen image in the desired form after the main screen image is taken. It makes it possible to acquire.

1 is a diagram showing a configuration of an electronic device according to an embodiment of the present invention
2A to 2C are diagrams for explaining a procedure for displaying and photographing dual images in an electronic device having a dual image sensor according to an embodiment of the present invention;
3 is a flowchart showing an image capturing procedure according to the first embodiment of the present invention in a device having a dual image sensor
4 is a flowchart illustrating a procedure for performing a preview mode in a dual mode
5 is a flowchart showing a specific example of photographing a dual image according to the first embodiment of the present invention in a device having a dual image sensor
6 is a diagram for explaining a procedure for photographing a dual image by the same procedure as in FIG. 5
7 is a flowchart showing a procedure for sequentially photographing dual images according to a second embodiment of the present invention in a device having a dual image sensor
8 is a diagram showing the configuration of a portable terminal that performs a photographing operation according to an embodiment of the present invention

Hereinafter, with reference to the accompanying drawings, a detailed description of preferred embodiments of the present invention will be described in detail. It should be noted that the same components in the drawings represent the same reference numerals wherever possible.

A camera device or a portable terminal having a camera according to an embodiment of the present invention includes a plurality of cameras, and displays images acquired from the plurality of cameras in a preview mode in the form of a main screen and a sub screen on one screen, Alternatively, the main screen and the sub screen can be displayed in parallel on the display unit. When a photographing request is generated while displaying the preview image as described above, the device including the camera according to the embodiment of the present invention first captures the main screen image and the sub-screen image, and then captures the captured main screen and sub-screen images. Can be synthesized and processed, and then stored. In the following description, a camera device or a device including a camera will be described as an electronic device, and the electronic device may be a device including a smartphone, a tablet, a computer, a camera device, and the like. In addition, the electronic device may be a device including two or more image sensors.

Looking at a method of photographing a main screen image and a sub screen image in the first embodiment of the present invention, the electronic device captures a main screen image output from an image sensor that generates a main screen image when a photographing is requested, and generates a sub screen image. You can preview and display the sub-screen image output from the generated image sensor. At this time, when detecting a set action, the electronic device drives a timer, and when the timer expires, the subscreen image output from an image sensor that generates the subscreen image may be captured. Thereafter, the electronic device may synthesize the captured main screen image and sub screen image, and store the synthesized image after processing. Here, the main screen may be a first screen, and the second screen may be a sub screen.

Looking at a method of photographing a main screen image and a sub screen image in the second embodiment of the present invention, first, the electronic device captures the main screen image output from the image sensor that generates the main screen image, and generates the sub screen image. After previewing the image output from the image sensor, you can start the timer. When the timer expires, the electronic device may capture a sub-screen image output from a second image sensor that generates a sub-screen image. Thereafter, the electronic device may synthesize the main screen image and the sub screen image captured by the timer, and may process and store the synthesized image.

In the following description, it will be assumed that the electronic device includes dual image sensors. In addition, it is assumed that the dual image sensor of the device is mounted on the rear surface of the device and the front surface on which the display unit is mounted, respectively, and the front image sensor is described as being located in the bezel area of the display unit. Here, the image sensor mounted on the rear side of the device may be a first camera mounted on the first side of the device, and the front camera mounted on the front side of the device may be a second camera mounted on the second side of the device. .

In addition, the image sensor may include image sensors that are functionally connected using a wireless connection, such as near field communication (NFC) and wireless fidelity (WIFI), in addition to the image sensor connected to the terminal.

Accordingly, it is assumed that an electronic device according to an embodiment of the present invention includes a first image sensor mounted on the rear side of the device and second image sensors mounted on the front side of the device including the display unit. In the preview mode in the dual photographing mode, the electronic device may display an image photographed from the first image sensor as a main screen and images photographed from the second image sensor as a sub-screen of the main screen. In this case, in the preview mode, the electronic device may display the main screen image captured by the first image sensor on the entire display unit or on a portion of the display unit. In addition, in the electronic device, the sub-screen image photographed by the second image sensor may be displayed in the form of a PIP of the main screen, and may be displayed in parallel.

In addition, when a photographing is requested in the preview mode, the electronic device first captures the first image of the main screen photographed by the first image sensor, and then the second image of the sub-screen photographed by the second image sensor after a set time elapses. After capturing, it can be synthesized to be processed and stored. In this case, the sequential photographing method may be set by a timer as described above, and when a set action is detected after capturing the main screen image, a method of driving a timer and automatically capturing a sub screen when the timer is terminated may be used.

1 is a diagram showing a configuration of an electronic device according to an embodiment of the present invention.

Referring to FIG. 1, a first image sensor 120 (a rear image sensor) may be mounted on a rear surface of a device, and a second image sensor 125 (a front image sensor) may be mounted on a front bezel area of the device. Here, the first image sensor 120 may be a high-pixel image sensor having a larger number of pixels than the second image sensor 125. The first image sensor 120 and the second image sensor 125 may be independently driven under the control of the controller 100, and may be driven simultaneously. In addition, the first image sensor 120 and the second image sensor 125 may capture a still image and a video by the control unit 100.

The controller 100 controls the overall operation of the electronic device. In addition, the control unit 100 may drive the first image sensor 120 and/or the second image sensor 125, respectively, and may also simultaneously drive, and according to an embodiment of the present invention, images captured when simultaneously driven are displayed on the main screen and It is controlled to be displayed as a dual screen of the sub screen, and shooting and editing of the dual screen may be controlled. Here, the sub-screen may be a picture in picture (PIP). In addition, the control unit 100 includes a timer 113 and an action detection unit 115, and controls the first image sensor 120 and the second image sensor 125 using the timer 113 and/or the action detection unit 115 when shooting is requested in the preview mode. Screen and sub-screen images are captured, and the captured images can be synthesized, processed, and then saved.

The storage unit 110 may include a program memory for storing an operation program of an electronic device and a program according to an embodiment of the present invention, and a data memory for storing photographed images.

The display unit 130 may be mounted on the front side of the device like the second image sensor 125, and may display an image photographed by the first image sensor 120 and/or the second image sensor 125 under the control of the controller 100. The display unit 130 may be an LCD or OLED. The input unit 140 may generate inputs for controlling photographing of the electronic device. Here, the display unit 130 and the input unit 140 may be configured as an integrated touch screen.

The sensor 150 may include at least one sensor for detecting the state of the electronic device. Here, the sensor may be a geomagnetic sensor, an acceleration sensor, and/or a gyro sensor, and a signal for detecting a change according to a rotation and/or a state of the device may be generated and output to the controller 100.

The control unit 100 of the electronic device having the above configuration acquires a plurality of images by simultaneously driving the first image sensor 120 and the second image sensor 125 (dual mode), and displays the obtained images on the display unit 130 on the main screen and the sub screen. It can be marked as. In this case, the control unit 100 may change the main screen and the sub screen according to a user's selection. In addition, the electronic device may take a picture by driving the first image sensor 120 and the second image sensor 125 according to a set sequence when a dual shot is requested in the dual mode, and combines the captured main screen and sub screen images into one It can be created as a still image of.

First, a dual image may be photographed using the timer 113 and the action detection unit 115. In this case, the control unit 100 captures the main screen image output from the image sensor that generates the main screen image, and previews the sub screen image output from the image sensor that generates the sub screen image when the user requests shooting in the preview mode. Can be displayed. Thereafter, when the action detection unit 115 detects a set action, the control unit 100 drives the timer 113 and captures a sub-screen image output from an image sensor that generates a sub-screen image when the timer 113 is ended. Thereafter, the electronic device may synthesize the captured main screen image and sub screen image, and store the synthesized image after processing.

Secondly, the main screen and sub screen images may be photographed using the timer. In this case, the control unit 100 captures the main screen image output from the image sensor that generates the main screen image when a photographing is requested, and displays a preview of the image output from the image sensor that generates the sub-screen image, and then sets the timer 113. It can be driven. When the driving of the timer 113 is terminated, the controller 100 may capture a sub-screen image output from a second image sensor that generates a sub-screen image. Thereafter, the controller 100 may synthesize the captured main screen and sub screen images, and may process and store the synthesized image.

2A to 2C are diagrams for explaining a procedure for displaying and photographing a dual image in an electronic device including a dual image sensor according to an exemplary embodiment of the present invention. Here, FIGS. 2A to 2C illustrate an example in which the first image sensor 120 generates a main screen image and the second image sensor 125 generates a sub screen image. However, according to the user's selection, the second image sensor 125 may generate the main screen image and the first image sensor 120 may generate the sub screen image.

2A to 2C, in the preview mode in which the first image sensor 120 and the second image sensor 125 are driven, the first image sensor 120 photographs a first image (here, a landscape image) as indicated by reference numeral 210, and , The second image sensor 125 shows an example of capturing a second image such as 220 (here, a person image of a photographer). In this case, the controller 100 may display the first image 210 photographed by the first image sensor 120 as a main screen and display the second image 220 photographed by the second image sensor 125 as a sub-screen as shown in FIG. 2B. .

In this case, when photographing is requested in the preview mode, the electronic device according to an embodiment of the present invention may photograph a main screen image and a sub screen image as shown in FIG. 2C. Referring to FIG. 2C, when a photographing request is generated in the preview mode, the controller 100 captures a main screen image such as 255 generated by the first image sensor 120, such as 250, and sets a reference number 260 and a timer. It can be driven. At this time, the control unit 100 may display the main screen image captured on the display unit 130, display a preview image of the sub screen as shown by reference number 277, and display the timer operation as shown by reference number 275. Thereafter, when the timer expires, the controller 100 may capture the subscreen image 277. In this case, the captured image may be an image such as reference number 280.

Here, FIGS. 2B and 2C illustrate an example in which the second image 220 photographed by the second image sensor 125 is fixedly displayed on the lower left area of the display unit 130. However, the image displayed on the sub-screen may be variably set as a display area as needed. For example, the sub-screen display area may be displayed in an area adjacent to the image sensor (for example, the second image sensor 125). This is because in the case of the second image sensor 125, a user's image is captured in the dual mode, and in this case, a natural sub-screen image can be obtained by setting the sub-screen display area at a position close to the second image sensor 125. That is, if the sub-screen display area is set in a position close to the second image sensor 125, the photographing direction (angle of view) of the second image sensor 125 and the user's gaze can be matched as much as possible, thereby obtaining a natural user's image. have. In the case of displaying the sub-screen image as described above, the controller 100 captures the sub-screen image displayed in the preview mode when a photographing is requested, and moves the captured sub-screen image to the sub-screen area to be originally located at the time of synthesis. You can use the method of compositing to the image.

3 is a flowchart showing an image capturing procedure according to the first embodiment of the present invention in a device having a dual image sensor.

Referring to FIG. 3, when driving of the dual image sensor is requested through the input unit 140, the control unit 100 may perform the dual image sensor mode in operation 311. 4 is a flowchart illustrating a procedure of performing a preview mode in a dual mode.

Referring to FIG. 4, in the electronic device driving the first image sensor 120 and the second image sensor 125, the controller 100 may display an image sensor selection button and a dual button on a preview screen in operation 411. Here, the image sensor selection button The image sensor selection button is a button for selecting an image sensor for photographing an image displayed on the main screen, and the dual button is a button for selecting a dual-on mode and a dual-off mode. The image sensor selection button and the dual button may be buttons that operate in a toggle manner. When the first image sensor 120 is selected by the camera button, the controller 100 may detect it in operation 413 and select the first image sensor 120 as an image sensor that generates a main screen image. In addition, when the user touches the dual button while the first image sensor 120 is selected, the controller 100 may detect that the dual on mode is in operation 415 and drive the second image sensor 125 in operation 417. Then, the control unit 100 processes the image obtained from the first image sensor 120 in operation 417 and displays it as a main screen on the display unit 130, and processes and resizes the image obtained from the second image sensor 125, and then the display unit It can be displayed as a sub-screen at the set position of 130. Here, the control unit 100 knows the size and display position of an image displayed as a sub-screen, and the sub-screen can be displayed overlaid at a set position on the main screen. At this time, the display position and size of the sub-screen is set to a default display position and size, and the display position and/or display size may be changed by the user's selection. However, when the dual mode is turned off through the dual button while the first image sensor 120 is selected, the controller 100 detects it in operation 415 and processes the image acquired by the first image sensor 120 in operation 419. Thus, it can be displayed on the display unit 130. At this time, the driving of the second image sensor 125 may be turned off.

In addition, when the user selects the second image sensor 125 using the camera button, the control unit 100 may detect it in operation 413 and analyze the state of the dual button in operation 421. In this case, in the dual-off state, the controller 100 may detect this in operation 421, drive the second image sensor 125 in operation 425, and turn off the driving of the first image sensor 120. In addition, the image acquired by the second image sensor 125 may be processed and output to the display unit 130. At this time, the driving of the first image sensor 120 may be turned off.

However, when the user touches the dual button while the second image sensor 125 is selected as the image sensor that generates the main screen image as described above, the control unit 100 detects it in operation 421 and the first image sensor 120 in operation 423. It can be driven to perform dual mode. In this case, the control unit 100 processes the image acquired from the second image sensor 125 in operation 423 and displays it as a main screen on the display unit 130. After processing and resizing the image acquired from the first image sensor 120, the display unit It can be displayed as a sub-screen at the set position of 130.

In the following description, it is assumed that the first image generated by the first image sensor 120 is selected as the main screen image and the dual mode is turned on. Further, in the following description, it is assumed that in the photographing mode, a first image, which is a main screen image, is first captured, and a second image, which is a sub screen image, is captured. However, the electronic device may capture a sub-screen image first and then a main screen image when a photographing is requested.

Returning to FIG. 3, in the preview mode of the dual image sensor, the control unit 100 processes the first image output from the first image sensor 120 and displays it as a main screen on the display unit 130, and photographs it by the second image sensor 125. The second image can be processed and displayed on the sub-screen display area. In this case, the control unit 100 may include an image signal processor, and the image processing unit may include an image preprocessor, a postprocessor, an image scaler, and a CODEC. In this case, the first image output from the first image sensor 120 may be pre-processed and post-processed in the image processing unit, and may be scaled to the main screen size of the display unit 130 by an image scaler and then output. In addition, the second image output from the second image sensor 125 may be pre-processed and post-processed by an image processing unit, and may be scaled to a sub-screen size by an image scaler and then output.

When a photographing request occurs in the preview mode, the control unit 100 captures a first image output from the first image sensor 120 and a second image photographed from the second image sensor 125, and then combines the captured images to generate the It can be stored in the storage unit 110. In general, when taking a dual shot using a dual image sensor, it is common for a user to set the main screen image to suit the shooting composition, and in this case, the shooting composition of the sub screen image may not be in a desired state. Therefore, in this case, it is preferable to use the sequential photography method. can do. In the sequential photographing method, first, a main screen image is set as a desired photographing composition, and then, a sub-screen image is set as a desired photographing composition, and then photographed. In addition, in the sequential photographing mode, the control unit 100 may display on the display unit 130 that the currently displayed main screen and sub-screen are being photographed.

Therefore, if a photographing request occurs from the user while performing the preview mode, the control unit 100 detects it in operation 313, processes and captures the first image generated by the first image sensor 120 in operation 315, and captures the second image. The second image generated by the sensor 125 may be processed and displayed as a preview image on the display unit 130. In this case, the controller 100 may control the display unit 130 to display the first image of the main screen as a still image and display the second image of the sub screen as a preview image.

Thereafter, the control unit 100 may check whether a set action has occurred. Here, the action is an input for capturing the second image, and may be preset in a setting mode or an action set when a product is shipped. Here, the action may include a touch, a gesture, and the like. For example, the action can be rotated, shake or swing, motion detect, brightness change (e.g., illuminance sensor touch or mask), shutter switch on, screen hovering, screen touch Can be, etc. In addition, the action may be set as a user's action, and in this case, the action may be a blink or a smile. Therefore, in the sequential photographing mode of the dual image sensor, the user can register a specific action among the above actions in the setting mode. The action as described above may be detected through the sensor 150, the input unit 140, and/or the second image sensor 125, and the action detection unit 115 may detect the occurrence of the action detection signal.

Accordingly, when the setting action occurs after capturing the first image, the control unit 100 may detect it through the action detection unit 115 in operation 317 and drive a timer in operation 319. In addition, while the timer 113 is running, the control unit 100 may display the first image as a still image on the display unit 130 in operation 321 and display the second image generated by the second image sensor 125 as a preview image. Here, the timer may provide a time for the user to set the composition of the second image to be captured through the second image sensor 125 after capturing the main screen image. Thereafter, when the operation of the timer 113 is terminated, the controller 100 may detect this in operation 323 and capture a second image in operation 325. Here, the second image may be a sub-screen image. At this time, the main screen image is captured as a still image, and thus the main screen may be displayed as a still image. Thereafter, in operation 327, the control unit 100 synthesizes the captured second image into a sub-screen region of the first image, and encodes the synthesized image and stores it in the storage unit 110.

In addition, when a request to stop driving the image sensor is generated in the preview mode, the controller 100 may detect it in operation 331 and terminate the driving of the dual image sensor in operation 333.

5 is a flowchart illustrating a specific example of photographing a dual image according to the first embodiment of the present invention in a device having a dual image sensor. 5 illustrates an example in which the setting action is rotation (shaking) of the electronic device. 6 is a diagram for explaining a procedure for capturing a dual image by the same procedure as in FIG. 5.

5 and 6, the control unit 100 performs a preview mode in operations 511 to 515, and captures a first image 610 as shown in 630 of FIG. 6 and displays a second image 620 when a photographing is requested in the preview mode. It can be displayed as a preview image. The control unit 100 may control the electronic device in operations 511 to 515 in the same manner as operations 311 to 315 of FIG. 3. Here, the action for capturing the second image is when the device is set to rotate (or shake). Therefore, after capturing the first image, when the user rotates the electronic device as shown in 640 of FIG. 6, the sensor 150 generates a rotation detection signal of the device and applies it to the control unit 100, and the action detection unit of the control unit 100 115 may detect the rotation of the device in operation 517. Here, the rotation of the device may be rotate, shake and/or swing. At this time, the user needs time to set the composition of the desired second image after rotating the device. Accordingly, when the controller 100 detects the rotation of the device in operation 517, it may drive the timer 113 as shown in 650 of FIG. 6 in operation 519. Here, the timer 113 may perform a function of ensuring a time required for setting the second image to a desired composition after the user rotates the device as described above.

When the timer 113 is driven in operation 519, the control unit 100 displays the operation of the timer 113 on the display unit 130 as shown in 665 of FIG. 6, and when the operation of the timer 113 ends, the control unit 100 performs operations 521 to 527. Meanwhile, after capturing the second image as shown in 660 of FIG. 6, the first image and the second image may be combined and stored. The operation at this time may be performed in the same manner as operations 321 to 327 of FIG. 3.

7 is a flowchart illustrating a procedure for sequentially photographing dual images according to a second embodiment of the present invention in a device having a dual image sensor.

Referring to FIG. 7, when driving the dual image sensor, the controller 100 may perform a preview mode of the dual image sensor in operation 711. In the preview mode, when a user requests a photograph, the controller 100 may detect it in operation 713, capture the first image in operation 715, and display the second image as a preview image. Operations 711 to 715 may be performed in the same manner as operations 311 to 315 of FIG. 3.

After capturing the first image, the controller 100 drives the first timer in operation 717 to capture the second image, and may check whether an action set in operation 719 is detected. That is, the controller 100 may capture the second image when a set action is detected after driving the first timer and before the timer expires. The set action here may be a set interaction (eg, hovering or touch interaction of a set method, shaking, rotating, swing, motion detecting, etc.) of the device.

At this time, if the first timer is driven and the set action is not detected, the control unit 100 detects it in operation 731 and displays the captured first image as a still image on the main screen in operation 733, and the second image sensor The second image captured at 125 may be displayed as a preview image on the sub-screen display area. Here, the first timer driven in operation 717 may provide a time for the user to set the composition of the second image to be captured through the second image sensor 125 after capturing the main screen image. However, if the driving of the first timer is terminated in a state in which the set action is not detected after the first timer is driven, the control unit 100 may detect this in operation 731 and capture the second image in operation 727. Here, the second image may be a sub-screen image. At this time, the main screen image is captured as a still image, and thus the main screen may be displayed as a still image.

However, if a set action is detected before the end of the first timer, the controller 100 may detect it in operation 719 and drive the second timer in operation 721. In this case, the second timer driven in operation 721 is to provide time for the user to set the composition of the second image to be captured through the second image sensor 125 after performing a set action. Here, the period of the second timer may be set to be the same as the period of the first timer, or may be set to have a different period. And until the drive of the second timer is terminated, the control unit 100 processes the first image captured in operation 723 as a still image and displays it as a main screen, and the second image captured by the second image sensor 125 is displayed as a sub screen. It can be displayed as a preview image in the display area. In addition, when the driving of the second timer is terminated, the controller 100 may detect it in operation 725 and capture a second image in operation 727.

After capturing the second image in operation 727, the control unit 100 synthesizes the second image with the first image in operation 729, encodes the second image, and stores it in the storage unit 110, and returns to operation 711 to enter the preview mode. You can do it. In addition, when a request to terminate the driving of the image sensor occurs in the preview mode, the control unit 100 detects it in operation 741 and turns off the dual mode in operation 743.

In the description of FIGS. 3 to 7, the first image generated by the first image sensor 120 is assumed to be the main screen image, and the second image generated by the second image sensor 125 is assumed to be the sub screen image. However, the second image generated by the second image sensor 125 may be selected as the main screen image, and the first image generated by the first image sensor 120 may be selected as the sub screen image. In addition, in the description of FIGS. 3-7 above, it is assumed that the main screen image is captured first when sequential shooting, and the sub screen image is captured when the timer expires, but the subscreen image is captured first and the main screen image is captured when the timer ends. You can also perform the shooting mode.

8 is a diagram showing the configuration of a portable terminal that performs a photographing operation according to an embodiment of the present invention. Here, the portable terminal may be various digital devices such as a mobile phone including a smart phone, an MP3 terminal, a tablet, and a computer.

Referring to FIG. 8, the communication unit 170 may perform a wireless communication function such as a base station or an Internet server. Here, the communication unit 170 may include a transmission unit that amplifies a frequency of a transmission signal and amplifies power, and a reception unit that amplifies a received signal with low noise and a frequency down converter. In addition, the communication unit 170 may include a modulator and a demodulator. Here, the modulator modulates the transmission signal and transmits it to the transmitter, and the demodulator may demodulate a signal received through the receiver. In this case, the modem may be LTE, WCDMA, GSM, or the like, and may also be WIFI, WIBRO, or the like, NFC, Bluetooth, or the like. In an embodiment of the present invention, it is assumed that the communication unit 170 includes LTE, WIFI, and bluetooh communication units.

The first image sensor 120 is a rear image sensor mounted on the rear side of the terminal device and can capture an image of high resolution. The second image sensor 125 is a front image sensor mounted on the front surface of the terminal device, and may capture a relatively low-resolution image compared to the first image sensor 120.

The control unit 100 controls the overall operation of the mobile terminal, an application processor (AP) 810 that controls processing of various applications of the mobile terminal, and a communication processor (CP) 820 that controls communication functions of the mobile terminal. It can be provided. In addition, the application processing unit 810 includes a timer 113 and an action detection unit 115, and displays the first image and the second image as a main screen image and a sub screen image in a preview mode in a dual mode, and the timer 113 and an action in a capture mode. After capturing the first image and the second image according to an embodiment of the present invention using the sensing unit 115, an operation of synthesizing and storing the first image and the second image may be controlled.

The storage unit 110 may include a program memory for storing an operation program of a terminal and a program according to an embodiment of the present invention, and a data memory for storing data generated during execution of a program and tables for operation of the terminal.

The display unit 130 may display information on an application being executed under the control of the control unit 100. In addition, the display unit 130 may display images of a main screen and a sub screen under the control of the controller 100 in a dual mode. The display unit 130 may be an LCD or OLED. The input unit 140 may be implemented in a capacitive method or a resistive method, and may output location information of a user's touch (hereinafter, assumed to be a finger touch) to the control unit 100. In addition, the input unit 140 may further include an EMR sensor pad, and may sense a pen touch input and output it to the control unit 100. Here, the display unit 130 and the input unit 140 may be integrally configured.

The sensor 150 may include sensors for detecting movement of the portable terminal. Here, the sensors may be composed of an acceleration sensor, a geomagnetic sensor, a gyro sensor, and/or a position sensor. The audio processing unit 180 may process a voice signal generated in a communication mode under the control of the controller 100.

The control unit 100 of the portable terminal having the above configuration may include a camera button for selecting an image sensor for displaying a main screen image on the display unit 130 and a dual button for turning on/off a dual mode setting. In addition, the controller 100 performs a dual mode of driving the first image sensor 120 and the second image sensor 125 when the user's dual button is turned on, and in the dual mode, the controller 100 performs images output from the image sensors 120 and 125. The display unit 130 may display dual images of a main screen and a sub screen. In addition, the controller 100 may select an image sensor that outputs a main screen image by selecting an image sensor selection button in the dual mode. In the embodiment of the present invention, when the portable terminal is driven in the dual mode, it is assumed that the first image sensor 120 is selected as the main screen image sensor and the second image sensor 125 is selected as the sub screen image sensor. .

In the portable terminal, in the preview mode when driving the dual image sensor, the control unit 100 displays the first image generated by the first image sensor 120 on the display unit 130 as a main screen, and displays the second image generated by the second image sensor 125. It can be displayed as a sub-screen. In this case, the application processing unit 810 of the control unit 100 includes a timer 113 and an action detection unit 115, and may take a first image and a second image when a photographing is requested by using them.

In a method of capturing a main screen image and a sub screen image according to the first exemplary embodiment of the present invention, the controller 100 may capture a main screen image and a sub screen image while performing the same procedure as in FIG. 3 when a photographing is requested. That is, when shooting is requested in the preview mode, the control unit 100 first captures the main screen image output from the image sensor that generates the main screen image, and displays a preview of the sub screen image output from the image sensor that generates the sub screen image. can do. Thereafter, the control unit 100 may check whether an action for capturing a second image is generated through the action detection unit 115. Here, the action may be set in advance, and may be generated through the sensor 150, the second image sensor 125, and/or the input unit 140 which is a touch panel. If the set action is an action that detects movement (rotation, swing, or shaking) of the device, the action detection unit 115 may detect whether an action has occurred through the sensor 150. In addition, if the set action is a user's facial expression (eg, a wink, a smile, etc.), the action detection unit 115 may detect whether an action has occurred through the second image sensor 125. In addition, if the set action is a gesture or a touch interaction on the display unit 130, the action detection unit 115 may detect whether an action has occurred through the input unit 140. At this time, when the action set by the action detection unit 115 is detected, the control unit 100 drives the timer 113, and when the operation of the timer ends, the control unit 100 captures a sub-screen image output from an image sensor that generates a sub-screen image. I can. Thereafter, the control unit 100 may synthesize the captured main screen image and sub screen image, and store the synthesized image after processing.

In addition, looking at a method of capturing a main screen image and a sub screen image in the second embodiment of the present invention, the control unit 100 may take a main screen image and a sub screen image while performing the procedure shown in FIG. 7. Here, the timer 113 may include a first timer and a second timer. First, when shooting is requested in the preview mode, the control unit 100 captures the main screen image output from the image sensor generating the main screen image, and displays a preview image output from the image sensor generating the sub screen image, The first timer can be driven. At this time, if the set action is not detected until the end of the first timer, the control unit 100 captures the sub-screen image output from the second image sensor, synthesizes the captured main screen and sub-screen images, and the synthesized Images can be processed and saved. However, if the action set by the action detection unit 115 is detected before the first timer ends, the control unit 100 drives the second timer, and when the driving of the second timer ends, the control unit 100 generates a sub-screen image. The subscreen image output from the sensor can be captured. Thereafter, the control unit 100 may synthesize the captured main screen image and sub screen image, and store the synthesized image after processing. Here, the periods of the first timer and the second timer may have the same period or may have different periods. In addition, the set action may be the same action as the action in the first embodiment.

The embodiments of the present invention disclosed in the present specification and drawings are only provided for specific examples to easily explain the technical content of the present invention and to aid understanding of the present invention, and are not intended to limit the scope of the present invention. In addition to the embodiments disclosed herein, it is obvious to those of ordinary skill in the art that other modified examples based on the technical idea of the present invention can be implemented.

Claims (11)

In the electronic device,
Image sensor;
display; And
Including a processor,
The processor,
Display a preview image acquired in real time from the image sensor on the display,
In response to a first user input on the display input at a first point of view, a first still image is generated using the image sensor,
While maintaining the display of the preview image, displaying the first still image in the first area of the display,
In response to a second user input on the display input at a second point after the first point of view, a second still image is generated using the image sensor,
An electronic device configured to display the first still image on a first area of the display and the second still image on a second area of the display.
The method of claim 1,
The processor,
An electronic device configured to change at least one of a location and a size of the first area and the second area in response to a third user input on the display.
The method of claim 1,
The first region is separated from the second region and is located adjacent to the second region.
The method of claim 1,
The processor,
Combining the first still image and the second still image,
An electronic device configured to store the combined image in a memory.
The method of claim 1,
The processor,
An electronic device configured to display the preview image in the first area and in a third area different from the second area.
In the image processing method using an electronic device,
Displaying a preview image acquired in real time from an image sensor on a display;
Generating a first still image using the image sensor in response to a first user input on the display input at a first point of view;
Displaying the first still image in the first area of the display while maintaining the display of the preview image;
Generating a second still image using the image sensor in response to a second user input on the display input at a second point after the first point of view; And,
And displaying the first still image on a first area of the display and the second still image on a second area of the display.
The method of claim 6,
And changing at least one of a location and a size of the first area and the second area in response to a third user input on the display.
The method of claim 6,
The first region is separated from the second region and is a position adjacent to the second region.
The method of claim 6,
Combining the first still image and the second still image; And
And storing the combined image.
The method of claim 6,
The displaying of the preview image includes displaying the preview image in the first area and in a third area different from the second area.
A computer-readable recording medium storing a program for executing the method of any one of claims 6 to 10 on a computer.

Images