KR20210029905A - Method and computer program for remove photo background and taking composite photograph - Google Patents

Abstract

Disclosed is background-removed photographing and method for taking a composite photograph using the same. According to the present invention, the photographing method comprises the steps of: generating a depth map by analyzing a plurality of images photographed by a plurality of cameras when photographing is requested from a user; determining one or more partial images making up an image by using the depth map; displaying a user interface for separating and selecting the one or more partial images as a live-view image over a main image displayed on a screen of a digital photographing device; receiving an input of a user through the user interface; and generating a photograph, from which a partial image selected and excluded from the main image is deleted, in accordance with the input of the user through the user interface as a new main image, and storing the photograph in a storage unit when photograph storage is requested from the user. According to the present invention, a user can easily take a photograph of only an object that the user desires without a background image in real-time.

Description

[Method and computer program for remove photo background and taking composite photograph}

The present invention relates to a method of taking a picture in a digital photographing apparatus, and more specifically, a separate editing process for a photo (a photo from which the background has been deleted) or a photo synthesized with another image by using images acquired through a digital camera. It is about a video recording method and a computer program that allows you to shoot in real time without the need.

The dual camera mounted on a mobile phone is a camera module for smartphones with two camera lenses. Shooting with two lenses improves the quality of photos, acquires photos with different angles of view, is used for security or object recognition by AI (Artificial Intelligence) with accurate facial recognition (recognizing depth as well as length), or 3D By creating an image, the effects of virtual reality and augmented reality can be realized.

On the other hand, nouki (ぬき) in Japanese, which means'to subtract', refers to a printing method of subtracting color from text and coloring the background, removing unnecessary parts and printing only necessary subjects. In graphic design work, we use the expression'picking up luck', and cutting out only the necessary subject from an image with a background using Photoshop or the like, that is, removing the background image while leaving only the necessary objects is called ‘lucking’. The term'nuki photo' here refers to a picture without a background image other than a specific object, and this nutty image is mainly used for photo composition work or graphic design work.

However, editing photographic images using graphic editing software such as Photoshop requires professional skills as well as a lot of time and effort even if you are a graphic expert. The general user who shoots does not have expensive graphic editing software, and it is difficult to perform graphic editing using it.

In addition, since the recorded video cannot be used as a design source for emoticons, icons, and design synthesis without separate processing and editing, it is possible to use various images taken in applications such as notepad, word, calendar, widget, and SNS. Not only is it not available, and photo compositing work on mobile phones is not generalized.

In particular, the camera's properties and stereotypes that'reproduce' existing objects and phenomena as they are, and the current technology developed based on this, create another image by processing an image or adding an image of the past and the present. Photography is acting as an impediment to the development of a new area. This does not go beyond the limit of'records' and does not combine imagination with photographs that have been developed mainly for'real-life pictures', and at present, it is limited to a technology that only changes the color of photographic images.

The present invention is to solve the problems of the prior art as described above, so that after taking a picture in a digital photographing device, a user can easily take a picture of an object desired by a user in real time (a picture without a background image) without a separate editing process. It is to provide a video recording method.

In addition, the present invention is to provide an image capturing method in which a digital photographing apparatus can easily grasp the outline of each object without a separate editing process after taking a picture, and then select and include the desired objects in real time. will be.

In addition, the present invention is to provide a method of erasing a partial image that the user does not want, such as a background, from an image photographed by a camera, and synthesizing other photographed images, graphic images, and color backgrounds to the erased background portion to take a real-time image will be.

In addition, the present invention is to provide an image capturing method capable of erasing a partial image that a user does not want, such as a background, from an image captured by a rear camera and combining it with an image captured by the front camera.

In addition, the present invention is to provide an image capturing method capable of capturing in real time a picture synthesized with an image that is previously stored or downloaded and read out in real time.

In order to achieve the above object, according to an embodiment of the present invention, a method for taking a picture of a nouki and taking a composite picture using the same, when a user requests to take a picture, analyzes a plurality of images taken through a plurality of cameras. Generating a depth map; Determining one or more partial images constituting an image using the depth map; Displaying, as a live view image, a user interface for dividing and selecting the one or more partial images on the main image displayed on the screen of the digital photographing apparatus; Receiving a user's input through the user interface; And when the user requests to store the photo, generating a photo from which the partial image selected and excluded from the main image has been deleted as a new main image and storing it in a storage unit according to the user's input through the user interface. have.

The storing may include storing, as a main image, a photo obtained by combining a photo from which the selected-excluded partial image is deleted and another image selected by the user.

The storing may include storing, as a main image, a photo obtained by combining a photo from which the selected-excluded partial image is deleted and an image captured by a front camera of the terminal.

The storing step may include storing an original main image including even a partial image not selected by the user as a layer of the new main image.

In the step of generating the depth map, when the plurality of cameras have different angles of view, an image photographed by a wide-angle camera so that the horizontal or vertical size of the same object included in the plurality of images photographed through the plurality of cameras is the same. And determining a separation distance between the same objects in the plurality of images after cropping.

The photographing method includes the steps of reading out and displaying a Noki image from the storage unit from which the partial image selected and excluded by the user has been deleted; The method may further include obtaining a photo obtained by synthesizing an image captured by one of the plurality of cameras into a background image of the displayed Nuki image.

The photographing method includes the steps of, when a photographing is requested from the user, downloading and displaying a noki image matched with location information of a location where photographing is requested from a server; The method may further include obtaining a photo obtained by synthesizing an image captured by one of the plurality of cameras into a background image of the displayed Nuki image.

The photographing method automatically recognizes a face of a subject from a plurality of partial images selected through the user interface, and automatically adjusts the size of the selected partial images, the tone, color, and contrast of the images so that the size of the recognized face and image information are similar to each other. The step of adjusting to; may further include.

According to the present invention, after taking a picture in a digital photographing apparatus, without a separate editing process, an object desired by a user can be photographed in real time.

In addition, according to the present invention, after taking a picture in the digital photographing apparatus, the outline of each object can be grasped without a separate editing process, and a picture including only the object desired by the user can be taken in real time.

In addition, according to the present invention, a partial image that the user does not want, such as a background, may be erased from an image photographed by a rear camera, and other photographed images, graphic images, and colors may be combined with the erased background portion to take a picture in real time.

In addition, according to the present invention, a partial image that the user does not want, such as a background, can be erased from an image captured by the rear camera, and a photo synthesized with an image captured by the front camera can be captured in real time.

In addition, according to the present invention, it is possible to easily take a picture synthesized with an image that is pre-stored or downloaded and read in real time.

1 is a block diagram illustrating the structure of a digital photographing apparatus according to an embodiment of the present invention.
2 is a view for explaining a camera configuration of a digital photographing apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a method of synthesizing an image in real time in a digital photographing apparatus according to an embodiment of the present invention.
4 is a view for explaining the operation of the digital photographing apparatus according to an embodiment of the present invention.
5 is a view for explaining an image photographed by two cameras having the same angle of view according to an embodiment of the present invention.
6 is a view for explaining an image captured by two cameras having different angles of view according to an embodiment of the present invention.
7 is a diagram illustrating a user interface and a screen for selecting or excluding a partial image desired by a user according to an embodiment of the present invention.
8 illustrates a screen for pre-editing a captured image according to an embodiment of the present invention.
9 is a view for explaining an operation of taking a picture with a plurality of cameras including a front camera according to an embodiment of the present invention.
10 illustrates a screen for synthesizing an image captured by a front camera and an image captured by a rear camera according to an embodiment of the present invention.
FIG. 11 is a diagram illustrating a screen for synthesizing photographing after reading and displaying a nouki image without a background according to an embodiment of the present invention.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

Terms used in the present invention have selected general terms that are currently widely used as possible while taking functions of the present invention into consideration, but this may vary according to the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "...means", "...units", and "modules" described in the specification mean a unit that processes at least one function or operation, which is implemented in hardware or software, or is implemented in hardware and software. It can be implemented by a combination of.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

1 is a block diagram illustrating the structure of a digital photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the digital photographing apparatus 100 may include a camera 110, a processor 120, a display 130, and a memory 140.

The digital photographing apparatus 100 may include a smartphone, a tablet PC, a camera, and a wearable device. However, the digital photographing apparatus 100 is not limited thereto, and may include any device including a lens and an image sensor capable of photographing a subject and generating an image.

The camera 110 may include a plurality of cameras. Here, the camera refers to a component that includes at least one lens and an image sensor, and acquires an image signal through the image sensor. In FIG. 1, it is shown that three cameras are provided, but the present invention is not limited thereto, and the image capturing method of the present invention may be implemented in a digital photographing apparatus provided with two or more cameras.

In one embodiment, the camera 110 includes a first camera 111 and a second camera 112 as a rear camera, and in another embodiment may additionally include a third camera 113 as a front camera.

In an embodiment, the first camera 111 and the second camera 112 may be cameras having lenses having different angles of view. For example, the first camera 111 may be a wide-angle camera having a wide-angle lens to obtain a wide-angle image including a main subject, and the second camera may be a telephoto camera having a telephoto lens. In another embodiment, the first camera 111 and the second camera 112, which are rear cameras, may be cameras having lenses having the same angle of view. Also, the first camera 111 may be a camera that photographs a color image, and the second camera 112 may be a camera that photographs a black and white photo.

The processor 120 may control the camera 110, the display 130, and the memory 140 connected to the processor 120, and may perform various data processing and operations. The processor 120 may be implemented as, for example, a system on chip (SoC). The processor 120 may further include a graphic processing unit (GPU) and/or an image signal processor.

The processor 120 acquires depth information of objects included in the image based on a plurality of images captured by the first camera 111 and the second camera 112, and uses the depth map. map).

In addition, the processor 120 may obtain information on a global motion and a local motion of an image. Accordingly, the processor 120 may obtain the depth map after correcting the motion of the main subject in the image or blur due to hand shake.

The processor 120 determines one of the images captured by the plurality of cameras 110 as the main image, or an image generated by using the plurality of images as the main image, and based on the depth map acquired above. One or more partial images (each object, background, etc.) constituting the main image may be distinguished and identified.

The processor 120 may control the display 130 to display the main image as a live view image. In addition, the processor 120 may display a user interface that divides partial images constituting the main image and allows a user to select or delete a desired partial image by overlapping the user interface on the main image.

The memory 140 may include an internal memory or an external memory. The memory 140 is a storage unit capable of storing the main image under the control of the processor 120, and stores the pre-edited image through the user interface as the main image when a photographing button is clicked. In addition, the original main image may be stored as a layer in addition to the edited main image under the control of the processor 120.

The processor 120 may execute a dedicated application stored in the memory 140 to perform an image capturing method according to various embodiments of the present invention, and an image capturing method performed by a dedicated application with reference to FIGS. 2 to 11 below. Let's explain.

2 is a view for explaining a camera configuration of a digital photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the digital photographing apparatus may include two rear cameras 210 and 220 on the rear side and one front camera 230 on the front side, according to the control of a dedicated application executed by the processor. At least two of these cameras 210, 220, and 230 are used to obtain a desired photo by using images captured by at least two cameras.

The digital photographing apparatus determines the depth by using the difference in coordinates of the same object in the images captured by the plurality of rear cameras 210 and 220, divides each object and the background according to the depth, and then converts the captured image to the main image. It is possible to display a user interface for receiving an edit input from a user before storing it in the memory, and control to obtain a photo modified according to the user's intention.

At this time, the background or the image of which some objects have been deleted, or the combined image of a plurality of videos are saved after showing to the user in real time.Therefore, without a separate post-editing process, you can directly check the noki photo or composite photo with the background removed in real time. You can shoot.

In an embodiment, the digital photographing apparatus may delete a background or some objects from an image photographed through the rear cameras 210 and 220 according to a user's selection, or may replace a pre-stored image. Accordingly, the user may take a picture of a nouki without a background, or take a picture from which only the object selected by the user is left and the remaining objects are removed. Alternatively, a photo synthesized in real time with another background image previously stored in the object selected by the user may be obtained.

In one embodiment, the digital photographing apparatus may synthesize, in real time, a pre-edited image to include only the object selected by the user from the image photographed through the rear cameras 210 and 220 with the image photographed through the front camera 230. have.

3 is a flowchart illustrating a method of capturing an image in real time in a digital photographing apparatus according to an embodiment of the present invention.

In step S31, a photo-taking request is received from a user of a digital photographing apparatus having a plurality of cameras. The photo capture request corresponds to camera preparation, and when a user selects a camera icon or requests to prepare a photo through a dedicated application, an image is captured from each of the plurality of rear cameras. An image generated based on one or a plurality of images captured through a plurality of rear cameras is determined as the main image, and the following steps for image synthesis according to a user's input will be performed.

In step S32, a depth map is generated by analyzing a plurality of images captured by a plurality of cameras. The distance information of the subject may be detected using a stereo image, histogram, pixel value, and other physical methods, and a depth map may be generated using this. The depth map image may be acquired through passive depth sensing and stereo matching acquired using two or more cameras.

For example, in stereo matching using stereo images obtained from two cameras, a wide-angle image including a main subject is obtained from a first camera and a telephoto image obtained by zooming a main subject from a second camera is obtained. The distance information of the main subject is detected based on these wide-angle and telephoto images, and a depth map is generated based on the distance information.

4 is a view for explaining the operation of the digital photographing apparatus according to an embodiment of the present invention.

Referring to FIG. 4, since the plurality of rear cameras 410 and 420 are spaced apart by a predetermined interval, the positions (distances) of the objects, which are the subject 430, can be more accurately identified using images captured by these cameras.

In step S33, one or more partial images constituting the entire image are determined using the depth map. The partial image may include a background, an object (subject), and the like. Classification of each object or background can be basically performed using a depth map, but accuracy is improved by using brightness values, color values, contrast values, negative images, and black and white image information as auxiliary indicators. You can increase it. In other words, when subjects are close to each other by a certain distance or more, and it is not clear to distinguish the outlines using only the depth map, the outlines can be extracted by combining and using the color information and brightness information of each subject, and noise of each subject generated at this time is removed. By (loss correction), it is possible to more accurately distinguish the outer shape of the image for each object.

5 is a view for explaining an image photographed by two cameras having the same angle of view according to an embodiment of the present invention.

By comparing the distance difference between the left and right images, the same portion may be determined as a background, a small pixel difference may be determined as a distant object, and a large pixel difference may be determined as a nearby object. At this time, those with a similar pixel difference within a threshold value may be determined as the same object.

6 is a view for explaining an image captured by two cameras having different angles of view according to an embodiment of the present invention.

When the angles of view of the cameras are different, an image captured at a wide angle may be cropped to fit the size of another image before generating a depth map by comparing each image. Referring to FIG. 6, since the image in (a) was photographed through a wide-angle camera, (b) the image of the cropped part (a) after cropping a part 600 of the image (a) so that the size of the object in the image is the same ( 600) and (b) images are compared and analyzed to generate a depth map.

In one embodiment, the dedicated application for performing the image synthesis method of the present invention collects information on the basic arrangement direction of camera lenses and information on whether to take a horizontal grip or a vertical grip when taking a picture. If the camera lens is arranged left and right on the terminal, the default value is set to the left and right arrangement, and depending on whether the photo is taken vertically or horizontally by the gravity sensor, whether to count the pixels spaced between objects horizontally or vertically. You can decide.

In this way, it is possible to obtain the outline of each object by dividing it according to the depth, and to obtain a background part by dividing it. As described above, each object or background classified in the image is partial images of the image, and may be selected or excluded by the user.

In step S34 of FIG. 3, a user interface for classifying and selecting partial images is displayed on the main image displayed on the screen of the digital photographing apparatus. In this case, the main image and the user interface overlaid thereon may be displayed as a live view image.

The user interface includes the outline of each partial image, and when the user selects or unselects the desired partial image, the entire background or object corresponding to the point touched by the user is selected or excluded based on the outline. It will show the appearance of being erased.

In step S35 of FIG. 3, an edit input is received from a user through the user interface. The user may select or exclude one or more of the background and objects displayed on the main image, and dynamically change the image on the screen by reflecting the selection or deletion of the partial image in real time according to the user's input. That is, in the present invention, the main image, the depth map, the outline image, the noki image, the composite image of the noki image, and the background image may be displayed as a live view image.

Meanwhile, a face of a subject may be recognized from a plurality of partial images selected through the user interface, and the sizes of the selected partial images may be automatically adjusted and displayed so that the recognized face sizes are similar to each other. For example, if the user selects a human figure and a Lego doll, but the human face size and the face size of the Lego doll are different, the ratio of the two subjects can be automatically adjusted by enlarging the Lego doll or reducing the figure of the person. .

7 is a diagram illustrating a user interface and a screen for selecting or excluding a partial image desired by a user according to an embodiment of the present invention.

Referring to FIG. 7, the outline of each object is displayed so that the user can distinguish selectable partial images 710, 720, 730, and 740, and the desired object or background can be selected or deleted according to the user's touch input. The user interface that can be used is displayed above the main image. In this example, a partial image of the user interface is displayed as an outline of each object, but each object may be displayed with a difference in color, contrast, blur, or texture.

When the dedicated application receives a first user input (e.g., 710 touch) on a predetermined object, the object is selected and included in the final main image, and when a second user input (e.g., 720 touch) is received on a predetermined object, the corresponding The object is included in the final main image, and the remaining objects and background are deleted. At this time, the user's selection and selection exclusion are reflected in real time on the image of the screen so that the user can intuitively know the pre-editing situation.

In addition, the user interface may include a list area 750 through which an image to be combined is selected instead of a position of a partial image that is excluded from selection by the user. For example, the user may erase the background 740 and select a pre-stored image from the list area 750 to enter the background 740 instead.

8 illustrates a screen for pre-editing a captured image according to an embodiment of the present invention.

The user can select an object through a user interface that is displayed overlaid on the main image and move it to another location. In the example of FIG. 8A, it can be seen that the user moved the object by dragging and dropping the object.

As shown in (b) of FIG. 8, the user can enlarge or reduce the size of the object, rotate or invert the object through a predetermined input.

As shown in (c) of FIG. 8, the user may select a new background image from the background list and combine it with the main image instead of the erased background as shown in (d). That is, a new photo file may be created and stored by synthesizing the spoiled partial image with a previously stored image.

When a user selects a background color, graphic image, or pattern instead of a background image, a color is applied to the background of the damaged partial image, or a composite picture in which the graphic image or pattern is used as a background is created and saved. In addition, when blurring is set on a partial image selected by the user, a composite picture with the blur of the corresponding partial image adjusted is created and saved, resulting in a picture that is out of focus, similar to a shot using a telephoto lens. It becomes possible to acquire an image. At this time, the user can set the blur level of the selected background image and blur effects such as motion blur and radial blur to obtain the appropriate out-focusing that the user intends. By adjusting the color, shade, saturation, brightness, pixel size, and texture of the background, you can create a photo of a new image according to the user's intention.

In step S36 of FIG. 3, a user's photo storage request is received. On the screen displaying the user interface in which the user can select or delete the main image and each partial image in the form of a live view, after finishing the line editing as desired, the user will press the shoot button. Touching or clicking the photographing button corresponds to a request to save a photo.

In step S37 of FIG. 3, a photograph taken in response to the selection of the photographing button is stored. At this time, the stored photo is an image displayed on the screen, in which some partial images of the background and objects have been deleted from the main image according to the user's editing input in step S35, and this image is created as a new main image and Save it to. When the size of the subject is automatically adjusted, the adjusted image will be saved. The new main image can be saved in an image file format such as PNG (Portable Network Graphics) or PSD (Photoshop Document). In addition, the original main image before being edited can be saved in a layer format together with the new main image.

As described above, according to an embodiment of the present invention, it is possible to check an image while taking a picture to remove a background or to take a picture from which an unwanted object has been deleted in real time and store it.

Additionally, it is possible to select a background or an object even after completing and saving the photo. As described above, a photo (Luke image) from which the partial image selected by the user has been deleted is stored in the storage unit, and then read from the storage unit when the user requests it, and displayed on the screen, according to the user's manipulation. It is possible to obtain a photo obtained by synthesizing the image taken with one of the cameras in the background image of the Nuki image being displayed on the screen.

9 is a view for explaining an operation of taking a picture with a plurality of cameras including a front camera according to an embodiment of the present invention.

Referring to FIG. 9, a user 910 executes a dedicated application and turns on the rear cameras of the digital photographing apparatus 900 to select and photograph only desired objects while viewing subjects 920 including a Lego doll, etc. If you turn on and take a picture of yourself, you can combine the image taken with the rear camera and the user's image taken with the front camera.

The dedicated application will provide a user interface that selects only a partial image desired by the user from the images photographed by the subjects 920 and displays an image overlaid on the image of himself/herself taken by the front camera on the screen. When a user requests to save a photo after checking the image being synthesized, the synthesized image will be created and stored as a main image. Accordingly, the user can complete an image obtained by synthesizing an image taken with the front camera with a character image without a background prepared in the previous step.

10 illustrates a screen for synthesizing an image captured by a front camera and an image captured by a rear camera according to an embodiment of the present invention.

Referring to FIG. 10, in the dedicated application, if the user selects only the object desired by the user from the image taken with the rear camera and then turns on the front camera to take a picture of himself, as shown in (a), the user image 1010 and the selected object are selected. It shows a live view in which the object image 1020 is displayed together. The user may edit the size of the object 1020 or move the position of the object 1020 on the screen of (a).

As shown in (b) of FIG. 10, when the size of the object image 1020 is enlarged as desired and then the photographing button is touched or clicked, a photo containing the user image 1010 and the object image 1020 is created and stored in the memory. Will be. At this time, since the dedicated application provides a preview in a live view method, the user can shoot while changing his or her facial expression or appearance.

In one embodiment, when the user turns on the front camera to photograph his or her face, the directions of the images captured by the front camera and the rear camera are opposite to each other, so a dedicated application flips the front image left and right, so that the final main image suits the user's needs. Can be obtained.

In one embodiment, the dedicated application may recognize a face of a subject photographed with a front camera and a face of a subject photographed with a rear camera, and automatically adjust the ratio of both. As in the example of FIG. 10, even if the user does not enlarge the size of the lego doll image 1020 to fit the size of the face of the user, the dedicated application automatically recognizes the faces of the user 1010 and the lego doll 1020, and the size is reduced. The lego doll image 1020 can be enlarged to be similar. In particular, because the front and rear cameras have different lens types and the subject distances of the front and rear are different, the size of the face is captured differently, but a natural composite shot image by automatically adjusting the size and proportion of the person photographed at the same time in front and rear. Can be obtained.

In one embodiment, the dedicated application may automatically adjust the horizontal and vertical heights based on the shoulder positions or eye heights of people included in the front image and the rear image. Different from that shown in FIG. 10, when the face of the lego doll 1020 taken with the rear camera is positioned above or below the face of the user 1010 taken with the front camera, the dedicated application is the face of the lego doll 1020 ) To be positioned next to his face.

In one embodiment, the dedicated application can automatically adjust the left and right areas each occupied by a front image taken with a front camera and a rear image taken with a rear camera. For example, the number of faces included in the front image and the rear image may be recognized and automatically adjusted to occupy an area in the horizontal direction in proportion to the number of faces included in each image.

Two or more of the above embodiments may be implemented in combination. For example, if a dedicated application recognizes one user's face in the front image and determines that two faces are included in the selected objects in the rear image, 1/3 of the horizontal area of the screen is allocated to the front image and the rear image is Both images are synthesized by allocating 2/3 of the horizontal area of the screen, and the result of correcting the position of the rear image so that the face height of the front image and the face height of the rear image are similar can be displayed on the screen.

At this time, by acquiring image information such as color value, exposure value, brightness and contrast value between two subjects (object and background image) and adjusting similarly, an image that is naturally adjusted not only the size or area of the subject, but also the brightness level and color tone can be obtained. You will be able to.

FIG. 11 is a diagram illustrating a screen for synthesizing photographing after reading and displaying a nouki image without a background according to an embodiment of the present invention.

Referring to FIG. 11, when shooting with a front camera or a rear camera after displaying an image of a nouki image of an object without a background (Lego doll, 1110) that has already been photographed and stored, a nouki object 1110 and a photographed image 1120 It will be possible to shoot naturally and instantly composite images. That is, after displaying an object (eg, a person, a character, a piece, etc.) of an image without a background, only the background image of the object is photographed in real time, thereby obtaining an image in which a new background image is synthesized. By using this, since it is possible to shoot while changing the background of the object as shown in (b), (c), and (d) of FIG. 11, it is possible to easily create image photos in which different background images are combined with the same object.

For example, if you want to take a picture of a Lego doll stored on a desk in the sand at the seaside, now you have to take the Lego doll directly to the beach to take a picture, but if you apply the above technique, you can take a picture of Lego's nooki (a) indoors. It is convenient to shoot only the background at the beach after shooting. In addition, since it is possible to take a picture while changing to a desired background such as the sea (c), the bush (d), and the sky (e) as well as the sand background (b), a new image that the user imagines can be easily acquired.

At this time, the Noki image that is read out and displayed in advance for composite photography is not only real photos of people, dolls, objects, etc., but also characters, drawings, cartoons, logos, typographic fonts, and symbol marks that have been drawn or designed, and any images without a background can be used. This is possible.

In addition, it is also possible to zoom in or out by touching and dragging the displayed Noki image as shown in FIG. 11, or to rotate or reverse it, so that the background can be taken freely in a layout state, so that the user can easily take a picture while checking the desired composite picture. .

In one embodiment, the Nuki image that is read and displayed before photographing may be downloaded and used in real time from the server in addition to the Nuki image previously stored in the photographing device. Using location information such as GPS linked to the camera, the location of the shooting location is identified, and when the Noki image matching this location information is transmitted from the server, it is downloaded, displayed, and then photographed. At this time, not only must the Noki images matching each location information have to be prepared in advance in the server that can be linked with the photographing device, but also the Noki images displayed according to the changed location information when the user attempts to shoot by moving the location. It will be changed and displayed.

For example, when shooting with the above technology applied, a photo of Noki of Bae Yong-jun, the lead actor of'Winter Sonata', is displayed on Nami Island, and you can take a composite picture by taking a background with this person image floating. It is possible to take a composite picture 1170 while downloading and displaying the character 1130 of Lotte and the Mickey Mouse 1150 or Disney's symbol logo 1160 in Disneyland.

Using this, when shooting at a famous tourist spot, theme park, or festival place, it is possible to take a composite photo by putting the symbol image downloaded in real time from that place in one place, so you can leave a different commemorative photo, etc. When shooting at a chain store, it is possible to take a composite picture containing downloaded phrases, characters, and images based on the location of the chain store.

As described above, unlike the current camera property, which only reproduces the actual phenomenon or as it is, the above embodiments enable real-time acquisition of a new image in which an image intended or imagined by a user and a current real image are combined. From the concept of'record' to the concept of'creation', the scope of photography will be wider, and the use of composite photos and creative photos that remain in the realm of experts will be further expanded to the general public. It will also be another detonator.

The method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

100: digital photographing device
110: camera module
111: first camera (rear camera)
112: second camera (rear camera)
113: third camera (front camera)
120: processor
130: display
140: memory

Claims (9)

In the method of taking pictures of Nouki and taking a composite picture using the same,
Generating a depth map by analyzing a plurality of images captured through a plurality of cameras when a user requests to take a picture;
Determining one or more partial images constituting an image using the depth map;
Displaying, as a live view image, a user interface for dividing and selecting the one or more partial images on the main image displayed on the screen of the digital photographing apparatus;
Receiving a user's input through the user interface;
When a photo storage is requested from the user, generating a photo from which the partial image selected and excluded from the main image has been deleted as a new main image according to the user's input through the user interface and storing it in a storage unit; including; Photography method characterized by. The method of claim 1,
The storage step,
And storing, as a main image, a photo obtained by combining a photo from which the selected-excluded partial image is deleted and another image selected by the user as a main image. The method of claim 1,
The storage step,
And storing, as a main image, a photo obtained by combining a photo from which the selected-excluded partial image is deleted and an image captured by a front camera of the terminal as a main image. The method of claim 1,
The storage step,
And storing an original main image including even a partial image not selected by the user as a layer of the new main image. The method of claim 1,
The depth map generation step,
When the plurality of cameras have different angles of view, after cropping the image captured by the wide-angle camera so that the horizontal or vertical size of the same object included in the plurality of images captured by the plurality of cameras becomes the same, the plurality of And determining a separation distance between the same objects in the image. The method of claim 1,
Reading from the storage unit and displaying a noki image from which the partial image selected-excluded by the user has been deleted;
And obtaining a photo obtained by synthesizing an image captured by one of the plurality of cameras into a background image of the displayed Nouki image. The method of claim 1,
When a photographing request is requested from the user, downloading and displaying a Noki image matched with location information of a location where photographing is requested from a server;
And obtaining a photo obtained by synthesizing an image captured by one of the plurality of cameras into a background image of the displayed Nouki image. The method of claim 1,
Recognizing a face of a subject from a plurality of partial images selected through the user interface, and automatically adjusting the size of the selected partial images, the tone, color, and contrast of the images so that the size of the recognized face and image information are similar to each other; A photographing method comprising a further. A computer program recorded on a recording medium to perform the method of taking a picture of a noki according to any one of claims 1 to 8 and a method of taking a composite picture using the same.

Images