WO2014175616A1

WO2014175616A1 - Apparatus and method for synthesizing images using multiple cameras

Info

Publication number: WO2014175616A1
Application number: PCT/KR2014/003460
Authority: WO
Inventors: 권오형
Original assignee: 주식회사 모리아타운
Priority date: 2013-04-25
Filing date: 2014-04-21
Publication date: 2014-10-30
Also published as: KR101303279B1

Abstract

A method for synthesizing images using multiple cameras according to an embodiment of the present invention comprises the steps of: displaying an appearance of another person or an object, as a first preview image, on a screen through one camera among two or more cameras of a first portable terminal, and displaying an appearance of a user of the first portable terminal, as a second preview image, together with the first preview image on the screen through at least another camera among the two or more cameras of the first portable terminal; photographing the first and second preview images according to photographing manipulation of the user; and synthesizing first and second images related to the first and second preview images, generated by the photographing.

Description

Image Synthesis Device and Method Using Multiple Cameras

Embodiments of the present invention relate to an image synthesizing apparatus and method using multiple cameras.

Cameras and devices with built-in cameras capture a subject and then store the captured subject image file, play back the stored image, use it as a display screen on a display unit, or a PC or other device when equipped with an interface unit or a wireless communication unit. It has a function to transmit to.

In addition, a certain camera and a device with a built-in camera may set an image pre-stored in a storage unit, an image downloaded by connecting to a PC through an interface unit, or an image downloaded by wireless connection with an external server through an RF unit as a background screen on a display unit, and may be captured. Displayed on the display unit with the subject, and has the function to save as an image having a variety of background screens.

However, in the image capturing and storing method as described above, when a user selects an image to be used as a background screen in a camera or a device having a camera, the background image selected on the display unit is displayed and the image of the selected background image can be entered. To display the subject image currently captured in the captured image area.

That is, an area in which the subject image captured by the camera can be input is limited among the selected background image areas. Therefore, since the background image area and the captured subject image do not overlap each other and have overlapping boundaries, the area displaying the captured subject image is limited, and there is a problem in that various images cannot be generated and stored by overlapping the background image over the subject image. .

On the other hand, the prior art related to the present invention is disclosed in Japanese Patent Laid-Open Publication No. 10-2007-0027870 (name of the invention: a method for synthesizing a photographed image and a mobile device having a camera or a camera built therein for synthesizing a photographed image by the method) Date: March 12, 2007).

According to an embodiment of the present invention, by taking two or more images through two or more camera devices when taking a selfie (selfie) and combining them, images of not only the user but also another person (or object) are displayed on one image. Provided are an image synthesizing apparatus and method using multiple cameras using SNS.

According to an embodiment of the present invention, when a video call is made by synthesizing a photographed image of the other party with a user's own captured image, an image synthesizer using multiple cameras can be displayed as a single image on the screen by a person at a distance with the user himself. An apparatus and method are provided.

The problem to be solved by the present invention is not limited to the problem (s) mentioned above, and other object (s) not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, a method of synthesizing an image using multiple cameras displays a state of another person or an object as a first preview image on a screen through one of two or more cameras of a first portable terminal, Displaying a view of the user of the first portable terminal on the screen together with the first preview image as a second preview image through at least one camera of two or more cameras of the first portable terminal; Photographing the first and second preview images according to a photographing operation of the user; And synthesizing first and second images relating to the first and second preview images generated by the photographing.

The photographing of the first and second preview images may include: driving the two or more cameras of the first portable terminal when the second preview image is disposed at a desired position of the user. It may include the step of photographing.

In the multi-camera selfie synthesis method using the SNS according to an embodiment of the present invention, when the first mobile terminal makes a video call with the second mobile terminal, through one of the two or more cameras of the second mobile terminal Receiving a third image photographed and a fourth image photographed through at least one other camera among two or more cameras of the second mobile terminal; And synthesizing the third and fourth images with the second image and sharing them through a social network service (SNS).

An image synthesizing apparatus using multiple cameras according to an embodiment of the present invention may further include setting the third image as a background screen when the third and fourth images and the second image are synthesized.

Synthesizing the first and second images of the first and second preview images may include applying an image effect including a season, weather, or moving picture to a background screen.

An image synthesizing apparatus using an S multiple camera according to an embodiment of the present invention displays a state of another person or an object as a first preview image on a screen through one of two or more cameras of the first portable terminal, A display unit configured to display a view of a user of the first portable terminal along with the first preview image on the screen through at least one of two or more cameras of the first portable terminal as a second preview image; A photographing unit configured to photograph the first and second preview images according to a photographing operation of the user; And a synthesizer configured to synthesize first and second images related to the first and second preview images, generated by the photographing.

The photographing unit may photograph the first and second preview images by driving two or more cameras of the first portable terminal when the second preview image is disposed at a position desired by the user.

In the image synthesizing apparatus using the multiple cameras according to an embodiment of the present invention, when the first portable terminal makes a video call with the second portable terminal, the image is captured by one of two or more cameras of the second portable terminal. A third image, and a fourth image photographed through at least one other camera among two or more cameras of the second portable terminal, and receiving the SNS by synthesizing the third and fourth images with the second image. Social network service) may further include a sharing unit to share.

The synthesis unit may set the third image as a background screen when the third and fourth images and the second image are synthesized.

The synthesis unit may give an image effect including a season, weather, or a moving picture to a background screen.

Specific details of other embodiments are included in the detailed description and the accompanying drawings.

Advantages and / or features of the present invention and methods for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only the present embodiments to make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

According to an embodiment of the present invention, by taking two images through the front and rear camera when taking a selfie (selfie) and synthesized them, the image of not only the user but also another person (or object) is displayed on one image Do it.

According to an embodiment of the present invention, when a video call is made by synthesizing the captured image of the counterpart with the user's own captured image, a remote person can be displayed together with the user as an image on the screen.

According to an embodiment of the present invention, two or more pictures may be overlapped to take a 3D effect when taking a self-camera or two or more video calls.

According to an embodiment of the present invention, the 3D effect may be obtained by overlapping two or more videos (or two frames) when taking a self-camera or two or more video calls.

1 is a block diagram illustrating an image synthesizing apparatus using multiple cameras according to an embodiment of the present invention.

2 is a flowchart illustrating an image compositing method using multiple cameras according to an exemplary embodiment of the present invention.

3 is a flowchart illustrating an image compositing method using multiple cameras according to another exemplary embodiment of the present invention.

In an embodiment of the present invention, the camera is described as being limited to the front and the rear of the portable terminal, but is not limited thereto and may be provided at various positions. For example, the camera may be positioned in various ways such as left / right side, top / bottom side, or external mounting in addition to the front / rear side. It is apparent that the reason for limiting the position of the camera to the front / rear of the portable terminal is only for the purpose of understanding the present invention and does not limit the scope of the right.

Hereinafter, with reference to the accompanying drawings will be described embodiments of the present invention;

Referring to FIG. 1, an image synthesizing apparatus 100 using multiple cameras according to an exemplary embodiment of the present invention may include a display unit 110, a photographing unit 120, a combining unit 130, a sharing unit 140, and a controller. 150 may be included.

The display unit 110 displays the appearance of another person or object on the screen as a first preview image through the rear camera of the first portable terminal.

In addition, the display unit 110 displays a view of the user of the first portable terminal on the screen together with the first preview image as a second preview image through a front camera of the first portable terminal.

The photographing unit 120 captures the first and second preview images according to a photographing operation of the user.

In detail, the user may arrange the second preview image at a location desired by the user through a screen touch drag operation of the first mobile terminal. Then, the photographing unit 120 may drive the front camera and the rear camera of the first portable terminal to capture the first and second preview images.

Here, the position of the second preview image may be controlled by a hand gesture or an eye movement or a body movement, an operation by a voice or an action, an operation by a brain wave, or an operation or a remote control by a sensor or a camera on a screen of the first mobile terminal. The touch screen may be disposed by various methods as well as a touch screen drag operation such as a button operation.

The synthesizing unit 130 synthesizes the first and second images related to the first and second preview images generated by the photographing. At this time, the synthesis unit 130 may give an image effect including a season, weather or moving picture on the background screen.

When the first portable terminal makes a video call with a second portable terminal located at a long distance, the sharing unit 140 includes a third image photographed by a rear camera of the second portable terminal, and the second portable terminal. The fourth image photographed through the front camera may be received, and the third and fourth images may be synthesized and the second image may be shared through a social network service (SNS).

In this case, the combining operation of the third and fourth images and the second image may be performed by the combining unit 130. The synthesizing unit 130 may set the third image as a background image when the third and fourth images and the second image are synthesized.

The controller 150 is an image synthesizing apparatus 100 using multiple cameras according to an embodiment of the present invention, that is, the display unit 110, the photographing unit 120, the synthesizing unit 130, and the sharing unit. It is possible to control overall operations such as 140.

On the other hand, the multi-camera selfie synthesizing apparatus 100 may synthesize to make the face of the user or the face of another user who is talking to the user as another person's face when two or more calls. In addition, the multi-camera selfie synthesizing apparatus 100 may delete and display a specific part or a specific object in the picture during the call. In addition, the multi-camera selfie synthesizing apparatus 100 may insert or synthesize a specific image in the picture during the call.

1 and 2, in step 210, the display unit 110 of the multi-camera selfie synthesizing apparatus 100 displays a first preview image of another person or an object through a rear camera of the first mobile terminal. On the screen.

Next, in operation 220, the display unit 110 of the multi-camera selfie synthesizing apparatus 100 displays the image of the user of the first mobile terminal as a second preview image through the front camera of the first mobile terminal. 1 Display with the preview image on the screen.

Next, in step 230, the user places the second preview image at a location desired by the user through a screen touch drag operation.

Next, in operation 240, the capturing unit 120 of the multi-camera selfie synthesizing apparatus 100 drives the front camera and the rear camera of the first portable terminal according to the photographing operation of the user. Shoot the second preview image.

Next, in step 250, the combining unit 130 of the multi-camera selfie synthesizing apparatus 100 synthesizes first and second images related to the first and second preview images generated by the photographing. .

3 is a flowchart illustrating an image compositing method using multiple cameras according to another exemplary embodiment of the present invention. In particular, FIG. 3 is a diagram for describing an exemplary embodiment in which the first mobile terminal makes a video call with a second mobile terminal located at a long distance.

1 and 3, in operation 310, the sharing unit 140 of the multi-camera selfie synthesizing apparatus 100 transmits third and fourth images from the second mobile terminal in the first mobile terminal. Get it.

Here, the third image is an image captured by the rear camera of the second portable terminal, and the fourth image is an image photographed by the front camera of the second portable terminal.

Next, in operation 320, the synthesis unit 130 of the multi-camera selfie synthesis apparatus 100 synthesizes the third image and the fourth image with the second image. In this case, the third image may be set as a background screen and displayed.

Next, in step 330, the sharing unit 140 of the multi-camera selfie synthesizing apparatus 100 shares the synthesized image with the second mobile terminal through a social network service (SNS). do.

Hereinafter, an image synthesis apparatus and method using multiple cameras according to an embodiment of the present invention will be described in detail with reference to examples.

<셀카 모드><Selfie Mode>

1. In general, when taking a selfie (self-camera), a group of people with one camera is gathered, and one of them takes a camera.

However, if one or two or three selfies are slightly distorted, they can be roughly packed on one screen, but the quality is not good, and the composition and angle are not good.

If you have three or more people here, no matter how straight your arms are, taking multiple selfies is hard.

Therefore, in the selfie of the present invention, one person who takes a selfie is holding the camera, and the other person stands in the pose of the original camera opposite to the camera.

In this case, the video on the opposite side is taken by the rear camera, and the front camera is taken by one person.

The photographer takes a look at the preview image of the friends who are in the shooting mode in front of the camera, and looks at the preview image composed of their own images.

This will naturally combine your selfies with your friends in the background and take a picture.

2. In this way, you can combine photos while having a video call with a friend who is not currently with you.

In other words, during a video call, the image of the Eiffel Tower in front of the Eiffel Tower in Paris was shown to a friend in his house in Seoul, and the (Rear Camera image of the Eiffel Tower) + (My Front camera image) + (video of a friend who makes a video call in Seoul's own house), and then combines the three images, synthesizing the video as if a friend in her home in Seoul took a picture together in front of the Eiffel Tower. Can be shared via SNS.

<화상 통화 모드><Video call mode>

1. Combining two faces into one image during a video call

2. Combining two faces in a video call with the background of user A or user B's back camera

3. Herein, although A and B users are described in the video call, the present invention is not limited thereto, and there may be a C user and a D user.

4. For group video calls with 3 or more people, one default screen can be designated and all other users can be synthesized.

5. And, when a group video call with three or more people, one basic screen is designated, or two or more screens are divided or overlapped with different transparency to be combined and provided as a basic screen. You can composite, subtract or modify parts of your screen.

<이미지 효과 모드><Image Effect Mode>

1. Composition that changes the seasons of the background to spring summer autumn winter

2. Composition that turns sunny days into cloudy, cloud, rain, thunder, snow, fog, etc.

3. A composition that converts rainy days into sunny or rainy days

4. A composition that changes the snowy day into a rainy or sunny day

<셀카를 이용한 파노라마 효과 모드><Panorama effect mode using selfie>

1. When taking a panoramic picture while moving the rear camera from left to right, take a picture of yourself in the front camera, and combine the panorama picture with my picture to take a panoramic selfie picture

<움직이는 그림 효과 모드><Moving picture effect mode>

1. Construct a moving picture (GIF) with the background moving forward

<재미있는 이미지 효과 모드><Funny image effect mode>

1. When composing multi-camera selfies, human face is automatically recognized and only the face part of the body is enlarged to make interesting pictures.

2. When composing multi-camera selfies, it automatically recognizes a person's face and synthesizes a friend's face in my body and a friend's body in my

3. When synthesizing multi-camera selfies, it automatically recognizes people's faces and randomizes or assigns them to other friends' bodies.

4. When compositing multi-camera selfies, the composition is made for fat people slim or slim people fat, short people tall or short people tall.

5. When compositing multi-camera selfies, black is composed of white or yellow, white is black or yellow, and yellow is composed of white or black.

6. Combining hair styles, makeup or clothing styles or accessories (earrings, bracelets, headbands, watches, rings, etc.) when composing multiple camera selfies.

<3D 효과 1><3D effect 1>

I assume a, my terminal is A, the front camera is A1 and the rear camera is A2,

Suppose your friend is b, your friend's terminal is B, your friend's front camera is B1, and your friend's back camera is B2.

1. In a method of compositing with one or more camera sources,

2. A configuration that receives more than one screen from a source and displays them as 3D stereoscopic images by superimposing these screens

<3D 효과 2><3D effect 2>

1. In a method of compositing with one or more camera sources,

2. Configuration of a method of receiving one or more screens from a source and stacking or composing them with different transparency of each screen source

<3D 효과 3><3D effect 3>

1. In a method of compositing with one or more camera sources,

2. A composition that receives one or more screens from a source and displays 3D effects on one or more sections of a specific area when they overlap.

<3D 효과 4><3D effect 4>

1. In a method of compositing with one or more camera sources,

3. During a video call with a friend,

I am a, my terminal is A, the front camera is A1, the rear camera is A2,

A friend is b, a friend's terminal is B, a friend's terminal's front camera is B1, and a friend's rear camera is B2. ).

In this case, A2 is photographing the Eiffel Tower, A1 is photographing the upper body of me or a,

B1 is photographing the Statue of Liberty and B2 is photographing the face of friend b.

I have a video call with each other, and a wants to see the Statue of Liberty of b in stereoscopic 3D.

In this case, one or more pictures received from b can be superimposed on one or more pictures and viewed as a 3D effect picture.

<3D 효과 5><3D effect 5>

1. A1 single screen or A2 single screen or composite screen of A1 and A2 by overlapping one or more screens to create and save 3D effects, share on SNS, or transmit to other terminals, servers or people

<3D 효과 6><3D effect 6>

1. Taking a picture shows a 3D effect by stacking two or more pictures, and a video can also display a 3D effect by stacking two or more pictures.

2. Here, the video refers to all non-stop video such as general video shooting, moving video of a friend's moving preview screen during a call, and preview video previewing a selfie.

In the above 3D effects 1 to 6, the contents shown in the 3D screen may include a preview, a photo, a video, a call picture, a call video, an SNS picture, an SNS video, and the like.

Embodiments of the invention include a computer readable medium containing program instructions for performing various computer-implemented operations. The computer readable medium may include program instructions, local data files, local data structures, or the like, alone or in combination. The media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical recording media such as CD-ROMs, DVDs, magnetic-optical media such as floppy disks, and ROM, RAM, flash memory, and the like. Hardware devices specifically configured to store and execute the same program instructions are included. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

While specific embodiments of the present invention have been described so far, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below, but also by the equivalents of the claims.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

[Description of the code]

110: display unit

120: the photographing unit

130: synthesis section

140: sharing unit

150: control unit

Claims

Displaying a picture of another person or an object on the screen as a first preview image through one of two or more cameras of the first mobile terminal, and through at least one other of two or more cameras of the first mobile terminal Displaying a view of a user of the first mobile terminal as a second preview image on the screen together with the first preview image;

Photographing the first and second preview images according to a photographing operation of the user; And

Synthesizing first and second images relating to the first and second preview images generated by the photographing;

Multi-camera selfie synthesis method using SNS comprising a.
The method of claim 1,

Taking the first and second preview images,

Photographing the first and second preview images by driving two or more cameras of the first portable terminal when the second preview image is disposed at a position desired by the user;

Multi-camera selfie synthesis method using SNS comprising a.
The method of claim 1,

When the first portable terminal makes a video call with the second portable terminal,

Receiving a third image photographed by one camera of two or more cameras of the second portable terminal and a fourth image photographed by at least one other camera of two or more cameras of the second portable terminal; ; And

Synthesizing the third and fourth images with the second image and sharing them through a social network service (SNS);

Multiple camera selfie synthesis method using SNS, characterized in that it further comprises.
The method of claim 3,

Setting the third image as a background image when the third and fourth images are combined with the second image;

Multiple camera selfie synthesis method using SNS, characterized in that it further comprises.
The method of claim 1,

Synthesizing the first and second images of the first and second preview images,

Steps to give your wallpaper an image effect that includes a season, weather, or animated picture

Multi-camera selfie synthesis method using SNS comprising a.
The method of claim 1,

When the first mobile terminal makes a video call with a second mobile terminal in a long distance,

Superimposing one or more images (pictures) received from the second portable terminal on the first portable terminal and synthesizing one or more images to display the photo with a 3D effect

Multiple camera selfie synthesis method using SNS, characterized in that it further comprises.
The method of claim 1,

Receiving and overlapping at least one screen shot by the first portable terminal to display a 3D stereoscopic screen;

When receiving and overlapping at least one screen shot by the first mobile terminal, displaying the 3D stereoscopic screen by overlaying or combining the screen sources with different transparency;

Displaying and displaying a 3D effect on at least one portion of a specific area when receiving and overlapping at least one screen shot by the first portable terminal;

The single screen of the first portable terminal, or the single screen of the first and second portable terminals, or the composite screen of the first and second portable terminals are overlapped with one or more screens to create and store a 3D effect. SNS sharing, or transmitting to another terminal or server; or

Superimposing two or more frames of a video taken by the first mobile terminal to create and store a 3D effect, share it on SNS, or transmit it to another terminal or server

At least one of:

Contents displayed on the 3D screen is a multi-camera selfie synthesis method using SNS, characterized in that it comprises at least one of preview, photo, video, call picture, video call, SNS picture, SNS video.
The method of claim 1,

Synthesizing a face of the user or a face of another user who is in a call with the user as a different person during two or more calls;

Deleting and displaying a specific part or a specific object in the picture during the call; or

Inserting or composing a specific image in a photo during the call

Multiple camera selfie synthesis method using SNS, characterized in that it further comprises at least one of.
Displaying a picture of another person or an object on the screen as a first preview image through one of two or more cameras of the first mobile terminal, and through at least one other of two or more cameras of the first mobile terminal A display unit configured to display a state of the user of the first portable terminal on the screen together with the first preview image as a second preview image;

A photographing unit configured to photograph the first and second preview images according to a photographing operation of the user; And

A synthesizer for synthesizing the first and second images related to the first and second preview images generated by the photographing;

Multi-camera selfie synthesizing apparatus using SNS, characterized in that it comprises a.
The method of claim 9,

The photographing unit,

When the second preview image is disposed at a location desired by the user, driving the two or more cameras of the first mobile terminal to capture the first and second preview images, and synthesizing the multi-camera selfie using SNS. Device.
The method of claim 9,

When the first portable terminal makes a video call with the second portable terminal,

Receiving a third image photographed through one camera of two or more cameras of the second portable terminal and a fourth image photographed through at least one other camera among two or more cameras of the second portable terminal, A sharing unit for synthesizing the third and fourth images and the second image and sharing them through a social network service (SNS).

Multi-camera selfie synthesizing apparatus using SNS, characterized in that it further comprises.
The method of claim 11,

The synthesis unit,

The multi-camera selfie synthesizing apparatus using SNS, wherein the third image is set as a background image when the third and fourth images and the second image are synthesized.
The method of claim 9,

The synthesis unit,

Multi-camera selfie synthesizing apparatus using SNS, characterized in that to give the background image effects including the season, weather or moving picture.
The method of claim 9,

The multi-camera selfie composite device

When the first portable terminal makes a video call with the second portable terminal,

Multi-camera selfie synthesizing apparatus using SNS, characterized in that the one or more images (pictures) received from the second portable terminal in the first portable terminal by superimposing one or more images to synthesize the 3D effect.
The method of claim 9,

The multi-camera selfie composite device

Each screen is displayed when the at least one screen shot by the first portable terminal is overlaid and displayed as a 3D stereoscopic screen, or when the at least one screen shot by the first portable terminal is overlaid. Overlapping or synthesizing different sources of transparency to show a 3D stereoscopic screen, or when receiving and superimposing at least one screen shot by the first mobile terminal, a 3D effect is displayed on at least one portion of a specific area. A single screen of the first portable terminal, a single screen of a second portable terminal remote from the first portable terminal, or a composite screen of the first and second portable terminals. Create and store 3D effects, share them on social media, send them to other terminals or servers, or by the first mobile terminal Overlapping two or more frames of the video to create a 3D effect, save it, share it on SNS, or send it to another terminal or server. Multi-camera selfie synthesizer using SNS, characterized in that it comprises at least one of, SNS picture, SNS video.
The method of claim 9,

The multi-camera selfie composite device

Synthesis that makes the face of the user or the face of another user who is talking to the user appear as another person's face when two or more people talk, or by deleting a specific part or a specific object from the picture during the call, Multi-camera selfie synthesizing apparatus using SNS, characterized in that by inserting or combining a specific image in the picture.