TWI637347B - Method and device for providing image - Google Patents

Abstract

An image providing method includes: displaying a first image, the first image including an object and a background; receiving a user input selecting the object or the background as a region of interest; based on the first image Attribute information to obtain first identification information associated with the region of interest; obtaining a second image from the target image, the second image includes second identification information, and the second identification information is identical to the first identification Information; and generating an effect image based on at least one of the first image and the second image.

Description

Method and device for providing images

This application claims No. 10-2014-0098589 filed by the Korea Intellectual Property Office on July 31, 2014, and No. 10-2014-0111628 filed on August 26, 2014, and in 2015 The priority of the Korean Patent Application No. 10-2015-0078777, filed on Jan. 3, the entire disclosure of which is hereby incorporated by reference.

One or more exemplary embodiments are directed to a method and apparatus for providing images.

Electronic devices have become diverse, and the types of electronic devices used by each have become more diverse. Users use a variety of electronic devices they use to use a variety of images, applications, and services, and the number of images available to users is increasing.

Therefore, the user can encounter many types of images, but the images preferred by the user may be different. In addition, the user may be interested in specific parts of the image. Therefore, it is still necessary to efficiently provide a portion of the image that is of interest to the user.

According to an aspect of an exemplary embodiment, an image providing method may include: displaying a first image, the first image including an object and a background; and receiving a user input selecting the object or the background as a region of interest Obtaining first identification information associated with the region of interest based on the first attribute information of the first image; acquiring a second image from the target image, the second image including second identification information, The second identification information is the same as the first identification information; and the effect image is generated based on at least one of the first image and the second image.

The first attribute information may include at least one of context information associated with the generation of the first image and annotation information about the first image, the annotation information being added by a user.

The first identification information can be obtained by summarizing the first attribute information based on a word network (WordNet).

The obtaining the second image may include: acquiring, by using at least one of the second attribute information of the second image and the image analysis information of the second image, the second identification of the second image News.

The first identification information of the region of interest may be obtained from the first attribute information, and the first attribute information may include a plurality of attributes of the first image.

The method can include displaying a list of the plurality of attributes of the first image.

The method can include receiving user input, the user inputting Selecting at least one of the plurality of attributes of the first image; and generating the first identification information based on the selected at least one attribute, wherein the step of obtaining the second image comprises: Comparing the first identification information with the third identification information of the target image.

The step of generating the effect image may include: displaying a partial image of the second image, the partial image corresponding to the first identification information.

The effect image may be generated by at least one of highlighting a halo effect of the partial image, reducing a blur effect of a difference between pixel values of the partial image, and changing the partial image The size effect of the size and the depth effect of changing the depth information of the partial image.

The effect image may be obtained by combining a partial image of the second image with the region of interest of the first image, wherein the partial image corresponds to the first identification information.

The first image may be a live view image.

The second image may be a temporary image generated from the live view image before receiving a user input for storing an image.

The temporary image may be generated whenever a partial image of the live view image is sufficiently changed, wherein the partial image corresponds to the first identification information, and wherein the sufficient change is equal to or greater than a reference The value associated with the value of the change.

The effect image may be a moving picture, and the effect image includes the first image and the second image.

According to another aspect of the exemplary embodiments, a mobile device may include: a display for displaying a first image, the first image including an object and a background; and user input for receiving the object or the The background is selected as a user input of the region of interest; and the controller is configured to obtain the first identification information of the region of interest based on the first attribute information of the first image, and obtain the second image from the target image, The second image includes second identification information, and the second identification information is the same as the first identification information.

The controller can be configured to generate an effect image based on at least one of the first image and the second image.

The effect image may be obtained by combining a partial image of the second image with the region of interest of the first image, and wherein the partial image is corresponding to the second image The first part of the identification information.

The first attribute information may include at least one of context information associated with the generation of the first image and annotation information about the first image, the annotation information being added by a user.

The controller may be configured to obtain the first identification information by summarizing the first attribute information based on WordNet.

The controller may be configured to generate the effect image by combining a partial image of the second image with the region of interest of the first image, wherein the partial image and the first identification information Associated.

According to still another aspect of the exemplary embodiments, a method of providing an image may include: receiving a first image, the first image including at least an object and a background; and receiving Selecting the object or the background as a user input of a region of interest; determining identification information associated with the region of interest; searching for a plurality of target images using the identification information; selecting to associate with the identification information a second image; and generating at least one effect image by applying an effect to at least one of the first image or the second image.

The step of generating the at least one effect image can include applying the effect to the region of interest, or combining at least one of the first image and the second image.

The step of generating the at least one effect image may include generating a first effect image by applying the effect to the region of interest, and generating a second effect image by applying the effect to the second image, And the method further includes storing the first effect image and the second effect image.

The step of determining the identification information may include generating the identification information using one or more attributes of the first image.

The step of generating the identification information using one or more attributes of the first image may include generating the one or more attributes of the first image by performing image analysis on the first image.

100‧‧‧ device

110‧‧‧User input

120‧‧‧ Controller

130‧‧‧ display

140‧‧‧ memory

141‧‧ User Interface (UI) Module

142‧‧‧Notification module

143‧‧‧Image Processing Module

150‧‧‧Communicator

151‧‧‧Short-range wireless communicator

152‧‧‧Mobile Communicator

153‧‧‧Broadcast receiver

160‧‧‧Camera/digital camera

170‧‧‧Output

172‧‧‧ audio output

173‧‧‧Vibration motor

180‧‧‧ sensor

181‧‧‧Magnetic sensor

182‧‧‧Acceleration sensor

183‧‧‧ tilt sensor

184‧‧‧Infrared sensor

185‧‧‧Gyro sensor

186‧‧‧ position sensor

187‧‧‧Atmospheric pressure sensor

188‧‧‧ proximity sensor

189‧‧‧ Optical Sensor

190‧‧‧ microphone

200‧‧‧External devices

200‧‧‧Cloud Server

200-1‧‧‧ indicates the part shown in Figure 2

200-2‧‧‧ indicates the part shown in Figure 2

210‧‧‧ function window

210‧‧‧Cloud Server

210‧‧‧Communicator

212‧‧‧"Edit" item

220‧‧‧Edit Window/Controller

222‧‧‧"Effect Editing" item

230‧‧‧Storage

300-1‧‧‧ indicates the part shown in Figure 3

300-2‧‧‧ indicates the part shown in Figure 3

310‧‧‧ objects

320‧‧‧ objects

400-1‧‧‧ indicates the part shown in Figure 4

400-2‧‧‧ indicates the part shown in Figure 4

410‧‧‧ objects

420‧‧‧ objects

500-1‧‧‧ indicates the part shown in Figure 5

500-2‧‧‧ indicates the part shown in Figure 5

510‧‧‧ objects

520‧‧‧ objects

600-1‧‧‧ indicates the part shown in Figure 6

600-2‧‧‧ indicates the part shown in Figure 6

610‧‧‧ objects

620‧‧‧ objects

700-1‧‧‧ indicates the part shown in Figure 7

700-2‧‧‧ indicates the part shown in Figure 7

710‧‧‧ objects

720‧‧‧ objects

800-1‧‧‧ indicates the part shown in Figure 8

800-2‧‧‧ indicates the part shown in Figure 8

810‧‧‧ objects

820‧‧‧ objects

900-1‧‧‧ indicates the part shown in Figure 9.

900-2‧‧‧ indicates the part shown in Figure 9.

910‧‧‧ objects

920‧‧‧ effect list

1000-1‧‧‧ indicates the part shown in Figure 10

1000-2‧‧‧ indicates the part shown in Figure 10

1000-3‧‧‧ indicates the part shown in Figure 10

1010‧‧‧First object

1012‧‧‧First object

1020‧‧‧Second object

1022‧‧‧Second object

1100-1‧‧‧ indicates the part shown in Figure 11

1100-2‧‧‧ indicates the part shown in Figure 11

1110‧‧‧Background

1120‧‧‧ blurred background

1200-1‧‧‧ indicates the part shown in Figure 12A

1200-2‧‧‧ indicates the part shown in Figure 12A

1200-3‧‧‧ indicates the part shown in Figure 12A

1200-4‧‧‧ indicates the part shown in Figure 12B

1200-5‧‧‧ indicates the part shown in Figure 12B

1200-6‧‧‧ indicates the part shown in Figure 12B

1210‧‧‧First object

1212‧‧‧An object with a halo effect

1220‧‧‧Background

1222‧‧‧blurred background

1260‧‧‧Background

1262‧‧‧ flowing background

1270‧‧‧ indicator

1300-1‧‧‧ indicates the part shown in Figure 13

1300-2‧‧‧ indicates the part shown in Figure 13

1310‧‧‧Inquiry window

List of 1320‧‧

1601‧‧‧ first image

1602‧‧‧Second image

1603‧‧‧ Third image

Type 1610‧‧‧

1611‧‧‧Time

1612‧‧‧GPS

1613‧‧‧ resolution

1614‧‧‧ Size

1615‧‧‧Weather Information

1616‧‧‧ Temperature Information

1617‧‧‧Collection device

1618‧‧‧Users add information

1619‧‧‧ Object Information

1710‧‧‧ images

1712‧‧‧Background

1720‧‧‧Attribute Information

1730‧‧‧ Identification Information

1810‧‧‧Image

1812‧‧‧First object

1820‧‧‧ Identification Information

1900-1‧‧‧ indicates the part shown in Figure 19

1900-2‧‧‧ indicates the part shown in Figure 19

1900-3‧‧‧ indicates the part shown in Figure 19

1910‧‧‧ first image

1912‧‧‧First object

1920‧‧‧ Identification Information List

1930‧‧‧Second image

2000-1‧‧‧ indicates the part shown in Figure 20

2000-2‧‧‧ indicates the part shown in Figure 20

2010‧‧‧Effects folder

2020‧‧‧ effect image

2300-1‧‧ means the part shown in Figure 23

2300-2‧‧‧ indicates the part shown in Figure 23

2310‧‧‧ Effect Folder

2320‧‧‧Menu Window

2322‧‧‧Transfer

2330‧‧‧Select window

2332‧‧‧Contact

2600-1‧‧‧ indicates the part shown in Figure 26A

2600-2‧‧‧ indicates the part shown in Figure 26A

2600-3‧‧‧ indicates the part shown in Figure 26B

2600-4‧‧‧ indicates the part shown in Figure 26B

2600-5‧‧‧ indicates the part shown in Figure 26C

2600-6‧‧‧ indicates the part shown in Figure 26C

2610‧‧‧ first image

2612‧‧‧ objects

2620‧‧‧ effect list

2630‧‧‧ Identification Information List

2640‧‧‧Target image list

2650‧‧‧Second image

2660‧‧‧Second image

2662‧‧‧ objects

2670‧‧‧ Effect image

2710‧‧‧ first image

2712‧‧‧ Objects

2714‧‧‧Image/Part 1 Image

2720‧‧‧Second image

2722‧‧‧Objects / Part 2 Image

2730‧‧‧ Effect image

2732‧‧‧ District

2734‧‧‧ District

2800-1‧‧‧ indicates the part shown in Figure 28A

2800-2‧‧‧ indicates the part shown in Figure 28A

2800-3‧‧‧ indicates the part shown in Figure 28B

2800-4‧‧‧ indicates the part shown in Figure 28B

2800-5‧‧‧ indicates the part shown in Figure 28C

2800-6‧‧‧ indicates the part shown in Figure 28C

2810‧‧‧ first image

2814‧‧‧Background

2820‧‧‧ effect list

2830‧‧‧ Identification Information List

2840‧‧‧ Target image list

2850‧‧‧Second image

2860‧‧‧second image

2864‧‧‧Background

2870‧‧‧ Effect image

2910‧‧‧ first image

2912‧‧‧Image/Part III Image

2920‧‧‧Second image

2924‧‧‧Partial image/Part 4 image

2930‧‧‧Background imagery

2932‧‧‧Part 4 Image 2924 Area where no pixel information exists / Area without pixel information

2940‧‧‧ Effect image

3100-1‧‧‧ indicates the part shown in Figure 31

3100-2‧‧‧ indicates the part shown in Figure 31

3100-3‧‧‧ indicates the part shown in Figure 31

3110‧‧‧ Live view image

3112‧‧‧ Objects

3120‧‧‧ Interested images

3130‧‧‧ Live view image

3140‧‧ images

3200-1‧‧‧ indicates the part shown in Figure 32

3200-2‧‧‧ indicates the part shown in Figure 32

3200-3‧‧‧ indicates the part shown in Figure 32

3210‧‧‧ live view image

3212‧‧‧First object

3220‧‧‧ Interested images

3230‧‧‧Live view image/final temporary image

3240‧‧‧ Effect image

3300-1‧‧‧ indicates the part shown in Figure 33

3300-2‧‧‧ indicates the part shown in Figure 33

3300-3‧‧‧ indicates the part shown in Figure 33

3310‧‧‧ live view image

3312‧‧‧First object

3320‧‧‧Video of interest

3330‧‧‧ live view image

3332‧‧‧First object

3340‧‧‧ Effect image

3500-1‧‧ indicates the part shown in Figure 35

3500-2‧‧‧ indicates the part shown in Figure 35

3500-3‧‧‧ indicates the part shown in Figure 35

3510‧‧‧ live view image

3512‧‧‧First object

3520‧‧‧ Temporary imagery

3530‧‧‧ final temporary image

3532‧‧‧First object

3540‧‧‧ Temporary imagery

3542‧‧‧First object

3550‧‧‧ Effect image

3800-1‧‧‧ indicates the part shown in Figure 38

3800-2‧‧‧ indicates the part shown in Figure 38

3800-3‧‧‧ indicates the part shown in Figure 38

3810‧‧‧Menu image

3812‧‧‧ menu item

3820‧‧‧ effect list

3822‧‧‧effect items

3830‧‧‧ menu item

4010‧‧‧Menu image

4012‧‧‧First menu item

4014‧‧‧Second menu item

S110‧‧‧ operation

S120‧‧‧ operation

S130‧‧‧ operation

S1410‧‧‧ operation

S1420‧‧‧ operation

S1430‧‧‧ operation

S1440‧‧‧ operation

S1450‧‧‧ operation

S1510‧‧‧ operation

S1520‧‧‧ operation

S1530‧‧‧ operation

S1540‧‧‧ operation

S2110‧‧‧ operation

S2120‧‧‧ operation

S2130‧‧‧ operation

S2140‧‧‧ operation

S2150‧‧‧ operation

S2160‧‧‧ operation

S2170‧‧‧ operation

S2180‧‧‧ operation

S2190‧‧‧ operation

S2210‧‧‧ operation

S2220‧‧‧ operation

S2230‧‧‧ operation

S2510‧‧‧ operation

S2520‧‧‧ operation

S2530‧‧‧ operation

S2540‧‧‧ operation

S2550‧‧‧ operation

S3010‧‧‧ operation

S3020‧‧‧ operation

S3030‧‧‧ operation

S3040‧‧‧ operation

S3050‧‧‧ operation

S3410‧‧‧ operation

S3420‧‧‧ operation

S3430‧‧‧ operation

S3440‧‧‧ operation

S3450‧‧‧ operation

S3610‧‧‧ operation

S3620‧‧‧ operation

S3630‧‧‧ operation

S3640‧‧‧ operation

S3660‧‧‧ operation

S3710‧‧‧ operation

S3720‧‧‧ operation

S3730‧‧‧ operation

S3740‧‧‧ operation

S3910‧‧‧ operation

S3920‧‧‧ operation

S3930‧‧‧ operation

S3940‧‧‧ operation

S3950‧‧‧ operation

The following description of the exemplary embodiments, which will be apparent and understood, Flow Cheng Tu.

2 illustrates a graphical user interface (GUI) for providing an effect to an image, in accordance with an exemplary embodiment.

FIG. 3 is a reference diagram for explaining a method of providing a halo effect to an object, according to an exemplary embodiment.

FIG. 4 is a reference diagram for explaining a method of providing a blurring effect to an object, according to an exemplary embodiment.

5 and 6 are reference diagrams for explaining a method of providing a size effect to an object, according to an exemplary embodiment.

7 and 8 are reference diagrams for explaining a method of providing a depth effect to an object, according to an exemplary embodiment.

FIG. 9 is a reference diagram for explaining a method of displaying an effect list, according to an exemplary embodiment.

FIG. 10 is a reference diagram for explaining a method of providing an effect to a plurality of objects in an image, according to an exemplary embodiment.

FIG. 11 is a reference diagram for explaining a method of providing an effect to a background, according to an exemplary embodiment.

FIG. 12A is a reference diagram for explaining a method of providing an effect to both an object and a background, according to an exemplary embodiment.

FIG. 12B is a reference diagram for explaining a method of providing an effect to an image in response to a plurality of user inputs, according to an exemplary embodiment.

FIG. 13 illustrates a diagram for providing effects to a plurality of images, according to an exemplary embodiment User interface.

14 is a flowchart of a method in which a device provides an effect to a second image using identification information of a first image, in accordance with an exemplary embodiment.

Figure 15 is a flow diagram of a method in which a device generates identification information, in accordance with an exemplary embodiment.

FIG. 16 illustrates attribute information of an image according to an exemplary embodiment.

17 is a reference diagram for explaining an example of identification information in which a device generates an image based on attribute information of an image.

18 is a reference diagram for explaining an example in which a device generates identification information using image analysis information.

FIG. 19 illustrates an example in which a device displays a list of identification information, according to an exemplary embodiment.

Figure 20 illustrates an example in which the device displays an effect folder.

21 is a flowchart of a method in which a device provides an effect to an image stored in an external device, according to an exemplary embodiment.

22 is a flowchart of a method in which a device shares an effect image with an external device, according to an exemplary embodiment.

Figure 23 illustrates an example in which a device shares an effect image with an external device.

FIG. 24 is a schematic diagram of an image management system, according to an exemplary embodiment.

FIG. 25 is a flowchart of a method of providing an effect image by combining a plurality of images with each other, according to an exemplary embodiment.

26A-26C illustrate utilizing multiple image orientations, according to an exemplary embodiment The body provides an example of the effect.

FIG. 27 is a reference diagram for explaining a method of combining a plurality of images, according to an exemplary embodiment.

28A through 28C illustrate an example of providing effects to a background using a plurality of images, according to an exemplary embodiment.

FIG. 29 is a reference diagram for explaining a method of combining a plurality of images, according to another exemplary embodiment.

FIG. 30 is a flowchart of a method of providing an effect image using a live view image, according to an exemplary embodiment.

FIG. 31 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to an exemplary embodiment.

FIG. 32 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to another exemplary embodiment.

FIG. 33 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to another exemplary embodiment.

FIG. 34 is a flowchart of a method of generating an effect image from an instant view image, according to another exemplary embodiment.

FIG. 35 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to an exemplary embodiment.

FIG. 36 is a flowchart of a method of generating a moving picture from an live view image, according to an exemplary embodiment.

FIG. 37 is a flowchart of a method of reproducing a moving picture, according to an exemplary embodiment Figure.

FIG. 38 is a reference diagram for explaining a method of displaying an effect on a menu image, according to an exemplary embodiment.

FIG. 39 is a flowchart of a method of providing an effect to a menu item according to the number of times an application corresponding to a menu item is executed, according to an exemplary embodiment.

FIG. 40 illustrates an example of displaying a menu image in which an effect has been provided to a menu item according to the number of times an application corresponding to a menu item is executed, according to an exemplary embodiment.

41 to 45 are block diagrams of apparatuses according to an exemplary embodiment.

FIG. 46 is a block diagram showing the structure of a cloud server, according to an exemplary embodiment.

The exemplary embodiments are now described in detail with reference to the exemplary embodiments in which the In this regard, the exemplary embodiments may have different forms and should not be construed as limited to the descriptions set forth herein. Accordingly, the exemplary embodiments are merely described by way of the accompanying drawings,

Although the present invention has been described in connection with the general terms that are widely used at the present time in consideration of the function of the present invention, the general terms may be based on the intention of the person having ordinary knowledge in the art, the situation precedent, the appearance of new technology, and the like. Changed. The terms arbitrarily chosen by the applicant may also be used in a specific case. In this case, the meaning of the term is given in the detailed description of the specification. Therefore, the terminology It must be defined on the basis of its meaning and the content of the entire specification, and is not defined solely by the stated terms.

The terms "comprises" and / or "comprising" or "includes" and / or "including" are used in this specification to indicate the existence of the stated elements, and The existence or addition of one or more other elements is not excluded. The terms "..." and "...module" as used in this specification refer to a unit in which at least one function or operation is performed, and can be implemented as hardware, software, or hardware and software. combination.

Throughout this specification, "images" may include objects and backgrounds. The object is a partial image that can be distinguished from the background by contouring by image processing or the like, and the object can be, for example, a person, an animal, a building, a vehicle, or the like. The background is a partial image other than the object. Some of the images that can be objects and backgrounds are not fixed, but can be relative. For example, in an image including a person, a vehicle, and a sky, a person and a vehicle may be objects, and the sky may be a background. In images including people and vehicles, a person may be an object and a vehicle may be a background. However, the size of a portion of the image of the object may be smaller than the size of a portion of the image of the background. Each device 100 can be pre-defined to standardize the object and background.

Throughout this specification, images can be still images (eg, pictures or drawings), moving images (eg, TV program images, Video On Demand (VOD), user-created content (user-created content) , UCC), music video, or YouTube video), live view images, menu images, and more.

Throughout the specification, the region of interest can be a partial image of the image, and Can be an object or background. Providing an effect to an image is an image editing type and means providing a region of interest in a completely different way than the previously provided region of interest. Providing images indicates displaying images, reproducing images, storing images, and the like.

An imaging system that provides an effect to an image will now be explained. The imaging system can include a device 100 capable of reproducing and storing images, and can further include a server that stores the images. The case where the image system includes the server will be explained in detail later.

The device 100 according to an exemplary embodiment may be a device capable of displaying an image and providing an effect to the image. The apparatus 100 according to an exemplary embodiment may be achieved in various types. For example, the device 100 can be a desktop computer, a mobile phone, a smart phone, a laptop, a personal computer (PC), an e-book terminal, a digital broadcast terminal, and a personal digital assistant. , PDA), portable multimedia player (PMP), navigation machine, MP3 player, digital camera, Internet Protocol television (IPTV), digital television (DTV), consumption Consumer electronics (CE) devices (for example, refrigerators and air conditioners each including a display) and the like, but the exemplary embodiments are not limited thereto. Device 100 can also be a device that can be worn by a user. For example, device 100 can be a watch, glasses, ring, bracelet, necklace, or the like.

1 is a flow chart of a method of providing an effect to an image, in accordance with an exemplary embodiment.

In operation S110, the device 100 can display an image. Images can include objects and backgrounds, and can be still images, moving pictures, live view images, menu shots Like waiting.

According to an exemplary embodiment, the image displayed on the device 100 may be a still image, a moving picture, or a menu image stored in a memory built in the device 100, and may be captured by the camera 160 built in the device 100. The live view image can be stored in an external device (for example, a portable terminal used by another user, a social networking service (SNS) server, a cloud server, or a web server) Still image, moving picture, or menu image, or can be a live view image taken by an external device.

In operation S120, the device 100 may select a region of interest. The region of interest is a partial image of the displayed image and may be an object or a background. For example, the device 100 may select one object from among a plurality of objects as a region of interest, or may select at least two objects from the plurality of objects as a region of interest. Alternatively, device 100 may select the background of the image as the region of interest.

According to an exemplary embodiment, device 100 may select a region of interest based on user input. For example, device 100 can receive a user input that selects a partial region on the image and determine an object or background that includes the selected partial region as the region of interest.

According to an exemplary embodiment, the user input selecting the region of interest may vary. In the present specification, the user input may be a key input, a touch input, a motion input, a bending input, a voice input, a multiple input, or the like.

The "touch input" indicates a gesture or the like made by the user on the touch screen to control the device 100. Examples of touch input can include tapping (tap), touch & hold, double tap, drag, panning, flick, and drag & drop.

"Tap" means that the user touches the screen with a fingertip or a touch tool (for example, an electronic pen) and then quickly lifts the fingertip or the touch tool from the screen without moving.

"Long press" means that the user keeps the touch input for a critical time period (for example, two seconds) or more after touching the screen with a fingertip or a touch tool (for example, an electronic pen). For example, the action indicates a situation in which the time difference between the touch start time and the touch end time is greater than the critical time period (eg, two seconds). In order to enable the user to determine whether the touch input is a tap or a long press, the feedback signal can be provided visually, audibly or tactilely when the touch input remains above a critical time period. The critical time period can vary depending on the exemplary embodiment.

"Double click" means that the user can quickly touch the screen twice with a fingertip or a touch tool (for example, an electronic pen).

"Drag" means that the user touches the screen with a fingertip or a touch tool and moves the fingertip or the touch tool to other positions on the screen while the screen is being touched. By dragging, the object can move, or the translational action that will be explained below can be performed.

"Pan" means that the user performs a drag action without selecting any object. Since no specific object is selected for the panning action, no objects in the page move. Instead, the entire page can be moved on the screen, or a group of objects can move within the page.

"Sliding" means that the user performs a dragging action at a critical speed (for example, 100 pixels/second) with a fingertip or a touch tool. The sliding motion and the drag (or translation) motion may be distinguished based on whether the moving speed of the fingertip or the touch tool is greater than the critical speed (eg, 100 pixels/second).

"Drag and drop" means the action of the user dragging an object with a fingertip or a touch tool and placing the object at a predetermined position in the screen.

"Pinch" means that the user touches the screen with a plurality of fingertips or a touch tool and widens or narrows the distance between the plurality of fingertips or the touch tool while touching the screen. . "Unpinching" means that the user touches the screen with two fingers (for example, the thumb and forefinger) and widens the distance between the two fingers while touching the screen, and "pinch ( Pinching) means an action in which the user touches the screen with two fingers and narrows the distance between the two fingers while touching the screen. A widening value or a narrowing value may be determined according to the distance between the two fingers.

"Swipe" means the user moves a certain distance while touching the object on the screen with a fingertip or a touch tool.

"motion input" means the motion applied by the user to the device 100 to control the device 100. For example, the motion input can be an input to the user rotating device 100, tilting device 100, or horizontal or vertical mobile device 100. The device 100 can sense an action input preset by a user by using an acceleration sensor, a tilt sensor, a gyro sensor, a three-axis magnetic sensor, or the like.

"Bending input" means that the device 100 is flexible When the device is shown, a portion of the device 100 or the entire device 100 is bent to control user input of the device 100. According to an exemplary embodiment, the device 100 may utilize a bending sensor to sense, for example, a bending position (coordinate value), a bending direction, a bending angle, a bending speed, a number of bending times, a time point at which the bending occurs, and a time period in which the bending is maintained.

"Key input" means that the user input of the device 100 is controlled by the physical keys attached to the device 100.

"Multiple inputs" means a combination of at least two input methods. For example, the device 100 can receive touch input and motion input from a user, or receive touch input and voice input from a user. Alternatively, device 100 can receive touch input and eye input from a user. The eyeball input indicates adjustment of blinking, gaze position, eye movement speed, and the like to control user input of the device 100.

For ease of explanation, the case where the user input is a key input or a touch input will now be explained.

According to an exemplary embodiment, device 100 may receive a user input selecting a preset button. The preset button can be a physical button attached to device 100 or a virtual button in the form of a graphical user interface (GUI). For example, when the user selects both a first button (eg, a home button) and a second button (eg, a volume control button), device 100 can select a partial region on the screen.

The device 100 can receive user input that touches a portion of the image displayed on the screen. For example, device 100 can receive an input that touches a portion of the displayed image for a predetermined period of time (eg, two seconds), or that touches the portion of the portion for a predetermined number of times (eg, a double tap). Then, the device 100 An object or background including the touched partial area may be determined as the area of interest. In other words, device 100 can select a region of interest.

Device 100 can determine a region of interest from an image. The device 100 can use the image analysis information to determine the region of interest from the image. For example, device 100 can detect an outline of an object displayed on the touched area. The device 100 can compare the contour of the object contained in the image with a predetermined template, and detect the type, name, and the like of the object. For example, when the outline of the object is similar to the vehicle template, the device 100 can identify the object contained in the image as the vehicle and determine the vehicle image as the region of interest.

According to an exemplary embodiment, the device 100 may perform face recognition on an object included in an image. Examples of the face detection method may include a knowledge based method, a feature based method, a template matching method, and an appearance based method, but the exemplary embodiments are not limited thereto.

Facial features can be extracted from the detected face (eg, the shape of the eyes, nose, and mouth that are the major portions of the face). To extract facial features from the face, a gabor filter or a local binary pattern (LBP) may be used, but the exemplary embodiments are not limited thereto.

Alternatively, device 100 may extract visual features (eg, color arrangement, pattern, and ambience of the image) as image analysis information by comparing a region of the image to a color map (color histogram).

In operation S130, the device 100 provides an effect to the image. The device 100 can provide an effect to the region of interest of the image to match the previously displayed region of interest The area of interest is provided completely differently. The effect can be provided in a variety of ways.

2 illustrates a graphical user interface (GUI) for providing effects, in accordance with an exemplary embodiment.

As shown in 200-1 of Figure 2, device 100 can display any screen image. Examples of the arbitrary screen image may include: a still image displayed by executing a photo album application, a live view image displayed by executing a photography application, and a moving image displayed by executing a mobile photo album application; The mobile image frame and the menu image that includes the menu items used to execute the application. Device 100 can provide a function window 210 for functions that can be used on any of the screen images.

Function window 210 provides various items that represent functions that can be used on any screen image. The user can select the "Edit" item from the function window 210. When the user selects the "Edit" item 212 on the function window 210, the device 100 can provide an edit window 220 including various edit items, as shown at 200-2 of FIG. The function window 210 and the edit window 220 can be graphical user interfaces.

Referring to 200-2 of FIG. 2, device 100 can display edit window 220 on the screen image to determine an editing method. When the user selects the "Effect Edit" item 212 on the edit window 220, the device 100 can provide an effect of displaying a partial image of the image completely different from the existing partial image.

An example of providing an effect to an image will now be elaborated.

FIG. 3 is a reference diagram for explaining a method of providing a halo effect to an object, according to an exemplary embodiment. As shown in 300-1 of FIG. 3, the device 100 can display at least one while a specific application (eg, a photo album application) is being executed. Images. The device 100 can receive a user input that selects an object 310 on the at least one image as a region of interest. The user can select an area in which the object 310 is displayed by touching a region in which the object 310 is displayed with a finger or a touch tool and then quickly lifting the finger or the touch tool without moving the finger. The device 100 can distinguish an object displayed on the touched area from the image by a graphic cutting method, a level setting method, or the like. Device 100 can determine object 310 as a region of interest.

As shown in 300-2 of FIG. 3, the device 100 can display an object 320 that is completely different from the object 310 displayed on 300-1 of FIG. 3 by fully highlighting the object 310 in response to user selection. Image processing in which the region of interest is fully highlighted compared to the previously displayed region of interest may be referred to as a halo effect. The outline of the area of interest can be highlighted, or the entire area of interest can be highlighted.

FIG. 4 is a reference diagram for explaining a method of providing a blurring effect to an object, according to an exemplary embodiment. As shown in 400-1 of FIG. 4, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. Device 100 can receive a user input selecting an object 410 on the at least one image. The user can select the object 410 by performing a sweeping motion that moves horizontally a certain distance while touching the area on which the object 410 is displayed, which is the region of interest. Then, as shown in 400-2 of FIG. 4, the device 100 can display the blurred object 420 by reducing the difference between the pixel values in the object 410 in response to the user's selection. The device 100 can vary the degree of blurring depending on the sweep time period or the number of sweeps. For example, as the sweep time period or the number of sweeps increases, the degree of blurring effect may increase.

5 and 6 are reference diagrams for explaining a method of providing a size effect to an object, according to an exemplary embodiment.

As shown in 500-1 of FIG. 5, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. Device 100 can receive a user input selecting an object 510 on the at least one image. For example, the user can select the object 510 by performing an enlargement action of widening the distance between the two fingers while touching the area on which the object 510 is displayed (which is the region of interest) with two fingers. . The device 100 can then display the magnified object 520 in response to selection of the object 510, as shown at 500-2 of FIG. The selected object 510 is enlarged, but the size of the unselected object and background does not change. The magnification can depend on the change in distance between the two fingers.

As shown in 600-1 of FIG. 6, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. Device 100 can receive a user input selecting an object 610 on the at least one image. For example, the user can select an object by performing a pinching action that narrows the distance between the two fingers while touching the area on which the object 610 is displayed (which is the region of interest) with two fingers. 610. The device 100 can then display the reduced size object 620 in response to the selection of the object 610, as shown at 600-2 of FIG. The size of the selected object 610 is reduced, and the size of the unselected object and background does not change. However, the space created between the selected object and other regions due to the size of the selected object can be filled with unselected objects and background using, for example, a mirroring technique. The size reduction rate can depend on two hands Refers to the change in distance between the fingers.

The effect provides a depth that can be adjusted or generated for the region of interest. 7 and 8 are reference diagrams for explaining a method of providing a depth effect to an object, according to an exemplary embodiment. As shown in 700-1 of FIG. 7, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. Device 100 can receive a user input selecting an object 710 on the at least one image. For example, the user can select the object 710 as the region of interest by raising the device 100 while touching the area on which the object 710 is displayed. Then, as shown in 700-2 of FIG. 7, in response to selection of the object 710, the device 100 can display the object 720 having a reduced depth such that the object 720 is displayed before the object 710 (ie, causing the user to feel the object 720 compared Closer to object 710).

As shown in 800-1 of FIG. 8, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. Device 100 can receive a user input selecting an object 810 on the at least one image. For example, the user can select the object 810 as the region of interest by moving the device 100 down while touching the area on which the object 810 is displayed. Then, as shown in 800-2 of FIG. 8, in response to selection of object 810, device 100 can display object 820 with increased depth such that object 820 is displayed after object 810 (ie, causing the user to feel object 820 remote).

The device 100 may determine the type of effect to be provided based on the gesture of the user, but may also determine the type of effect to be provided based on the effect item selected by the user from the provided effect list. FIG. 9 is a diagram for explaining one according to an exemplary embodiment A reference diagram of a method of displaying a list of effects.

As shown in 900-1 of FIG. 9, when the mode of the device 100 is set to the effect mode, the device 100 can display an image. The user can select a partial area of the object 910 on which the image is displayed. The device 100 can then determine the object 910 as the region of interest and display an effect list 920 regarding the effects applicable to the region of interest, as shown at 900-2 of FIG.

The effect list 920 can be displayed in the form of a pop-up window, and the effect items included in the effect list 920 can be displayed in the form of a text. For example, the effect item may include: a halo effect that highlights a region of interest, a blur effect that reduces a difference between pixel values of the region of interest, a size effect that adjusts a size of the region of interest, and a sense of adjustment The depth effect of the depth of the area of interest. The user can select one of the effect items, and the device 100 can provide an effect to the region of interest in response to user input.

So far, a method of selecting an object and providing an effect to the selected object has been described. However, device 100 can provide the same effect to multiple objects or provide different effects to at least two of the plurality of objects.

FIG. 10 is a reference diagram for explaining a method of providing an effect to a plurality of objects in an image, according to an exemplary embodiment. As shown in 1000-1 of FIG. 10, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. The device 100 can receive a user input that selects the first object 1010 on the at least one image as the region of interest. For example, the user can touch a portion of the at least one image on which the first object 1010 is displayed. Area. The device 100 can then determine that a user input having selected the first object 1010 as the region of interest has been received and can display the first object 1012 having the halo effect, as shown at 1000-2 of FIG. The device 100 can receive a user input that selects the second object 1020 on the at least one image as the region of interest. For example, by performing an enlargement action of widening the distance between the two fingers while touching the area on which the second object 1020 is displayed with two fingers, the user can input for selecting the second object. 1020 command. The device 100 can then zoom in on the second object 1020 and display the enlarged second object 1022 in response to selection of the second object 1020, as shown in 1000-3 of FIG.

The effect provides not only the objects that can be applied to the image, but also the background in the image. FIG. 11 is a reference diagram for explaining a method of providing an effect to a background, according to an exemplary embodiment. The device 100 can display at least one image while a particular application (eg, a photo album application) is being executed. As shown at 1100-1 of Figure 11, device 100 can receive a user input that selects background 1110 on the at least one image as the region of interest. The user can select the background 1110 by touching and sweeping a portion of the at least one image on which the background 1110 is displayed. Then, as shown at 1100-2 of FIG. 11, the apparatus 100 may provide a blur effect by reducing the difference between the pixel values in the background 1110 in response to the selection of the background 1110, and may display the blurred background 1120. In addition to the blurring effect, other types of effects that enable the background to be displayed completely differently than the previous background can be applied.

Provides effects to both the background and the object. FIG. 12A is a reference for explaining a method of providing an effect to both an object and a background, according to an exemplary embodiment. Figure. Referring to 1200-1 of FIG. 12A, the device 100 can display at least one image while a specific application (eg, a photo album application) is being executed. The device 100 can receive a user input that selects the first object 1210 on the at least one image as the region of interest. For example, the user can select the first object 1210 by touching a partial region of the at least one image on which the first object 1210 is displayed. Device 100 may then provide a halo effect to first object 1210 in response to selection of first object 1210. The halo effect is an effect that highlights the outline of the object selected by the user.

As shown at 1200-2 of Figure 12A, device 100 can display an object 1212 having a halo effect. The device 100 can receive a user input that selects the background 1220 on the at least one image as the region of interest. For example, the user can select the background 1220 by touching and then sweeping a portion of the at least one image on which the background 1220 is displayed. Then, as shown at 1200-2 of FIG. 12A, device 100 may display a blurred background 1222 by reducing the difference between pixel values in background 1220 in response to selection of background 1220.

So far, it has been explained that a preset effect is provided when a region of interest is selected in response to user input. However, the exemplary embodiments are not limited thereto. The user input selecting the region of interest and the user input for providing the effect can be separated from each other. Multiple user inputs can be received continuously or with a certain time difference. The user input selecting the region of interest and the user input for providing the effect may be identical to each other.

FIG. 12B is a reference diagram for explaining a method of providing an effect in response to a plurality of user inputs, according to an exemplary embodiment. Device 100 can be used in a specific application A program (eg, a photo album application) is executed while displaying at least one image. As shown at 1204-1 of FIG. 12B, device 100 can receive a first user input that selects background 1260 on the at least one image as the region of interest. The user can touch a partial area of the at least one image on which the background 1260 is displayed. Device 100 can then receive the touch as a first user input and determine background 1260 as the region of interest. The device 100 can separate the object from the background by detecting the contour of the object from the image. The device 100 can determine whether the area touched is the area on which the object is displayed or the area on which the background is displayed. Since the area on which the background is displayed has been touched in 1200-4 of FIG. 12B, the device 100 can determine the background 1260 as the region of interest.

As shown at 1200-5 of Figure 12B, device 100 can provide an indicator 1270 that highlights the boundaries of background 1260. The user can determine whether the region of interest has been properly selected by looking at the indicator 1270. The device 100 can selectively display the indicator 1270 according to the settings of the user. Device 100 can receive a second user input that provides an effect to the background. For example, the user can drag the area on which the background is displayed in a specific direction. The first user input and the second user input can be continuously received. For example, the user can touch (first user input) background 1260 and then immediately drag (second user input) background 1260.

Then, as shown at 1200-6 of FIG. 12B, the device 100 can receive the drag as a second user input, providing a flow effect that causes the background 1260 to flow in the drag direction and display the flowing background 1262. The flow effect is an effect of flowing the image and corresponds to a correction of the pixel value of the pixel based on the pixel value of the pixel previously arranged according to the drag direction.

By editing an image, other images can be provided with the same effect. FIG. 13 illustrates a graphical user interface for providing effects to multiple images, in accordance with an exemplary embodiment. Referring to 1300-1 of Figure 13, device 100 has provided an effect to an object of the image in response to user input. Device 100 may display an inquiry window 1310 asking if it will provide the same effect to other images.

Device 100 can receive user input requesting the same effect to be applied to other images. For example, the user can touch the area of the inquiry window 1310 on which the "Yes" is displayed. Then, in response to user input, device 100 can display a list 1320 of images that can be applied, as shown at 1300-2 of FIG. When the user selects a particular image from the list 1320, the device 100 can provide the same effect to the selected image.

For ease of explanation, the image that is reviewed to determine whether an image can be rendered effective is now referred to as a target image. The image used to select the region of interest is referred to as the first image, and the image in the target image that is provided with the effect or the image used for the effect is referred to as the second image.

The device 100 can search for or acquire a second image from the target image to provide an effect to the second image. The device 100 can search for the second image by using the identification information of the region of interest (ie, the object or the background) for identifying the first image.

"Identification Information" means keywords, key phrases, etc. used to identify images, and identification information can be defined for each object and each background. The object and the background may each have at least one piece of identification information. According to an exemplary embodiment, the identification information may be obtained by using the attribute information of the image or the image analysis information of the image.

FIG. 14 is a flowchart of a method for the device 100 to provide an effect to a second image using identification information of the first image, according to an exemplary embodiment.

In operation S1410, the device 100 may select a region of interest from the first image. For example, as described above, the device 100 can display the first image and select an object or background in the first image as the region of interest in response to user input. The device 100 can provide an effect to the region of interest of the first image, or can provide an effect to the region of interest of the first image and then provide an effect on the second image. The first image may be a still image, a moving picture frame that is part of the moving picture (ie, a still image of the moving picture), or a live view image. When the first image is a still image or a moving picture frame of a moving picture, the still image or the moving picture may be an image stored in the device 100 in advance, or may be an image stored in the external device and transmitted from the external device. . When the first image is a live view image, the live view image may be an image captured by a camera built in the device 100 or an image captured and transmitted by a camera as an external device.

In operation S1420, the device 100 may determine whether identification information is defined in the selected region of interest. For example, when storing images, identification information describing objects and backgrounds contained in the images may be matched and stored. In such a case, device 100 can determine that identification information is defined in the selected region of interest. According to an exemplary embodiment, the identification information corresponding to the object and the background may be stored in the form of metadata for each image.

In operation S1430, if the identification information is not defined in the selected region of interest, the device 100 may generate the identification information. For example, the device 100 can benefit The identification information is generated by using attribute information stored in the form of metadata or using image analysis information obtained by performing image processing on the image. Operation S1430 will be explained in more detail below with reference to FIG.

In operation S1440, the device 100 may search for a second image with identification information from the target image. The target image may be, for example, a still image or a moving picture stored in the device 100 according to a user input, or a still image or a moving picture stored in the external device. When searching for the second image from the moving picture, the device 100 may search for a moving picture frame with the identification information.

The identification information may be predefined in the target image or may not be predefined in the target image. If the identification information is predefined in the target image, the device 100 may search for the second image based on whether the identification information of the target image is the same as the identification information of the region of interest. If the identification information is not predefined in the target image (as generally done in operation S1430), the device 100 may generate identification information of the target image. The device 100 may search for the second image based on whether the generated identification information of the target image is the same as the identification information of the region of interest.

When there is a plurality of pieces of identification information for the region of interest of the first image, the device 100 may search for the image by using at least some of the plurality of pieces of identification information. Alternatively, the device 100 can provide the user with a list of identification information so that the user can select the identification information. The device 100 can receive at least one piece of identification information from the identification information list. According to an exemplary embodiment, device 100 may receive an input to select all of the identification information or to select an input of certain identification information.

According to an exemplary embodiment, the user input selecting the identification information may have Variety. For example, the user input can be at least one selected from the group consisting of a key input, a touch input, a motion input, a bending input, a voice input, and multiple inputs.

In operation S1450, the device 100 may provide an effect to the found second image. The device 100 can distinguish the partial image and the second image corresponding to the region of interest by using the identification information, and can provide the same effect to the segmented partial image as the effect applied to the region of interest of the first image. In operation S1410, the device 100 may distinguish the region of interest from the first image in response to the user's selection. In operation S1450, the device 100 may use the identification information to distinguish a partial image corresponding to the region of interest from the second image.

15 is a flow diagram of a method in which device 100 generates identification information, in accordance with an exemplary embodiment. Fig. 15 illustrates a case where the identification information of the region of interest in the first image is not defined in advance. The method of generating the identification information shown in FIG. 15 is also applicable to the case where the identification information of the target image is generated.

In operation S1510, the device 100 may determine whether there is attribute information corresponding to the region of interest. For example, device 100 can examine metadata corresponding to a region of interest. The device 100 may extract attribute information of the region of interest from the metadata.

According to an exemplary embodiment, the attribute information represents an attribute of the image, and may include context information associated with the image generation and annotation information added by the user.

Contextual information is environmental information associated with an image during image generation. For example, the context information may include at least one of the following information: information about the format of the image, information about the size of the image, and information about the image used to generate the image The information of the device 100, the time information generated by the image, the temperature information generated by the image, and the source information of the image. Device 100 can automatically retrieve and store contextual information.

Annotation information is information recorded by a user and may include information about objects contained in the image (eg, type, name, and state of the object) and information about the background contained in the image (eg, location information, Time information, and weather information).

In operation S1520 and operation S1540, the device 100 may summarize attribute information of the image and generate identification information.

Inductive attribute information can mean that attribute information is expressed based on the upper layer language of WordNet (hierarchical term reference system).

"WordNet" is a database that provides definitions or usage patterns of words and establishes relationships between words. The basic structure of WordNet includes a logical group called a synonym set with a series of semantically equivalent words, and a semantic relationship between the synonym sets. The semantic relationship includes an upper meaning word, a lower meaning word, a partial word (meronym), and a whole word (holonym). The nouns contained in WordNet have the entity as the top-level word and form a sub-word by extending the entity according to the meaning. Therefore, WordNet can also be called ontology with a hierarchical structure by classifying and defining conceptual vocabulary.

"Ontology" represents a formal and unambiguous specification of shared conceptualization. Ontology can be thought of as a dictionary of words and relationships. In ontology, words associated with a particular domain are expressed hierarchically, and inference rules for extended words are included.

For example, when the region of interest is the background, the device 100 may classify the location information included in the attribute information into upper layer information and generate identification information. For example, the device 100 can express a global positioning system (GPS) coordinate value (latitude: 37.4872222, longitude: 127.0530792) as a superior concept, such as a zone, a building, a address, a zone name, a city name, or a country name. . In this case, the building, the area name, the city name, the country name, and the like can be generated as identification information for the background.

In operation S1530 and S1540, if there is no attribute information corresponding to the region of interest, the device 100 may obtain image analysis information of the region of interest and use the image analysis information to generate identification information of the region of interest.

According to an exemplary embodiment, the image analysis information is information corresponding to an analysis result of data obtained through image processing. For example, the image analysis information may include information about an object displayed on the image (eg, type, state, and name of the object), information about a position displayed on the image, and a season or time displayed on the image. Information, as well as information about the atmosphere or mood displayed on the image, but the exemplary embodiments are not limited thereto.

For example, when the region of interest is an object, the device 100 can detect the contour of the object contained in the image. According to an exemplary embodiment, the device 100 may compare an outline of an object included in an image with a predetermined template, and obtain a type, a name, and the like of the object. For example, when the outline of the object is similar to the vehicle template, the device 100 can identify the object contained in the image as the vehicle. In this case, the device 100 can display information using information about objects contained in the image. Know the information "car".

Alternatively, device 100 may perform facial recognition of objects contained in the image. For example, the device 100 can detect a person's facial area from the image. Examples of the face area detecting method may include a knowledge based method, a feature based method, a template matching method, and an appearance based method, but the exemplary embodiments are not limited thereto.

The device 100 can extract facial features (eg, the shape of the eyes, nose, and mouth that are the major portions of the face) from the detected facial regions. To extract facial features from the face region, a Jaber filter, a local binary mode (LBP), or the like can be used, but the exemplary embodiments are not limited thereto.

The device 100 can compare the facial features extracted from the facial regions in the image with the facial features of the pre-registered user. For example, when the extracted facial feature is similar to the facial feature of the first registered registrant, the device 100 may determine that the first user is included as a partial image in the selected image. In this case, the device 100 may generate the identification information "first user" based on the result of the face recognition.

According to an exemplary embodiment, the device 100 may compare a certain region of the image with a color map (color histogram) and extract visual features (eg, color arrangement, pattern, and atmosphere of the image) as image analysis information. The device 100 can utilize the visual features of the image to generate identification information. For example, when the image includes a sky background, the device 100 can utilize the visual features of the sky background to generate the identification information "sky."

According to an exemplary embodiment, the device 100 may divide an image into units, Search for the cluster most similar to each zone and generate identification information associated with the found cluster.

If there is no attribute information corresponding to the image, the device 100 can obtain the image analysis information of the image, and use the image analysis information to generate the image identification information.

FIG. 15 illustrates an exemplary embodiment in which the device 100 acquires image analysis information of an image when there is no attribute information of the image, but the exemplary embodiment is not limited thereto.

For example, the device 100 may generate identification information using only image analysis information or attribute information. Alternatively, the device 100 may obtain image analysis information even when attribute information is present. In this case, the device 100 can use both the attribute information and the image analysis information to generate the identification information.

According to an exemplary embodiment, the device 100 may compare the identification information generated based on the attribute information with the identification information generated based on the image analysis information, and determine the common identification information as the final identification information. Compared with non-common identification information, the common identification information can have higher reliability. Reliability indicates that the identification information extracted from the image is trusted to the extent that the information is properly identified.

FIG. 16 illustrates attribute information of an image according to an exemplary embodiment. As shown in FIG. 16, the attribute information of the image can be stored in the form of metadata. For example, for each image, for example, type 1610, time 1611, GPS 1612, resolution 1613, size 1614, and collection device 1617 may be stored as attribute information.

Context information used during image generation, according to an exemplary embodiment It can also be stored in the form of metadata. For example, when the device 100 generates the first image 1601, the device 100 may collect weather information (eg, cloudy), temperature information (eg, 20 ° C), and the like from the weather application at the time the first image 1601 is generated. The device 100 can store the weather information 1615 and the temperature information 1616 as attribute information of the first image 1601. The device 100 can collect event information (not shown) from the schedule application at the moment the first image 1601 is generated. In this case, the device 100 may store the event information as attribute information of the first image 1601.

According to an exemplary embodiment, the user added information 1618 input by the user may also be stored in the form of metadata. For example, the user added information 1618 may include annotation information input by the user to interpret the image, and information about the object interpreted by the user.

According to an exemplary embodiment, image analysis information (for example, object information 1619) obtained as a result of image processing on an image may be stored in the form of metadata. For example, the device 100 may store information about objects (eg, user 1, user 2, me, and chair) included in the first image 1601 as attribute information of the first image 1601.

17 is a reference diagram for explaining an example in which the device 100 generates identification information of an image based on attribute information of an image.

According to an exemplary embodiment, device 100 may select background 1712 of image 1710 as the region of interest based on user input. In this case, the device 100 can view the attribute information of the selected background 1712 in the attribute information 1720 of the image 1710. The device 100 can detect the identification by using the attribute information of the selected background 1712. Information 1730.

For example, when the area selected as the region of interest is the background, the device 100 may detect the information associated with the background from the attribute information 1720. The device 100 can generate the identification information "park" by using the location information (for example, latitude: 37; 25; 26.928..., longitude: 126; 35; 31.235...) in the attribute information 1720, or use the attribute information 1720 Weather information (eg, the cloud) generates identification information that is "cloudy." The device 100 can combine a plurality of attribute information to generate new identification information. For example, when the time information in the attribute information 1720 is 2012.5.3.15:13 and the position information therein is latitude: 37; 25; 26.928... and longitude: 126; 35; 31.235... The location information is used to determine the area displayed on the image 1710, and the time displayed on the image 1710 can be determined using time information in addition to the location information. For example, when the location information is "Korea", the device 100 can use the time information to generate identification information "spring" about the season. As another example, the device 100 may use the identification information about the season generated based on the location information and the time information, and the weather information to generate the identification information "spring rainy day".

Alternatively, the device 100 may generate identification information "smile" and "user 1" corresponding to the object information from the annotation information added by the user.

When the context information and the annotation information contradict each other, the device 100 can generate the identification information by using the image analysis information. For example, when the weather information included in the context information is rainy and the weather information contained in the annotation information is cloudy, the device 100 can use the image analysis information to determine whether the weather information is rainy or cloudy. However, the exemplary embodiments are not limited thereto. When context information and annotation information contradict each other The device 100 may prioritize the annotation information and generate the identification information using the annotation information.

FIG. 18 is a reference diagram for explaining an example in which the device 100 generates identification information using image analysis information. According to an exemplary embodiment, device 100 may select first object 1812 of image 1810 as a region of interest based on user input. In this case, the device 100 can generate identification information (eg, a person and a smile) that illustrates the first object 1812 by performing image analysis on the first object 1812.

For example, device 100 can detect a facial region from a region of interest. The device 100 can extract facial features from the detected facial regions. The device 100 can compare the extracted facial features with pre-registered facial features of the user and generate identification information representative of the selected first object 1812 being the user 1. The device 100 can also generate the identification information "smile" based on the shape of the lips included in the detected face region. Then, the device 100 can obtain "user 1" and "smile" from the identification information 1820.

When there is a plurality of pieces of identification information of the region of interest, the device 100 may display the identification information list to enable the user to select the identification information. FIG. 19 illustrates an example in which device 100 displays a list of identification information, in accordance with an exemplary embodiment.

Referring to 1900-1 of FIG. 19, device 100 may select first object 1912 of first image 1910 as a region of interest based on user input. According to an exemplary embodiment, device 100 may obtain identification information that illustrates first object 1912. For example, device 100 may obtain identification information such as a smile, a mother, and a wink.

Referring to 1900-2 of FIG. 19, the device 100 can display the identification information list 1920 of the acquired identification information. In this case, the device 100 can receive self-identification The information list 1920 selects at least some user input for identifying information. For example, device 100 can receive a user input selecting a mother 1922. The device 100 may search for a second image having identification information (eg, a mother) selected by the user from the target image (eg, a photo album), provide an effect to a portion of the image corresponding to the mother in the second image, and then display the corresponding image. A second image 1930 of the application effect of the partial image of the mother is shown in 1900-3 of FIG.

When there are a plurality of second images to which the effect is applied, the device 100 may generate a data folder (hereinafter referred to as an effect folder), and store the second image (hereinafter referred to as an effect image) to which the effect is applied in the effect data. In the folder. Each effect image may include at least one selected from the group consisting of a first image and a second image. Although the device 100 can store the effect image in the effect folder, the device 100 can also store the link information of the effect image in the effect folder.

FIG. 20 illustrates an example in which the device 100 displays an effect folder. When the searching for the second image is completed, the device 100 can provide an effect to the second image. As shown in 2000-1 of FIG. 20, the device 2010 can display the effect folder 2010. The effect folder 2010 can store effect images. In other exemplary embodiments, the link information of the effect image may be stored in the effect folder 2010.

The user can enter a command for selecting the effect folder 2010. In response to user input, device 100 can display at least one effect image 2020, as shown in 2000-2 of FIG.

According to an exemplary embodiment, the device 100 may be based on selected from image generation time information, image generation location information, image capacity information, and image resolution. At least one effect image included in the effect folder 2010 is arranged in at least one of the messages.

When there are different types of effect images, the device 100 may generate an effect folder according to the type of the effect image, and store the same type of effect image in a single effect folder. The device 100 can select a new region of interest on the image contained in the effect folder and provide a new effect to the selected region of interest. When the effect folder contains many effect images to which a new effect has been applied, a new effect folder can be created in the effect folder.

As described above, it is possible to provide effects not only to images stored in the device 100 but also to images stored in the external devices. The external device can be, for example, a social network service (SNS) server, a cloud server or a remote server, or a device 100 used by another user. 21 is a flowchart of a method in which a device provides an effect to an image stored in an external device, according to an exemplary embodiment.

In operation S2110, the external device 200 may store at least one image. One of the images stored in the external device 200 may be the first image. The external device 200 may be a server that provides an SNS to the device 100 connected to the external device via a network, may be a portable terminal connected to the device 100 via a network, or may be a cloud server. SNS represents a service that enables online users to create new relationships or enhance existing relationships.

According to an exemplary embodiment, the external device 200 may store images uploaded from the devices 100 of several users.

The device 100 may be connected to the external device 200 in operation S2120. Loading The setting 100 can be connected to the external device 200 by performing login. The login may be a program that acquires access rights to images stored in the external device 200. For example, the device 100 may request the external device 200 to perform user authorization, and transmit the user's identification information (eg, email account information) and the user's authentication information (eg, password) to the external device 200. When the user is identified as an authorized user, device 100 can be allowed to connect to an external device and access images stored in the external device.

In operation S2130, the device 100 may receive the first image in the image stored in the external device 200. The device 100 may request one of the images stored in the external device 200 as the first image. In response to this request, the external device 200 can transmit the first image to the device 100. The first image may include an object and a background. The first image may be a still image, a moving picture frame, or a live view image.

In operation S2140, the device 100 may select a region of interest from the first image. For example, the device 100 can receive a user input from the first image selection portion region, detect an outline around the selected partial region, and select the partial region surrounded by the contour as the region of interest. . The area surrounded by the outline can be an object or a background.

According to an exemplary embodiment, the user input selecting the region of interest may vary. The user input can be, for example, a key input, a touch input, a motion input, a curved input, a voice input, or multiple inputs. For example, the device 100 can receive the specific content of the plurality of images stored in the external device 200 for a predetermined time period (for example, two seconds) or more, or touch the specific content for a predetermined number of times or more (for example) For example, double-click on the input.

In operation S2150, the device 100 may query the external device 20 for identifying information identifying the selected region of interest. In operation S2160, the device 100 may receive a response regarding the identification information from the external device 200. The device 100 can ask whether identification information about an object or background corresponding to the region of interest is predefined. When the identification information is predefined, the external device 200 may transmit identification information about the region of interest to the device 100.

In some exemplary embodiments, when the identification information about the object or the background is not predefined in the external device 200, the external device 200 may determine whether the external device 200 can generate the identification information. If it is determined that the external device 200 can generate the identification information, the external device 200 can generate identification information about the region of interest and transmit it to the device 100. When the external device 200 generates the identification information, the external device 200 may utilize at least one of the attribute information of the region of interest and the image analysis information. On the other hand, if it is determined that the external device 200 cannot generate the identification information, the external device 200 may transmit only the information about the region of interest that the external device 200 has to the device 100. In some exemplary embodiments, the external device 200 may transmit a response indicating that the external device 200 is unable to generate the identification information to the device 100.

In operation S2170, the device 100 may obtain identification information of the region of interest based on the response of the external device 200. The device 100 may receive the identification information of the region of interest from the external device 200, or may generate the identification information of the region of interest by using at least one of the attribute information and the image analysis information selected from the region of interest.

In operation S2180, the device 100 may search for a second with identification information. image. The device 100 can search for a second image with identification information from the target image. The target image may be an image stored in the device 100. Alternatively, the target image may be an image stored in the external device 200. Alternatively, the target image may be an image stored in an external device different from the external device 200 shown in FIG. When searching for the second image, the device 100 can utilize the identification information stored for the target image. When the identification information is not defined in advance, the device 100 may generate the identification information of the target image by using the attribute information or the image analysis information of the target image, and then search for the second image with the common identification information.

When there is a plurality of pieces of identification information, the device 100 may search for the second image by using at least some of the pieces of identification information according to user input.

In operation S2190, the device 100 may provide an effect to the found second image.

Although the region of interest of the first image stored in the external device is provided to the second image stored in the device 100 or the external device 200 in FIG. 21, the exemplary embodiment is not limited thereto. The region of interest of the first image stored in device 100 can be utilized to provide an effect to the second image stored in external device 200.

The device 100 can share an effect image with the external device 200. 22 is a flowchart of a method in which a device shares an effect image with an external device, according to an exemplary embodiment.

In operation S2210, the device 100 may display an effect image. For example, The device 100 can generate an effect image by providing an effect to the region of interest selected by the user on the first image. The device 100 can display an effect image. The device 100 can generate an effect image by providing an effect to the second image using the region of interest of the first image.

In operation S2220, the device 100 may receive a user input requesting sharing of the effect images.

According to an exemplary embodiment, the user input requesting sharing of the effect images may vary. For example, the user input can be a key input, a voice input, a touch input, or a curved input, although the exemplary embodiments are not limited thereto.

According to an exemplary embodiment, the device 100 may receive information about the external device 200 that will share the effect image via user input. The external device 200 may be at least one selected from the group consisting of a cloud server connected to the device 100, an SNS server, another device of the user, another user's device, and a wearable device, but the exemplary embodiment is not Limited to this.

For example, the user can input the account information of the cloud storage, the user's SNS account information, and the identification information of the friend device for transmitting all the images included in the first folder (for example, a phone number or a MAC address). ), and friends' email account information.

In operation S2230, the device 100 may share an effect image with an external device.

For example, the device 100 can transmit link information (eg, storage location information or URL) of the effect image to the external device 200. The device 100 can transmit the effect image to the external device 200. According to an exemplary embodiment, device 100 may effect The image is uploaded to a specific server, and the external device 200 is authorized to access the specific server.

Although the device 100 shares the effect image with the external device 200 in FIG. 22, the exemplary embodiment is not limited thereto. The device 100 can share an effect folder with the external device 200. The device 100 can share at least one effect folder with the external device 200.

Figure 23 illustrates an example in which a device shares an effect image with an external device. Referring to 2300-1 of FIG. 23, device 100 can generate and display effect folder 2310 in response to user input. The effect folder 2310 can store at least one effect image. The effect folder 2310 can store link information of the effect image or the effect image.

In this case, device 100 can receive user input to select effect folder 2310. For example, device 100 can receive touch effect folder 2310 for input for a predetermined period of time (eg, two seconds). The device 100 can provide a menu window 2320 including various items such as a folder search, a bookmark addition, and a transmission item 2322 in response to user input.

When the user selects the transfer item 2322 on the menu window 2320, the device 100 can provide a selection window 2330 via which the receiving device can be selected, as shown at 2300-2 of FIG. Device 100 can receive user input to contact 2332 on selection selection window 2330. The user can select a specific friend from the contact person. In this case, the device 100 can share the effect folder 2310 with the device 100 of the particular friend.

For example, the device 100 can include the effect clip 2310 The effect image is transmitted to the device 100 of the specific friend. In other exemplary embodiments, the device 100 may transmit link information of the effect image included in the effect folder 2310 to the device 100 of the specific friend.

According to an exemplary embodiment, the device 100 may transmit the effect image (or link information of the effect image) included in the effect folder 2310 to the device 100 of the specific friend via an email or a body message.

FIG. 24 is a schematic diagram of an image management system, according to an exemplary embodiment.

As shown in FIG. 24, the image management system may include the device 100 and the cloud server 210. In some exemplary embodiments, a cloud server may refer to a remote server. The image management system can be constructed from more or less components than those illustrated in FIG.

The apparatus 100 according to an exemplary embodiment may be achieved in various types. For example, the device 100 can be a desktop computer, a mobile phone, a smart phone, a laptop, a tablet personal computer (PC), an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), and a portable device. Multimedia Player (PMP), Navigator, MP3 Player, Digital Camera 160, Internet TV (IPTV), Digital TV (DTV), Consumer Electronics (CE) devices (eg, refrigerators and air conditioners each including a display) At least one of the machines and the like, but the exemplary embodiments are not limited thereto. Device 100 can also be a device that can be worn by a user. For example, device 100 can be at least one selected from the group consisting of a watch, glasses, a ring, a bracelet, and a necklace.

Since the device 100 is identical to the device 100 described above, it will not be described again here. For ease of explanation, it will now be explained that the device 100 is the first device, The case of one of the second device to the Nth device.

The cloud server 210 can be connected to the device 100 and thus communicate with the device 100. For example, the cloud server 210 can be connected to the device 100 via account information.

According to an exemplary embodiment, the cloud server 210 may transmit data to or receive data from the device 100. For example, the device 100 can upload at least one image to the cloud server 210. The device 100 can receive attribute information, image analysis information, identification information, and the like about the image from the cloud server 200.

The cloud server 210 can include a smart engine and can analyze images controlled by the device 100 via the smart engine. For example, the cloud server 210 can generate identification information from the attribute information of the image, and obtain image analysis information by performing image processing on the image. The cloud server 210 can analyze the event information generated by the device 100 and infer the state of the user, the condition of the device 100, and the like. The cloud server 210, like the device 100 described above, can generate at least one selected from the group consisting of an effect image and an effect folder in response to user input.

The effect image has been described above as an image obtained by providing an effect to an object or background of a single image. Image data from the region of interest has been used to provide results. In other words, the halo effect, the blur effect, and the like adjust the pixel values of the region of interest, and the size effect applies the pixel values of the region of interest to a relatively wide region or a relatively narrow region. The depth effect uses the pixel values of the region of interest to produce three-dimensional (3D) images (eg, left eye images and right eye images).

Image data of another image may be utilized during the performance of the effect. In other words, the device 100 can provide an effect to the first image by using the image data of the second image. Now, providing an effect of using image data of another image to a region of interest of an image is said to provide an effect by combining a plurality of images with each other. FIG. 25 is a flowchart of a method of providing an effect image by combining a plurality of images with each other, according to an exemplary embodiment.

In operation S2510, the device 100 may select a region of interest from the first image. For example, as described above, the device 100 can display a still image or a moving picture frame stored in the device 100 or an external device as the first image. Alternatively, the device 100 may display the live view image captured by the device 100 or an external device as the first image. In response to user input, device 100 may select an object or background of the first image as the region of interest.

In operation S2520, the device 100 may obtain identification information of the selected region of interest. For example, when storing the first image, the identification information of the object and the background respectively included in the first image may be matched and stored. According to an exemplary embodiment, the identification information corresponding to the object and the background, respectively, may be stored in the form of metadata. In such a case, device 100 may determine that identification information has been defined for the selected region of interest. The device 100 can obtain the identification information by reading the pre-stored identification information.

When the identification information of the selected region of interest is not defined, the device 100 can obtain the identification information by generating the identification information. For example, the device 100 may generate the identification information based on the attribute information of the first image stored in the form of metadata or the image analysis information obtained by performing image processing on the first image.

In operation S2530, the device 100 may search for a second image having the same identification information as the region of interest from the target image. The target image may be at least one image in which the second image is to be searched, and may be a still image or a moving picture stored in the device 100 or an external device.

When the region of interest has an identification information, the device 100 may search for the second image with the identification information from the target image. On the other hand, when the region of interest has a plurality of pieces of identification information, the device 100 may search for an image having the owner of the plurality of pieces of identification information. However, the exemplary embodiments are not limited thereto. The device 100 may search for an image having some of the plurality of pieces of identification information. The device 100 can provide the user with a list of identification information and receive user input for selecting at least one piece of identification information from the list of identification information. According to an exemplary embodiment, the device 100 may receive user input selecting all of the identification information or user input selecting certain identification information.

According to an exemplary embodiment, the user input selecting the identification information may vary. For example, the user input can be at least one selected from the group consisting of a key input, a touch input, a motion input, a bending input, a voice input, and a multiple input.

In operation S2540, the device 100 may select one of the at least one of the found second images. The device 100 can display the found at least one second image. The second image may be displayed in the form of a thumbnail. When a plurality of second images are found, the device 100 may display the order or display the second images in the order in which the second images were generated. The order in which the second images are displayed may be set according to user input settings. The device 100 can respond to user input when at least one second image is displayed And selecting one of the at least one of the found second images. When a second image is found, device 100 can select the second image found regardless of user input.

In operation S2550, the device 100 generates an effect image by combining the first image and the second image. The device 100 can generate an effect image by separating the region of interest from the first image and the second image and combining a partial image of the second image corresponding to the region of interest with a region of the region of interest of the first image. . Alternatively, the device 100 may generate an effect image by replacing the image data of the region of interest with the image data of the partial image corresponding to the region of interest in the second image.

26A-26C illustrate an example of providing an effect to an object using a plurality of images, according to an exemplary embodiment. As shown in 2600-1 of FIG. 26A, when the mode of the device 100 is set to the effect mode, the device 100 can display the first image 2610. For example, the first image 2610 can be a still image, a moving picture frame, or a live view image. Device 100 can select a region of interest from first image 2610. For example, the user can touch a partial region of the first image 2610 on which the object 2612 is displayed, and the device 100 can select the object 2612 including the touched region and obtained by performing image processing based on the touched region as Area of interest.

The device 100 can display a list of effects 2620 including effects that can be provided to the object, as shown at 2600-2 of Figure 26A. The effect list 2620 can overlap the displayed first image 2610. The user can perform an input of an effect from the effect list 2620. For example, the user can execute by touching the item "Use another image" Line input.

The device 100 can obtain identification information for identifying an object 2612 as a region of interest. When the identification information is stored in advance, the device 100 can obtain the identification information by reading the pre-stored identification information. When the identification information is not stored in advance, the device 100 may obtain the identification information by generating the identification information by using at least one of the attribute information selected from the object 2612 and the image analysis information. As shown at 2600-3 of Figure 26B, device 100 can display an identification information list 2630. The identification information list 2630 can also overlap the displayed first image 2610. By selecting at least some user input of the identification information from the identification information list 2630, the device 100 may determine the at least some identification information as identification information for searching.

As shown at 2600-4 of Figure 26B, device 100 can display a list of target images 2640 that represent information about the target image. By selecting user input of at least one image from the target image list 2640, the device 100 can determine the target image.

The device 100 can search for a second image having the selected identification information from the target image. When the target image is a still image, the device 100 may search for the second image in units of still images. When the target image is a moving picture, the device 100 may search for the second image by moving the picture frame as a unit.

The found second image 2650 can be displayed as shown at 2600-5 of Figure 26C. The first image 2610 and the second image 2650 can be displayed on separate regions, respectively. When a plurality of second images 2650 are found, the device 100 may sequentially arrange the plurality of second images 2650 in a search order or the like.

Selecting the second image 2660 from the plurality of second images 2650 Upon user input, the device 100 can display an effect image 2670 obtained by combining the first image 2610 with the selected second image 2660, as shown at 2600-6 of FIG. 26C. The device 100 can generate the effect image 2670 by replacing the selected object 2612 of the first image 2610 with the object 2622 of the selected second image 2660.

The size and shape of the object 2662 of the second image 2660 can be different than the size and shape of the object 2612 of the first image 2610. The device 100 can use a restoration technique when combining the first image 2612 with the second image 2660. FIG. 27 is a reference diagram for explaining a method of combining a plurality of images, according to an exemplary embodiment. The device 100 can obtain an image 2714 (hereinafter, referred to as a first partial image) by excluding an object 2712 as a region of interest from the first image 2710. The device 100 can separate the first portion of image 2714 from the first image 2710 using, for example, image edge characteristics. The device 100 can also separate an object 2722 (hereinafter referred to as a second partial image) corresponding to the region of interest from the second image 2720 by, for example, image edge characteristics.

The device 100 combines the first partial image 2714 with the second partial image 2722 such that the first partial image 2714 and the second partial image 2722 overlap each other to a minimum. The device 100 can delete a portion of the first partial image 2714 by using a region 2732 from which the first partial image 2714 and the second partial image 2722 overlap each other and restore the first partial image 2714 and the second partial image 2722 by using the first partial image 2714. The effect image 2730 is not generated in the area 2734 shown above.

28A through 28C illustrate an example of providing effects to a background using a plurality of images, according to an exemplary embodiment. As shown in 2800-1 of Figure 28A, at device 100 When the mode is set to the effect mode, the device 100 can display the first image 2810. For example, the first image 2810 can be a still image, a moving picture frame, or a live view image. Device 100 can select a region of interest from first image 2810. For example, the user can perform an input of touching a partial region of the first image 2810 on which the background 2814 is displayed, and the device 100 can select the background 2814 by performing image processing on the touched region in response to user input. Area of interest.

The device 100 can display an effect list 2820 that includes effects that can be provided to the background 2814, as shown at 2820-2 of Figure 28A. The effect list 2820 can overlap the displayed first image 2810. The user can perform an input of an effect from the effect list 2820. For example, the user can perform an input by touching the item "Use another image."

The device 100 can obtain identification information for identifying the background 2814 as the region of interest. When the identification information is stored in advance, the device 100 can obtain the identification information by reading the pre-stored identification information. When the identification information is not stored in advance, the device 100 may obtain the identification information by generating the identification information by using at least one of the attribute information and the image analysis information selected from the background 2814. As shown at 2800-3 of Figure 28B, device 100 can display an identification information list 2830. The identification information list 2830 can also overlap the displayed first image 2810. By selecting at least some user input of the identification information from the identification information list 2830, the device 100 may determine the at least some identification information as identification information for searching.

As shown at 2800-4 of Figure 28B, device 100 can display a list of target images 2840 that represent information about the target image. By the target image list 2840 The user input of at least one image is selected and the device 100 can determine the target image.

The device 100 can search for a second image having identification information for searching from the target image. When the target image is a still image, the device 100 may search for the second image in units of still images. When the target image is a moving picture, the device 100 may search for the second image by moving the picture frame as a unit.

The found second image 2850 can be displayed as shown at 2800-5 of Figure 28C. The first image 2810 and the second image 2850 can be displayed on separate regions, respectively. When a plurality of second images 2850 are found, the device 100 may sequentially arrange the plurality of second images 2850 in a search order or the like.

In response to the user input selecting the second image 2860 from the plurality of second images 2850, the device 100 can display the effect image 2860 obtained by combining the first image 2810 with the selected second image 2860, such as 2800-6 of Figure 28C. The device 100 can generate an effect image 2860 by combining the background 2864 of the second image 2840 with the region of interest of the first image 2810.

The size and shape of the background 2864 of the second image 2860 may be slightly different than the size and shape of the background 2814 of the first image 2810. The device 100 can use a restoration technique when combining the first image 2810 with the second image 2850. FIG. 29 is a reference diagram for explaining a method of combining a plurality of images, according to another exemplary embodiment. The device 100 can obtain an image 2912 (hereinafter, referred to as a third partial image) by excluding a background as a region of interest from the first image 2910. The device 100 can separate the third portion of the image 2912 from the first image 2910 using, for example, image edge characteristics. The device 100 can also utilize portions such as image edge characteristics to correspond to regions of interest. The image 2924 (hereinafter referred to as a fourth partial image) is separated from the second image 2920.

The device 100 may generate the background image 2930 by filling the region 2932 of the fourth partial image 2924 in which the pixel information is absent with a predetermined pixel value. In generating the background image 2930, the device 100 can utilize the mirroring technique to utilize the pixel values of the regions surrounding the region 2932 to determine the pixel values of the region 2932 that does not have pixel information. The device 100 can generate an effect image 2940 by combining the third partial image 2912 with the background image 2930. The device 100 can combine the third partial image 2912 with the background image 2930 by using the position information of the third partial image 2912 in the first image 2910. The portion of the background image 2930 that overlaps the third partial image 2912 can be deleted.

The effect image 2940 can be provided as a live view image. A live view image may refer to an image captured by a camera and displayed on the device, and is an image before the storage command is received. The camera may be a camera built into the device 100 or may be an external device. As described above, the device 100 can select an object or background of the live view image as the region of interest in response to user input, and display the selected region of interest completely differently from the region of interest that has not been selected. For example, device 100 can provide a halo effect, a blur effect, a size effect, or a depth effect to a region of interest. When the device 100 receives the save command, the device 100 can store the live view image that has been rendered effective. The device 100 can store the live view image that has been provided as a still image or a moving picture according to the shooting mode of the device 100.

The device 100 can extract a plurality of images from the live view image and provide an effect image. FIG. 30 is a flowchart of a method of providing an effect image using a live view image, according to an exemplary embodiment. In operation S3010, the device 100 can display Take a picture of the scene. The live view image is an image taken by the camera and displayed, and is an image before the storage command is received. The camera may be a camera built into the device 100 or may be an external device. For ease of explanation, the image produced after receiving the stored input will now be referred to as the captured image.

In operation S3020, the device 100 selects a region of interest. The user can input a command for selecting a partial region on the live view image, and the device 100 can determine an object or background including the selected partial region as the region of interest.

The user input for selecting a partial area may vary. For example, the user input can be at least one selected from the group consisting of a key input, a touch input, a motion input, a bending input, a voice input, and multiple inputs.

In operation S3030 and operation S3040, the device 100 may generate a temporary image from the live view image. The temporary image is an image including a region of interest, and is an image temporarily generated before receiving a user inputting the image.

The temporary image can be a partial image of the region of interest. For example, the temporary image may be a partial image including only an object or background selected as the region of interest. Alternatively, the temporary image may be a frame image including a region of interest in the live view image.

The temporary image may be temporarily generated and stored in the buffer, or may be temporarily generated and displayed on the display area of the device 100.

The temporary image can be generated between the time at which the region of interest is selected and the time at which the stored input is received. For example, a temporary image can be generated at the time of selecting the region of interest, and another temporary image can be generated at the time of receiving the stored input. Like, so a total of two temporary images can be generated. Alternatively, the temporary image may be generated at a predetermined time interval (eg, 3 seconds) after the time at which the region of interest is selected and before the storage input is received. Alternatively, a temporary image may be generated each time the region of interest is selected and before the storage input is received, whenever the change in the region of interest is equal to or greater than the reference value.

The temporary image generated at the time of selecting the region of interest is referred to as an initial temporary image, and the temporary image generated at the time of receiving the storage input is referred to as a final temporary image. In response to storing the input, the device 100 can obtain a plurality of temporary images including the initial temporary image and the final temporary image. Among the plurality of temporary images, the temporary image composed only of the region of interest may be the image of interest. The device 100 can generate an image of interest from one of the plurality of temporary images including an initial temporary image and a final temporary image.

User input for storing images may vary. For example, the user input can be at least one selected from the group consisting of a key input, a touch input, a motion input, a bending input, a voice input, and multiple inputs.

In operation S3050, when the device 100 receives a user input for storage, the device 100 may generate an effect image using the plurality of temporary images. In other words, the device 100 can generate an effect image by reading a plurality of temporary images temporarily stored in the buffer and combining the read plurality of temporary images with each other. The device 100 can store the resulting effect image. Temporary images stored in the buffer can be deleted. The device 100 can generate an effect image by combining an object or background of the initial temporary image as a region of interest with the final temporary image. As another option, the device The effect image may be generated by combining a partial image of the plurality of temporary images corresponding to the region of interest with the final temporary image.

FIG. 31 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to an exemplary embodiment. As shown in 3100-1 of FIG. 31, when the mode of the device 100 is set to the shooting mode, the device 100 can display the live view image 3110. The device 100 can select a region of interest from the live view image 3110. For example, the user can touch a partial region of the live view image 3110 on which the object 3112 is displayed. Device 100 can then select object 3112 as the region of interest. When object 3112 is selected as the region of interest, device 100 can generate an initial temporary image that includes object 3112. The device 100 can generate an image of interest 3120 consisting only of the region of interest from the initial temporary image. As shown in 3100-2 of FIG. 31, the device 100 can display another live view image 3130. In response to the user input for storage, as shown in 3100-3 of FIG. 31, the device 100 can generate the image 3140 by combining the image of interest 3120 with the live view image 3130 as the final temporary image, and store the image. 3140. The position of the image of interest 3120 can be fixed to the position of the display area where the initial temporary image is generated.

When the temporary image is fixed to the display area, the device 100 can produce images of various shapes. FIG. 32 is a reference diagram for explaining a method of generating an effect image from a live view image, according to an exemplary embodiment. As shown in 3200-1 of FIG. 32, when the mode of the device 100 is set to the shooting mode, the device 100 can display the live view image 3210. Device 100 may select a region of interest on live view image 3210. For example, the user can touch the live view image 3210 and display the first object on the top surface. Part of the 3212 area. The device 100 can then select the first object 3212 as the region of interest. When the first object 3212 is selected as the region of interest, the device 100 can generate an initial temporary image including the first object 3212 and generate an image of interest 3220 from the initial temporary image. The image of interest 3220 can be displayed on the device 100 by overlapping with the live view image 3210.

Since the position of the image of interest 3220 is fixed to the position at which the first temporary image is generated, the image of interest 3220 displayed on the display area can remain fixed even when the photographing angle or the position of the camera 160 changes. When the user rotates the camera 160 by 90 degrees, as shown in 3200-2 of FIG. 32, the image of interest 3220 fixed to the display area is also rotated by 90 degrees. However, the live view image 3230 does not rotate. In response to the storage command, as shown in 3200-3 of FIG. 32, the device 100 can generate the effect image 3240 by combining the image of interest 3220 with the live view image 3230 as the final temporary image, and store the effect image 3240. The final temporary image 3230 is a temporary image generated after receiving the storage input. The final temporary image 3230 can also include a first object 3212. The device 100 can generate the effect image 3240 by deleting the first object 3212 included in the final temporary image 3230 and combining the image of interest 3220 with the final temporary image 3230 from which the first object 3212 has been deleted. The area of the effect image 3240 that does not have the pixel information can be restored by the image restoration technique.

Although the image of interest is fixed to the display area as explained above, the exemplary embodiment is not limited thereto. The location of the image of interest can be changed based on user input. For example, when the image of interest overlaps with the live view image, use The person can perform an operation of touching a partial area on which the image of interest is displayed, and then dragging the partial area. Then, the device 100 can change the position of the image of interest to the position where the drag ends according to the user input. The device 100 can also change the size of the image of interest and the like according to user input.

The position of the image of interest can be changed to the position of the region of interest corresponding to the live view image. FIG. 33 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to another exemplary embodiment. As shown in 3300-1 of FIG. 33, when the mode of the device 100 is set to the shooting mode, the device 100 can display the live view image 3310. The device 100 can select an area of interest on the live view image 3310. For example, the user can touch a partial region of the live view image 3310 on which the first object 3312 is displayed. The device 100 can then select the first object 3312 as the region of interest. When the first object 3312 is selected as the region of interest, the device 100 can generate the image of interest 3320 that includes the first object 3312.

As shown in 3300-2 of FIG. 33, the device 100 can display the image of interest 3320 at a predetermined position of the display area. In other words, the device 100 may not display the image of interest 3320 at a location where the image of interest 3320 is generated, but may display the image of interest 3320 at a location predetermined by the device 100. The image of interest 3320 displayed on device 100 may overlap with live view image 3330. The live view image 3330 shown in 3300-2 of FIG. 33 may be different from the live view image 3310 shown by 3300-1 of FIG.

As shown in 3300-3 of FIG. 33, device 100 can generate an effect image 3340 in response to a stored input. The device 100 can move the image of interest 3320 to The effect image 3340 is generated by the first object 3332 corresponding to the region of interest of the live view image 3330 (which is the final temporary image). In FIG. 33, upon receiving the storage input, the image of interest is moved to the position of the first object of the final temporary image and combined with the final temporary image. However, the exemplary embodiments are not limited thereto. When the live view image is displayed, the device 100 can move the image of interest to an area corresponding to the image of interest (ie, the area on which the first object is displayed) and display the live view image on the fly.

It has been explained above that the effect image is generated using the initial temporary image and the final temporary image when the region of interest is selected on the live view image. However, the exemplary embodiments are not limited thereto. The device 100 can generate at least one temporary image other than the initial temporary image and the final temporary image. The device 100 can generate an effect image using the at least one temporary image. FIG. 34 is a flowchart of a method of generating an effect image from an instant view image, according to another exemplary embodiment.

In operation S3410, the device 100 may select an area of interest on the live view image. When the mode of the device 100 is set to the shooting mode, the device 100 can display a live view image. The user can touch a partial area of the live view image on which the object or background is displayed. The device 100 can then select an object or background displayed on the area containing the touched partial area as the area of interest.

In operation S3420 and operation S3430, when the change in the region of interest is equal to or greater than the reference value, the device 100 may generate a temporary image.

The temporary image may be a screen image displayed as a live view image, or may be an image of interest composed only of the region of interest. When selecting a region of interest, Device 100 produces an initial temporary image. The apparatus may calculate a difference between a pixel value of a region of interest of the previously generated temporary image and a pixel value of the region of interest included in the live view image. If the calculated difference is equal to or greater than the reference value, the device 100 may generate a live view image as a temporary image.

A change in the region of interest may occur due to: movement of an object or background corresponding to the region of interest, a change in the size of the object or background, or a change in the pixel value (ie, amount of light) of the object or background.

Device 100 can generate a temporary image until a storage input is received. In other words, the device 100 may store an image in which the change in the region of interest is equal to or greater than the reference value as a temporary image until a storage input is received. The temporary image may include an initial temporary image and a final temporary image.

When the device 100 receives the storage input in operation S3440, the device 100 may generate an effect image in operation S3450. The device 100 can generate an effect image by combining a plurality of temporary images with each other. Alternatively, device 100 may generate an effect image in response to user input selecting one of a plurality of temporary images. In a strict sense, an effect image corresponding to a user input selecting one of a plurality of temporary images may not be regarded as an effect image. However, for convenience of explanation, the effect image corresponding to the user input selecting one of the plurality of temporary images is referred to as an effect image, because the device 100 can display the plurality of temporary images so that the user can select from the image. A temporary image, and the selected temporary image may not be stored as a still image or the like until a user input selecting one of the plurality of temporary images is received.

FIG. 35 is a reference diagram for explaining a method of generating an effect image from an instant view image, according to an exemplary embodiment. As shown in 3500-1 of FIG. 35, when the mode of the device 100 is set to the still image shooting mode, the device 100 can display the live view image 3510. The device 100 can select an area of interest on the live view image 3510. For example, the user can touch a partial region of the live view image 3510 on which the first object 3512 is displayed. The device 100 can then select the first object 3512 as the region of interest. When the first object 3512 is selected as the region of interest, the device 100 can generate a temporary image that includes the first object 3512.

The device 100 can generate the temporary image 3520 at predetermined time intervals until the device 100 receives the storage input. Alternatively, device 100 may generate temporary image 3520 when the change in first object 3512 is equal to or greater than the reference value. The temporary image 3520 may be a screen image displayed as the live view image 3510, or may be an image of interest composed only of the first object 3512. In FIG. 35, the temporary image 3520 is a screen image displayed as the live view image 3510.

As shown in 3500-2 of FIG. 35, device 100 can generate final temporary image 3530 and other temporary images 3520 in response to storage inputs. The final temporary image 3530 and the temporary image 3520 can be displayed on separate areas, respectively. When a plurality of second images 3520 are found, the device 100 may sequentially arrange the plurality of temporary images 3520 in the order in which the plurality of temporary images 3520 are generated.

In response to user input selecting the temporary image 3540 from the plurality of temporary images 3520, the device 100 may display an effect image obtained by combining the final temporary image 3530 with the region of interest of the selected temporary image 3540. 3550, as shown in Figure 3, 3500-3. The device 100 can generate the effect image 3550 by replacing the first object 3532 of the final temporary image 3530 with the first object 3542 of the selected temporary image 3540. A method of generating an effect image using an object as a region of interest in a live view image has been described above. The above is for convenience of explanation, using an object as a region of interest, and the method of using the object as a region of interest to generate an effect image is equally applicable to a case where a background is used as a region of interest.

Device 100 may generate a moving picture due to changes in the region of interest. FIG. 36 is a flowchart of a method of generating a moving picture from an live view image, according to an exemplary embodiment.

In operation S3610, the device 100 may select an area of interest on the live view image. When the mode of the device 100 is set to the moving picture generation mode, the device 100 can display the live view image. The user can touch a partial area of the live view image on which the object or background is displayed. The device 100 can then select an object or background displayed on the area containing the touched partial area as the area of interest.

In operation S3620, the device 100 can receive a user input that generates a moving picture. User input can vary. For example, the user input can be at least one selected from the group consisting of a key input, a touch input, a motion input, a bending input, a voice input, and multiple inputs.

In operation S3630, the device 100 determines whether the change in the region of interest is equal to or greater than a reference value. If the change in the region of interest is equal to or greater than the reference value, the device 100 may generate a moving picture frame in operation S3640. Device 100 A live view image generated at a time when the user input for generating the moving picture is received is generated as a moving picture frame. The device 100 can generate a moving picture frame whenever the change in the region of interest is equal to or greater than the reference value. The moving picture frame may be a screen image displayed as a live view image, or may be information representing a change in a previously moved picture frame. A moving picture frame may be generated until a user input that ends the generation of the moving picture is received in operation S3660.

When the region of interest is an object, the change in the region of interest may be, for example, a change in the movement of the object, a change in the size of the object, or a change in the pixel value representing the object. When the region of interest is the background, the change in the region of interest may be, for example, a change in the background or a change in the pixel value representing the background.

In operation S3660, the device 100 may generate a moving picture file from the moving picture frame in response to the user input that ends the generation of the moving picture.

The device 100 may reproduce the moving picture using a moving picture frame in which the change in the region of interest is equal to or greater than the reference value. FIG. 37 is a flowchart of a method of reproducing a moving picture, according to an exemplary embodiment.

In operation S3710, the device 100 may select a region of interest on the moving picture frame. When the mode of the device 100 is set to the moving picture reproduction mode, the device 100 can display a moving picture. The user can enter a command to stop moving the picture. In response to user input, the device 100 can display a moving picture frame, and move the picture frame as a still picture of the moving picture. The user can touch a partial area of the moving picture frame on which the object or background is displayed. The device 100 can then select an object or background displayed on the area containing the touched partial area as the area of interest.

In operation S3720, the device 100 determines whether the change in the region of interest is equal to or greater than a reference value. When the change in the region of interest is equal to or greater than the reference value, the device 100 may display the moving picture frame in operation S3730. The device 100 can compare the displayed moving picture frame (hereinafter referred to as the current frame) with the moving picture frame (hereinafter referred to as the first next frame) to be reproduced after the reproduction of the current frame. The device 100 can calculate a change between regions of interest of the two frames. When the calculated change is equal to or greater than the reference value, the device 100 can reproduce and display the first next frame.

On the other hand, when the calculated change is less than the reference value, the device 100 does not display the first next frame. The device 100 may again calculate a change between the region of interest of the current frame and the region of interest of the moving picture frame (hereinafter referred to as the second next frame) to be reproduced after the reproduction of the first next frame. When the calculated change is equal to or greater than the reference value, the device 100 can reproduce and display the second next frame. On the other hand, when the calculated change is less than the reference value, the device 100 does not display the second next frame. S3720 and S3730 may be repeatedly performed until the reproduction of the moving picture is ended in operation S3740. In other words, the device 100 may repeatedly perform the operations S3720 and S3730 until the user input that ends the reproduction of the moving picture is received or the reproduction of the moving picture is completed in S3740. A method of reproducing a moving picture has been explained above with reference to FIG. However, the exemplary embodiments are not limited thereto. The method can also be applied to the case of reproducing a still picture in a slide show method.

A method of generating, reproducing, and displaying an image using a region of interest as a partial region of an image has been described so far. Device 100 can utilize a region of interest The field provides a variety of menu images. The menu image can include menu items for executing a particular application. The menu item can be an object, and the area of the menu image that is not a menu item can be defined as the background.

FIG. 38 is a reference diagram for explaining a method of displaying an effect on a menu image, according to an exemplary embodiment. First, as shown in 3800-1 of FIG. 38, when the mode of the device 100 is set to the effect mode for the menu image, the device 100 can display the menu image 3810. The user can select menu item 3812 as the region of interest on menu image 3810. When the menu item 3812 has been selected, as shown at 3800-2 of FIG. 38, the device 100 can display an effect list 3820 that includes effects that can be applied to the menu item 3812. When the user selects the effect item 3822 from the effect list 3820, as shown in 3800-3 of FIG. 38, the device 100 can display the menu item 3830 that has been provided with the effect.

The effect can be provided to the menu item based on the number of times the application corresponding to the menu item is executed. FIG. 39 is a flowchart of a method of providing an effect to a menu item according to the number of times an application corresponding to a menu item is executed, according to an exemplary embodiment.

In operation S3910, the device 100 may determine the number of times the application corresponding to the menu item is executed. The device 100 can determine the number of times the application corresponding to the menu item is executed within a preset time period.

If the number of times the application corresponding to the menu item is executed is equal to or greater than the first value in operation S3910, the device 100 may provide a positive effect to the menu item in operation S3930. The positive effect may be an effect of enhancing the uniqueness of the menu item, and may be, for example, a halo effect, a size increasing effect, or a depth reducing effect.

If the number of times the application corresponding to the menu item is executed is smaller than the first value in operation S3910 and the number of times is equal to or greater than the second value in operation S3920, the device 100 may not provide an effect to the menu item. In other words, the device 100 may display the menu item in a preset state in operation S3940. The second value can be less than the first value.

If the number of times the application corresponding to the menu item is executed is less than the first value in operation S3910 and is also smaller than the second value in operation S3920, the device 100 may provide a negative effect to the menu item in operation S3950. The negative effect may be an effect of weakening the uniqueness of the menu item, and may be, for example, a blur effect, a size reduction effect, or a depth increase effect. In some exemplary embodiments, if the number of times the application corresponding to the menu item is executed is less than the third value, the device 100 may delete the menu item from the single-selection image.

FIG. 40 illustrates an example of displaying a menu image in which an effect has been provided to a menu item according to the number of times an application corresponding to a menu item is executed, according to an exemplary embodiment. As shown in FIG. 40, device 100 can display menu image 4010. The device 100 can display the first menu item 4012 as being larger than other menu items. This means that the first menu item 4012 has been executed more frequently than other menu items. Compared to other menu items, the user is more likely to select the first menu item 4012 in the future. Since the first menu item 4012 is displayed in an enlarged manner and thus becomes prominent, the user can find the first menu item 4012 more easily than other menu items. The device 100 can display the second menu item 4014 as being smaller than other menu items. This means that the second menu item 4014 is executed less frequently than other menu items. The user is less likely to select the second menu item 4014 in the future.

41 through 45 are block diagrams of an apparatus 100, in accordance with an exemplary embodiment.

Referring to FIG. 41, device 100 can include user input 110, controller 120, and display 130. For example, device 100 can provide effects to still images, moving picture frames, live view images, or screen images displayed on display 130.

Referring to FIG. 42, in other exemplary embodiments, device 100 may include user input 110, controller 120, display 130, and memory 140. The device 100 can provide effects to still images or moving pictures stored in the memory 140.

Referring to FIG. 43, in other exemplary embodiments, device 100 may include user input 110, controller 120, display 130, and communicator 150. The device 100 can provide effects to still images or moving pictures stored in the external device, or to live view images captured by the external device.

Referring to FIG. 44, in other exemplary embodiments, device 100 may include user input 110, controller 120, display 130, and camera 160. The device 100 can provide an effect to the live view image captured by the camera 160. However, not all of the illustrated components are essential. The device 100 can be implemented by more or less components than those illustrated in FIG. 41, FIG. 42, FIG. 43, or FIG. 44, or by including FIG. 41, FIG. 42, FIG. 43, or FIG. Any combination of components is implemented.

For example, as illustrated in FIG. 45, in addition to the components included in each of the devices 100 illustrated in FIGS. 41-44, the device 100 may further include an outputter 170, a communicator 140, a sensor 180, and Microphone 190.

The above components will now be explained in detail.

User input 110 represents a unit for the user to enter data to control device 100. For example, the user input 110 can be, but is not limited to, a keypad, a dome switch, a touch pad (eg, a capacitive overlay, a resistive overlay, an infrared beam, an overall strain gauge, a surface acoustic wave). Type, piezoelectric type, etc.), jog wheel, or jog switch.

User input 110 can receive user input selecting a region of interest on the image. According to an exemplary embodiment, the user input selecting the region of interest may vary. For example, the user input can be a key input, a touch input, a motion input, a curved input, a voice input, or multiple inputs.

According to an exemplary embodiment, user input 110 may receive input of selecting a first image and a second image from a plurality of images.

The user input 110 can receive an input of at least one piece of identification information from the list of identification information.

Controller 120 typically controls all operations of device 100. For example, the controller 120 can control the user input 110, the outputter 170, the communicator 150, the sensor 180, and the microphone 190 by executing a program stored in the memory 140.

The controller 120 can obtain at least one piece of identification information identifying the selected region of interest. For example, the controller 120 may generate the identification information by checking the attribute information of the selected region of interest and summarizing the attribute information. The controller 120 can detect the identification information by using image analysis information about the selected region of interest. In addition to the identification information of the region of interest, the controller 120 may also obtain the identification information of the second image.

The controller 120 can provide an effect to the region of interest such that an object or background corresponding to the region of interest is displayed completely differently than the previously displayed object or background. The effect may be, for example, highlighting a halo effect of the region of interest, reducing a blur effect of a difference between pixel values of the region of interest, adjusting a size effect of a size of the region of interest, and changing a depth of the region of interest The depth effect of the information.

The controller 120 can provide an effect to the first image by separating a partial image of the region of interest corresponding to the first image from the second image and combining the separated partial image with the region of interest of the first image.

Display 130 can display information processed by device 100. For example, the display 130 can display a still image, a moving picture, or a live view image. The display 130 can also display identification information identifying the region of interest. The display 130 can also display an effect image and can display an effect folder including the effect image.

When the display 130 and the touch pad form a layer structure to construct a touch screen, the display 130 can be used as an input device and an output device. The display 130 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT-LCD), and an organic light-emitting diode (OLED). At least one of a flexible display, a 3D display, and an electrophoretic display. According to an exemplary embodiment of device 100, device 100 may include at least two displays 130.

The memory 140 can store programs used by the controller 120 to perform processing and control, and can also store input/output data (for example, multiple images, multiple folders, and a list of preference folders).

The memory 140 may include at least one storage medium selected from the group consisting of: flash memory type memory, hard disk type memory, multimedia card micro memory, card type memory (for example, secure digital (SD) or extreme Digital (XD) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM) ), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (ROMM), magnetic memory, disk, and optical disk. The device 100 can operate a web page store on the Internet for performing the storage function of the memory 140.

The programs stored in the memory 140 can be classified into a plurality of modules according to their functions, such as a user interface (UI) module 141, a notification module 142, and an image processing module 143.

The UI module 141 can provide a user interface, a graphical user interface, etc. dedicated to each application and interact with the device 100. The notification module 142 can generate a signal for notifying that an event has been generated in the device 100. The notification module 142 can output the notification signal in the form of a video signal via the display 130, in the form of an audio signal via the audio output 172, or in the form of a vibration signal via the vibration motor 173.

The image processing module 143 can obtain object information, edge information, atmosphere information, color information, and the like included in the captured image by analyzing the captured image.

According to an exemplary embodiment, the image processing module 143 can detect the captured image. The outline of the object contained in the image. According to an exemplary embodiment, the image processing module 143 may obtain the type, name, and the like of the object by comparing the contour of the object included in the image with a predefined template. For example, when the outline of the object is similar to the template of the vehicle, the image processing module 143 can recognize the object contained in the image as the vehicle.

According to an exemplary embodiment, the image processing module 143 may perform face recognition on an object included in the image. For example, the image processing module 143 can detect a person's face area from the image. Examples of the face area detecting method may include a knowledge based method, a feature based method, a template matching method, and an appearance based method, but the exemplary embodiments are not limited thereto.

The image processing module 143 can extract facial features (eg, the shape of the eyes, nose, and mouth that are the major portions of the face) from the detected facial regions. To extract facial features from the face region, a Jaber filter, a local binary mode, or the like can be used, but the exemplary embodiments are not limited thereto.

The image processing module 143 can compare the facial features extracted from the facial regions in the image with the facial features of the pre-registered user. For example, when the extracted facial features are similar to the facial features of a pre-registered first registrant (eg, Tom), the image processing module 143 may determine that the image of the first user is included in the image.

According to an exemplary embodiment, the image processing module 143 may compare a certain region of the image with a color map (color histogram) and extract visual features (eg, color arrangement, pattern, and atmosphere of the image) as image analysis information.

The communicator 150 can include at least one component that enables the device 100 to communicate with a cloud server, an external device, an SNS server, or an external wearable device. For example, the communicator 150 can include a short range wireless communicator 151, a mobile communicator 152, and a broadcast receiver 153.

The short-range wireless communicator 151 may include (but is not limited to) a Bluetooth communicator, a Bluetooth Low Energy (BLE) communicator, a near field communication (NFC) communicator, and a wireless local area network (wireless local Area network, WLAN) (eg Wi-Fi) communicator, ZigBee communicator, infrared data association (IrDA) communicator, Wi-Fi Direct (WFD) communicator, ultra-wideband (ultra wideband, UWB) communicator, Ant+ communicator, etc.

The mobile communicator 152 can exchange wireless signals with at least one of a base station, an external terminal, and a server selected from the mobile communication network. Examples of wireless signals may include voice call signals, video call signals, and various types of data generated during short message service (SMS)/multimedia messaging service (MMS).

The broadcast receiver 153 receives broadcast signals and/or broadcast associated information from an external source via a broadcast channel. The broadcast channel can be a satellite channel, a ground wave channel, or the like. According to an exemplary embodiment, the device 100 may not include the broadcast receiver 153.

The communicator 150 can share at least one selected from the group consisting of a first image, a second image, an effect image, an effect folder, and identification information with an external device. The external device may be selected from a cloud server connected to the device 100, an SNS server, and the same device. At least one of the other device 100 of the user and the device 100 of another user, but the exemplary embodiment is not limited thereto.

For example, the communicator 150 can provide an effect image or effect folder to an external device. The communicator 150 can receive a still image or a moving picture stored in the external device or a live view image taken by the external device from the external device.

The image frame obtained by the camera 160 can be stored in the memory 140 or transmitted to the outside via the communication device 150. According to an exemplary embodiment of device 100, at least two cameras 160 may be included.

The output unit 170 outputs an audio signal, a video signal, or a vibration signal, and may include an audio outputter 172 and a vibration motor 173.

The audio outputter 172 can output audio data received from the communicator 150 or stored in the memory 140. The audio outputter 172 can also output audio signals related to the functions of the device 100 (e.g., call signal reception sound, message reception sound, notification sound). The audio output 172 can include a speaker, a buzzer, and the like.

The vibration motor 173 can output a vibration signal. For example, the vibration motor 173 can output a vibration signal (for example, a call signal reception sound or a message reception sound) corresponding to the output of the audio material or the video material. The vibration motor 173 can also output a vibration signal when the touch screen is touched.

The sensor 180 may sense the state of the device 100, the state around the device 100, or the state of the user wearing the device 100, and may transmit information corresponding to the sensed state to the controller 120.

The sensor 180 can include, but is not limited to, at least one selected from the group consisting of One: magnetic sensor 181, acceleration sensor 182, tilt sensor 183, infrared sensor 184, gyro sensor 185, position sensor (eg, GPS) 186, atmospheric pressure sensor 187 The proximity sensor 188 and the optical sensor 189. The sensor 180 can include, for example, a temperature sensor, an illuminance sensor, a pressure sensor, and an iris recognition sensor. For most sensors, those of ordinary skill in the art will instinctively understand their function according to their names, and therefore will not be described in detail herein.

The microphone 190 can be included as an audio/video (A/V) input unit.

The microphone 190 receives an external audio signal and converts the external audio signal into electrical audio information. For example, the microphone 190 can receive audio signals from an external device or a person who is speaking. The microphone 190 can utilize various noise removal algorithms to remove noise generated when receiving external audio signals.

FIG. 46 is a block diagram showing the structure of a remote server or cloud server 200, according to an exemplary embodiment.

Referring to FIG. 46, the cloud server 200 may include a communicator 210, a controller 220, and a storage 230. However, not all of the illustrated components are essential. Cloud server 200 can be implemented by more or fewer components than those illustrated in FIG.

The above components will now be explained in detail.

The communicator 210 can include at least one component that enables communication between the cloud server 200 and the device 100. The communicator 210 can include a receiver and a transmitter.

The communicator 210 can store images or images stored in the cloud server 200 The list is transmitted to device 100. For example, when the communicator 210 receives a request for an image list from the device 100 connected via a specific account, the communicator 210 can transmit the image list stored in the cloud server 200 to the device 100.

The communicator 210 can transmit the identification information stored in the cloud server 200 or generated by the cloud server 200 to the device 100.

The controller 220 controls all operations of the cloud server 200. For example, the controller 220 can obtain a plurality of pieces of identification information for identifying an image. According to an exemplary embodiment, the plurality of pieces of identification information may be at least two core words or core phrases for identifying images.

For example, when a plurality of pieces of identification information are pre-defined in the metadata of the image, the controller 220 may obtain a plurality of pieces of identification information from the metadata of the image. The cloud server 200 can obtain a plurality of pieces of identification information for identifying the image by using at least one of the attribute information and the image analysis information selected from the image.

The storage 230 can store programs used by the controller 220 to perform processing, or can store input/output data. For example, the cloud server 200 can establish a video database (DB), a database of devices, a database of facial feature information of the user, and an object template database.

The storage 230 can store a plurality of images. For example, the storage 230 can store images uploaded from the device 100. In this case, the storage 230 can map and store the identification information of the device 100 with the image.

A method in accordance with an exemplary embodiment can be implemented as program commands that can be executed by various computer mechanisms and can be recorded on a computer readable recording medium. Electricity The brain readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program commands to be recorded on the computer readable recording medium can be specially designed and configured for the exemplary embodiments, or can be known by those having ordinary knowledge in computer software technology and can be commonly used in computer software technology. Used by the knowledge. Examples of computer readable recording media include: magnetic media such as hard disks, floppy disks, or magnetic tapes; optical media such as compact disk-read-only memory (CD-ROM) or digital versatile discs. (digital versatile disk, DVD); magneto-optical media, such as floppy disks; and hardware devices dedicated to storing and executing program commands, such as read-only memory, random-access memory (RAM), Or flash memory. Examples of program commands are high-level language code that can be executed by a computer using an interpreter or the like, and machine language code that is produced by a compiler.

The exemplary embodiments are to be considered in a Descriptions of features or aspects in each exemplary embodiment should be understood as being able to be used in other similar features or aspects in other exemplary embodiments.

While various exemplary embodiments have been shown and described, the embodiments of the present invention may .

Claims (15)

An image providing method includes: displaying a first image, the first image being a moving image including an object and a background; receiving a user input selecting the object or the background as a region of interest; based on the first The first attribute information of the image is used to obtain the first identification information associated with the region of interest; the second image is obtained from the target image, the second image includes second identification information, and the second identification information is the same as Determining the first identification information; and generating an effect image in the second image by providing an effect to the region of interest of the first image, and by providing a partial image of the second image Generating another effect image in the second image, wherein the partial image of the second image has the same effect as the region of interest provided to the first image, and the partial image corresponds to the first image An identification information, wherein the generating the effect image, the obtaining the first identification information, and the obtaining location are automatically performed in response to receiving the user input Generating said second image and said another image effect. The image providing method of claim 1, wherein the first attribute information includes at least one of context information associated with generation of the first image and annotation information regarding the first image The annotation information is added by the user. The image providing method according to claim 1, wherein the image providing method The first identification information is obtained by summarizing the first attribute information based on a word network. The image providing method of claim 1, wherein the obtaining the second image comprises: using second attribute information of the second image and image analysis information of the second image At least one of the second identification information of the second image is obtained. The image providing method of claim 1, wherein the first identification information of the region of interest is obtained from the first attribute information, and the first attribute information includes the first image Multiple attributes. The image providing method of claim 5, further comprising: receiving user input, the user input selecting at least one of the plurality of attributes of the first image; and based on the Selecting at least one attribute to generate the first identification information, and wherein the step of acquiring the second image comprises: comparing the first identification information with third identification information of the target image. The image providing method of claim 1, wherein the generating the effect image comprises: displaying the partial image of the second image, the partial image corresponding to the first identification information . The image providing method according to claim 1, wherein the effect image is obtained by combining the partial image of the second image with the region of interest of the first image, Where the partial image corresponds to The first identification information. The image providing method according to claim 1, wherein the first image is a live view image. The image providing method of claim 9, wherein the second image is a temporary image generated from the live view image before receiving a user input for storing the image. An apparatus for providing an image, comprising: a display for displaying a first image, the first image being a moving image including an object and a background; and a user input for receiving the object or the background selected as being of interest a user input of the area; the controller is configured to obtain the first identification information of the region of interest based on the first attribute information of the first image, and obtain a second image from the target image, wherein the second image The second identification information is included, and the second identification information is the same as the first identification information, and the controller provides an effect to the region of interest of the first image to be in the second image. Generating an effect image and generating another effect image in the second image by providing a partial image of the second image, wherein the partial image of the second image is provided to the first image The region of interest has the same effect, and the partial image corresponds to the first identification information, wherein the generating station is automatically executed in response to receiving the user input The resultant image, the acquisition of the first identification information, the acquisition of the second Movies Producing the other effect image as and as described. The apparatus for providing an image according to claim 11, wherein the controller is further configured to generate an effect image based on at least one of the first image and the second image. The apparatus for providing an image according to claim 12, wherein the effect image is obtained by combining the partial image of the second image with the region of interest of the first image. And wherein the partial image is a portion of the second image corresponding to the first identification information. The apparatus for providing an image according to claim 12, wherein the first attribute information includes at least one of context information associated with generation of the first image and annotation information about the first image. The annotation information is added by the user. The apparatus for providing an image according to claim 11, wherein the controller is further configured to combine the partial image of the second image with the region of interest of the first image And generating the effect image, wherein the partial image is associated with the first identification information.