KR102409103B1 - Method for editing image - Google Patents

Abstract

According to an embodiment of the present disclosure, it relates to an image modification method, performed by at least one processor. The method includes receiving one or more images, extracting a first object included in the received one or more images, transforming at least one of a first object or a first background region, a deformed first object or a second generating an image including at least one of one background region and outputting the generated image, wherein the first background region is a region other than the first object in the received one or more images.

Description

How to transform an image {METHOD FOR EDITING IMAGE}

The present disclosure relates to a method for modifying an image, and more particularly, to providing a method for deforming an image by extracting an object from one or more images, and deforming at least one of an extracted object or a background region.

Recently, as hardware performance of an image/video capturing apparatus such as a smart phone or a digital camera is improved and a multimedia industry is developed, users' interest in producing an image/video is increasing. Users can easily shoot images/videos, and they want to produce higher-quality images/videos by applying various effects or correcting images/videos to the images/videos they have taken or produced. To this end, users may use various camera applications or correction applications that can be executed on a smartphone or the like.

On the other hand, in the case of a correction service provided by a conventional camera application or a correction application, as a person or object included in an image is corrected, a surrounding area may be distorted or an image/image may appear unnatural due to correction.

The present disclosure provides an image transformation method and a computer program stored in a recording medium for solving the above problems.

The present disclosure may be implemented in various ways including a method or a computer program stored in a readable storage medium.

According to an embodiment of the present disclosure, the image modification method, performed by at least one processor, includes receiving one or more images, extracting a first object included in the received one or more images, the first object or transforming at least one of the first background area, generating an image including at least one of the deformed first object or the first background area, and outputting the generated image, wherein the first background area includes the steps of: is an area other than the first object in the received one or more images.

According to an embodiment, the transforming of at least one of the first object or the first background region includes performing out-of-focus (OUT-OF-FOCUS) on the first background region and adjusting the intensity of out-focus. includes steps.

According to an embodiment, the generating of the image including at least one of the deformed first object or the first background region may include, when one or more received images are moving images, the first background region according to the movement of the first object. This includes creating an out-of-focus video.

According to an embodiment, when the at least one received image is a moving picture, generating a moving picture in which the first background area is out-focused according to the movement of the first object may include: Selecting a second object included in one or more received images, setting a region other than the selected second object as a second background region, and displaying a video in which the second background region is out-focused according to the movement of the second object comprising the steps of creating

According to an embodiment, the transforming of at least one of the first object and the first background region may include deleting the first background region from the received one or more images to generate at least one first image including the first object. and generating an image including at least one of the deformed first object or the first background region, the step of receiving at least one second image and generating the at least one second image and compositing to the first image.

According to an embodiment, transforming at least one of the first object or the first background region includes deleting the first object from one or more received images, a productive adversarial neural network model for image inpainting ( Generating, via a Generative Adversarial Network (GAN), an image of a region from which a first object has been deleted in one or more images and applying the image of the region from which the first object has been deleted to one or more received images do.

According to an embodiment, the generating of an image including at least one of the deformed first object or the first background region may include: when one or more received images are moving images, and the first object is moved in the moving images, and generating a video from which the first object is deleted.

According to an embodiment, transforming at least one of the first object or the first background region includes: receiving a user input for the first object; transforming the first object based on the received user input; Generating an image for blank or distorted regions of one or more images formed according to the deformation of a first object through a productive adversarial neural network model for inpainting, and receiving images for blank or distorted regions of one or more images and applying to one or more images.

According to an embodiment, the step of transforming at least one of the first object or the first background region comprises: receiving a user input for a part of the first object and using a productive adversarial neural network model for image inpainting; deforming some of the first objects.

A computer program stored in a computer-readable recording medium is provided for executing the image transformation method according to an embodiment of the present disclosure in a computer.

In some embodiments of the present disclosure, a moving picture in which an object selected or automatically recognized by a user is focused and a background region is out-focused may be provided. According to an embodiment, a video in which a focused object is changed according to a user's input may be provided.

In some embodiments of the present disclosure, a video from which an object selected or automatically recognized by a user is deleted may be provided. Accordingly, it is possible to prevent an invasion of privacy caused by unintentional object being photographed, and to allow viewers to focus on the main object.

In some embodiments of the present disclosure, a user may extract an object from one or more images and store the object image in a storage device such as a memory in advance. After that, the user can retrieve the stored object image and combine it with another image.

In some embodiments of the present disclosure, an image similar to a multi-exposure shot may be generated by extracting and synthesizing an object from each of a plurality of images continuously taken in a specific time unit or a frame image extracted in a specific time unit from a moving picture. In this case, unlike the existing multiple exposure shot image, since only an object is extracted and synthesized in an image having a background area, the position of each object can be corrected and synthesized through a user input such as a touch input.

In some embodiments of the present disclosure, by transforming the object extracted from the image, and naturally filling the blank area generated by deforming the object using an artificial neural network model, the background due to correction is different from the existing Photoshop or correction application. Distortion may not appear.

In some embodiments of the present disclosure, some regions of an object extracted from an image, such as eyes, nose, mouth, ears, hair, and clothes, may be transformed into other natural shapes using an artificial neural network model.

The effect of the present disclosure is not limited to the above-mentioned effects, and other effects not mentioned are those of ordinary skill in the art to which the present disclosure belongs from the description of the claims (hereinafter referred to as 'person of ordinary skill') can be clearly understood by

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein like reference numerals denote like elements, but are not limited thereto.
1 is a diagram illustrating an example in which a user transforms a video output through a screen of a user terminal according to an embodiment of the present disclosure.
2 is a schematic diagram illustrating a configuration in which an information processing system is communicatively connected to a plurality of user terminals in order to provide an image modification service according to an embodiment of the present disclosure.
3 is a block diagram illustrating an internal configuration of a user terminal and an information processing system according to an embodiment of the present disclosure.
4 is a block diagram illustrating an internal configuration of a processor according to an embodiment of the present disclosure.
5 is a flowchart illustrating an image transformation method according to an embodiment of the present disclosure.
6 is a diagram illustrating an example of out-focusing a background area, which is an area other than some objects, among a plurality of objects included in a moving picture according to an embodiment of the present disclosure.
7 is a diagram illustrating an example of synthesizing an image including an object extracted from one or more images with another image according to an embodiment of the present disclosure.
8 is a diagram illustrating an example of generating an image similar to a multiple exposure shot by synthesizing an image including an object extracted from one or more images with another image according to an embodiment of the present disclosure.
9 is a diagram illustrating an example of deleting an object extracted from one or more images according to an embodiment of the present disclosure.
10 is a diagram illustrating an example of deforming an object extracted from one or more images according to an embodiment of the present disclosure.
11 is a diagram illustrating an example of deforming some of objects extracted from one or more images according to an embodiment of the present disclosure.

Hereinafter, specific contents for carrying out the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, the same or corresponding components are assigned the same reference numerals. In addition, in the description of the embodiments below, overlapping description of the same or corresponding components may be omitted. However, even if descriptions regarding components are omitted, it is not intended that such components are not included in any embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, and only the present embodiments allow the present disclosure to be complete, and the present disclosure will provide those of ordinary skill in the art to fully understand the scope of the invention. It is provided only to inform you.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail. Terms used in this specification have been selected as currently widely used general terms as possible while considering the functions in the present disclosure, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

References in the singular herein include plural expressions unless the context clearly dictates the singular. Also, the plural expression includes the singular expression unless the context clearly dictates the plural. In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, the term 'module' or 'unit' used in the specification means a software or hardware component, and 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside on an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, 'module' or 'unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, may include at least one of procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays or variables. Components and 'modules' or 'units' are the functions provided therein that are combined into a smaller number of components and 'modules' or 'units' or additional components and 'modules' or 'units' can be further separated.

According to an embodiment of the present disclosure, a 'module' or a 'unit' may be implemented with a processor and a memory. 'Processor' should be construed broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. In some contexts, a 'processor' may refer to an application specific semiconductor (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such configuration. You may. Also, 'memory' should be construed broadly to include any electronic component capable of storing electronic information. 'Memory' means random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read-only memory (EPROM); may refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor is capable of reading information from and/or writing information to the memory. A memory integrated in the processor is in electronic communication with the processor.

In the present disclosure, an 'image' may include one or more images, photos, drawings, and/or moving pictures. For example, the image may refer to the moving picture itself and/or each frame image constituting the moving picture, and conversely, the moving picture may refer to an image.

In the present disclosure, an 'object' may refer to each object included in an image, such as a person, an animal, a plant, or an object. For example, the 'object' may be a person, animal, plant, or thing that is the center of the image. Also, in the present disclosure, an 'object' may refer to an object image and/or an image including an object.

In the present disclosure, 'deformation' may refer to all or a part of an image being in a shape different from the existing shape. For example, the transformation may include making a part different from the existing shape based on the existing shape. As another example, the transformation may include applying a new shape that is completely different from the existing shape. As another example, the transformation may include creating a new one that does not already exist. As another example, the transformation may include deleting an existing one.

In this disclosure, 'out of focus' may refer to any aesthetic or aesthetic effect applied to at least a portion of an image. It may refer to processing so that an area other than a main object corresponding to a focus (ie, focus) in an image during and/or after image capturing is not clear. For example, 'out-focus' may include a bokeh effect and/or a blur effect for processing all or part of an image to be blurred. Here, the main object corresponding to the focus (ie, focus) of the image may be processed to be emphasized, highlighted, or sharpened.

1 is a diagram illustrating an example in which a user 110 transforms a video output through a screen 130 of a user terminal 120 according to an embodiment of the present disclosure. At least one processor of the user terminal 120 may receive one or more images. Here, the image may include a moving picture. In an embodiment, the processor of the user terminal 120 may receive one or more images input through an input device such as a camera in real time. For example, based on the user 110's input for starting shooting, the processor captures one or more images through an internal imaging device included in the user terminal 120, and receives one or more images shot in real time. can Alternatively, based on the user 110's input for starting shooting, the processor may capture one or more images through an external imaging device communicating with the user terminal 120, and receive one or more images shot in real time. can In another embodiment, the processor of the user terminal 120 may receive a pre-stored image and/or a recorded image from an internal storage device and/or an external storage device of the user terminal 120 . For example, based on the input of the user 110 for selecting at least one image from among a plurality of stored images, the processor receives at least one image selected from the internal storage device and/or the external storage device of the user terminal 120 . can do.

The processor of the user terminal 120 may extract one or more objects 140 included in one or more received images, and set a region other than the extracted one or more objects 140 as a background region. The processor may deform at least one of the one or more objects 140 or the background area, and generate an image including at least one of the one or more deformed objects or the background area. In one embodiment, the processor performs out-of-focus (OUT-OF-FOCUS) on the background area on the one or more received images, and adjusts the intensity of the out-focus, so that the extracted one or more objects 140 and the transformed background area You can create an image containing In an embodiment, when one or more received images are moving images, the processor may generate a moving image in which the background area is out-focused according to the movement of the object. Specifically, the processor extracts an object from each of a plurality of frames included in the received video and performs out-focus on the background area, thereby generating a moving picture in which the background area is out-focused according to the movement of the object. The processor may output one or more images and/or videos generated in this way through the screen 130 of the user terminal 120 , or output and store them to an internal storage device and/or an external storage device of the user terminal 120 .

The user 110 may record a video including the female object 140 and the vehicle object in real time using the camera of the user terminal 120 . The user 110 may perform a touch input for selecting the female object 140 on the screen 130 on which the video being captured is output. Based on such a user input, the processor may extract the female object 140 included in the video, set a region other than the female object 140 as the background region, and out-focus the background region. Also, the processor may adjust the intensity of the out-focus based on a user input for the out-focus. Additionally, when the female object 140 or the user terminal 120 moves, the processor may generate a video in which the background area is out-focused according to the movement of the female object 140 in the received video. That is, the out-focus effect may be applied to a background area that changes according to the movement of the object. Accordingly, as illustrated, the processor may generate a moving picture in which a background region other than the object is out-focused, and may output the generated moving picture through the screen 130 of the user terminal 120 . Also, the video generated in this way may be stored in a storage device.

1 illustrates an example in which a processor transforms and outputs a video received in real time while capturing a video, but is not limited thereto. For example, the processor may transform and output a pre-captured and pre-stored video.

2 is a schematic diagram illustrating a configuration in which the information processing system 230 is connected to communicate with a plurality of user terminals 210_1 , 210_2 , and 210_3 to provide an image modification service according to an embodiment of the present disclosure. Information processing system 230 may include system(s) capable of providing image modification services. In one embodiment, information processing system 230 may include one or more server devices and/or databases capable of storing, providing, and executing computer-executable programs (eg, downloadable applications) and data related to image transformation services, or It may include one or more distributed computing devices and/or distributed databases based on cloud computing services. For example, the information processing system 230 may include separate systems (eg, servers) for providing an image modification service.

The image modification service provided by the information processing system 230 may be provided to the user through an application for image modification service installed in each of the plurality of user terminals 210_1 , 210_2 , and 210_3 . Alternatively, the user terminals 210_1 , 210_2 , and 210_3 may process tasks such as out-of-focus, image synthesis, and image inpainting by using an image transformation program/algorithm stored therein. In this case, the user terminals 210_1 , 210_2 , and 210_3 may directly process tasks such as out-focus, image synthesis, and image inpainting without communicating with the information processing system 230 .

The plurality of user terminals 210_1 , 210_2 , and 210_3 may communicate with the information processing system 230 through the network 220 . The network 220 may be configured to enable communication between the plurality of user terminals 210_1 , 210_2 , and 210_3 and the information processing system 230 . Network 220 according to the installation environment, for example, Ethernet (Ethernet), wired home network (Power Line Communication), telephone line communication device and wired networks such as RS-serial communication, mobile communication network, WLAN (Wireless LAN), It may consist of a wireless network such as Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited, and the user terminals 210_1, 210_2, 210_3 as well as a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, etc.) that the network 220 may include. ) may also include short-range wireless communication between

Although the mobile phone terminal 210_1, the tablet terminal 210_2, and the PC terminal 210_3 are illustrated as examples of the user terminal in FIG. 2, the present invention is not limited thereto, and the user terminals 210_1, 210_2, and 210_3 are wired and/or wireless communication. It can be any computing device capable of this. For example, the user terminal may include a smart phone, a mobile phone, a computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, and the like. In addition, in FIG. 2 , three user terminals 210_1 , 210_2 , and 210_3 are illustrated as communicating with the information processing system 230 through the network 220 , but the present invention is not limited thereto, and a different number of user terminals is connected to the network ( It may be configured to communicate with information processing system 230 via 220 .

In one embodiment, the information processing system 230 is data from the user terminals 210_1, 210_2, 210_3 through an application for an image transformation service that operates in the user terminals 210_1, 210_2, 210_3 (eg, transformation target image, user input for the image, transformed image, etc.). Conversely, the information processing system 230 is data (eg, a transformation target image, User input for an image, a modified image, etc.) can be provided.

3 is a block diagram illustrating the internal configuration of the user terminal 210 and the information processing system 230 according to an embodiment of the present disclosure. The user terminal 210 may refer to any computing device capable of executing a camera application, a photo editing application, or a web browser and capable of wired/wireless communication, for example, the mobile phone terminal 210_1 of FIG. 2 , a tablet It may include a terminal 210_2, a PC terminal 210_3, and the like. As shown, the user terminal 210 may include a memory 312 , a processor 314 , a communication module 316 , and an input/output interface 318 . Similarly, the information processing system 230 may include a memory 332 , a processor 334 , a communication module 336 , and an input/output interface 338 . As shown in FIG. 3 , the user terminal 210 and the information processing system 230 are configured to communicate information and/or data via the network 220 using the respective communication modules 316 and 336 . can be In addition, the input/output device 320 may be configured to input information and/or data to the user terminal 210 through the input/output interface 318 or to output information and/or data generated from the user terminal 210 .

The memories 312 and 332 may include any non-transitory computer-readable recording medium. According to one embodiment, the memories 312 and 332 are non-volatile mass storage devices such as random access memory (RAM), read only memory (ROM), disk drives, solid state drives (SSDs), flash memory, and the like. (permanent mass storage device) may be included. As another example, a non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal 210 or the information processing system 230 as a separate permanent storage device distinct from the memory. In addition, an operating system and at least one program code (eg, a code for a camera application installed and driven in the user terminal 210 , a photo editing application, etc.) may be stored in the memories 312 and 332 .

These software components may be loaded from a computer-readable recording medium separate from the memories 312 and 332 . The separate computer-readable recording medium may include a recording medium directly connectable to the user terminal 210 and the information processing system 230, for example, a floppy drive, disk, tape, DVD/CD- It may include a computer-readable recording medium such as a ROM drive and a memory card. As another example, the software components may be loaded into the memories 312 and 332 through a communication module rather than a computer-readable recording medium. For example, the at least one program is loaded into the memories 312 and 332 based on a computer program installed by files provided through the network 220 by developers or a file distribution system that distributes installation files of applications. can be

The processors 314 and 334 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 314 , 334 by the memory 312 , 332 or the communication module 316 , 336 . For example, the processors 314 and 334 may be configured to execute received instructions according to program code stored in a recording device, such as the memories 312 and 332 .

The communication modules 316 and 336 may provide a configuration or function for the user terminal 210 and the information processing system 230 to communicate with each other via the network 220 , and the user terminal 210 and/or information processing The system 230 may provide a configuration or function for communicating with another user terminal or another system (eg, a separate cloud system). For example, a request or data (eg, image modification request, etc.) generated by the processor 314 of the user terminal 210 according to a program code stored in a recording device such as the memory 312 is transmitted to the communication module 316 . It may be transmitted to the information processing system 230 through the network 220 according to control. Conversely, a control signal or command provided under the control of the processor 334 of the information processing system 230 is transmitted through the communication module 336 and the network 220 through the communication module 316 of the user terminal 210 . It may be received by the user terminal 210 . For example, the user terminal 210 may receive a deformed image or the like from the information processing system 230 through the communication module 316 .

The input/output interface 318 may be a means for interfacing with the input/output device 320 . As an example, an input device may include a device such as a camera, keyboard, microphone, mouse, etc., including an audio sensor and/or an image sensor, and an output device may include a device such as a display, speaker, haptic feedback device, etc. can As another example, the input/output interface 318 may be a means for an interface with a device in which a configuration or function for performing input and output, such as a touch screen, is integrated into one. For example, when the processor 314 of the user terminal 210 processes a command of a computer program loaded in the memory 312, information and/or data provided by the information processing system 230 or other user terminals are used. A service screen and the like configured by doing this may be displayed on the display through the input/output interface 318 . In FIG. 3 , the input/output device 320 is not included in the user terminal 210 , but the present invention is not limited thereto, and may be configured as a single device with the user terminal 210 . In addition, the input/output interface 338 of the information processing system 230 is connected to the information processing system 230 or means for interfacing with a device (not shown) for input or output that the information processing system 230 may include. can be In FIG. 3, the input/output interfaces 318 and 338 are illustrated as elements configured separately from the processors 314 and 334, but the present invention is not limited thereto, and the input/output interfaces 318 and 338 may be configured to be included in the processors 314 and 334. have.

The user terminal 210 and the information processing system 230 may include more components than those of FIG. 3 . However, there is no need to clearly show most of the prior art components. According to an embodiment, the user terminal 210 may be implemented to include at least a portion of the above-described input/output device 320 . In addition, the user terminal 210 may further include other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database. For example, when the user terminal 210 is a smart phone, it may include components generally included in the smart phone, for example, an acceleration sensor, a gyro sensor, a camera module, various physical buttons, and touch Various components such as a button using a panel, an input/output port, and a vibrator for vibration may be implemented to be further included in the user terminal 210 . According to an embodiment, the processor 314 of the user terminal 210 may be configured to operate an application providing an image modification service. In this case, a code associated with the corresponding application and/or program may be loaded into the memory 312 of the user terminal 210 .

While a program for an application providing an image modification service is being operated, the processor 314 operates an input device such as a touch screen connected to the input/output interface 318, a keyboard, a camera including an audio sensor and/or an image sensor, a microphone, etc. It is possible to receive input or selected text, image, video, voice and/or action, etc., and store the received text, image, video, voice and/or action in the memory 312 or the communication module 316 and It may be provided to the information processing system 230 via the network 220 . For example, the processor 314 may receive the selected at least one image in response to a user input for selecting at least one image from among the plurality of images, and transmit the received image to the communication module 316 and the network 220 . ) through the information processing system 230 can be provided. Alternatively, the processor may receive one or more images from an input device, such as a camera, in response to a user input for image and/or video shooting, and transmit the received images to the communication module 316 and the network 220 . through the information processing system 230 . In an embodiment, the processor 314 may receive a user input for an image through an input device 320 such as a touch screen, a mouse, or a keyboard on an application or a web browser that provides an image transformation service. Thereafter, the processor may directly process the request corresponding to the user input or provide it to the information processing system 230 through the network 220 and the communication module 316 . Here, the user input for the image may include a user input for selecting an object included in the image, a user input for some of the objects, an input for object transformation, an input for out-focus control, and the like.

The processor 314 of the user terminal 210 manages, processes and/or stores information and/or data received from the input device 320 , another user terminal, the information processing system 230 and/or a plurality of external systems. can be configured to The information and/or data processed by the processor 314 may be provided to the information processing system 230 via the communication module 316 and the network 220 . The processor 314 of the user terminal 210 may transmit and output information and/or data to the input/output device 320 through the input/output interface 318 . In an embodiment, the processor 314 may display the received information and/or data on the screen of the user terminal. For example, the processor 314 may display the received transformation target image, the transformed image, and/or the generated image on the screen.

The processor 334 of the information processing system 230 may be configured to manage, process, and/or store information and/or data received from the plurality of user terminals 210 and/or the plurality of external systems. The information and/or data processed by the processor 334 may be provided to the user terminal 210 through the communication module 336 and the network 220 . For example, the processor 334 may provide the received transformation target image, the transformed image, and/or the generated image to the user terminal 210 through the communication module 336 and the network 220 . The processor 334 of the information processing system 230 through the output device 320 such as a display output capable device (eg, a touch screen, a display, etc.) of the user terminal 210, a voice output capable device (eg, a speaker) It may be configured to output processed information and/or data. For example, the processor 334 may output the received transformation target image, the transformed image, and/or the generated image through the screen of the user terminal.

In an embodiment, the processors 314 and 334 may extract one or more objects included in one or more received images. In an embodiment, the processors 314 and 334 may transform at least one of the extracted one or more objects or background regions, and generate an image including at least one of the one or more transformed objects or background regions. Here, the background region may refer to a region other than one or more objects extracted from one or more received images. For example, the processors 314 and 334 may perform out-of-focus (OUT-OF-FOCUS) on the background area and adjust the intensity of the out-focus to generate an image in which the extracted one or more objects are in focus. . Alternatively or additionally, when one or more received images are moving images, the processors 314 and 334 may generate a moving image in which the background area is out-focused according to the movement of the object. Alternatively or additionally, in response to a user response of selecting another object included in the received one or more images, the processor 314 , 334 sets an area other than the selected other object as the background area, and moves the selected other object. Accordingly, a video in which the background area is out of focus may be generated.

As another example, the processor 314 , 334 deletes a background region from the received one or more images to generate at least one first image including the one or more objects, and receives the at least one second image to at least One second image may be combined with the generated first image. As another example, the processors 314 and 334 may delete one or more objects extracted from one or more received images, and use a generative adversarial network (GAN) for image inpainting, from one or more images. One or more extracted objects may generate an image of the deleted area. Thereafter, the processor may apply an image of an area from which one or more objects are deleted to the received one or more images. Alternatively or additionally, when one or more received images are moving images and one or more objects extracted from the moving images are moved, the processors 314 and 334 may generate a moving image in which one or more objects to be moved are deleted. As another example, the processors 314 and 334 transform one or more extracted objects based on the received user input, and through a productive adversarial neural network model for image inpainting, formed according to the deformation of the extracted one or more objects. An image can be created for blank or distorted regions of one or more images. Thereafter, the processor may apply the image for the blank region or the distortion region of the one or more images to the received one or more images. As another example, the processors 314 and 334 may use a productive adversarial neural network model for image inpainting to transform some of the extracted one or more objects.

Although the information processing system 230 is illustrated as an element configured separately from the user terminal 210 in FIG. 3 , the present invention is not limited thereto, and the information processing system 230 may be configured to be included in the user terminal 210 .

4 is a block diagram illustrating an internal configuration of a processor 314 according to an embodiment of the present disclosure. As shown, the processor 314 may include an object extraction unit 410 , an object/background transformation unit 420 , and an image generation unit 430 . The object extractor 410 may receive one or more images. The object extraction unit 410 may extract one or more objects included in one or more received images. In an embodiment, the object extraction unit 410 may extract one or more objects included in the received one or more images by using the artificial neural network model trained to extract the object from the image. For example, the object extractor 410 may extract one or more objects included in one or more received images by using a segmentation neural network model and/or a detection neural network model. Here, various segmentation algorithms may be applied to the segmentation neural network model, and may be, for example, semantic segmentation and/or instance segmentation neural network models.

The object/background transforming unit 420 may transform at least one of an object or a background area extracted by the object extracting unit 410 . Here, the background area may refer to an area other than an object extracted from one or more images. In an embodiment, the object/background transforming unit 420 may out-focus the background area. Additionally, the object/background transforming unit 420 may adjust the intensity of out-focus. In another embodiment, the object/background transforming unit 420 may generate at least one first image including the extracted object by deleting a background area from one or more images. In another embodiment, the object/background transforming unit 420 deletes an object extracted from one or more images, and supplements a region in which an object is deleted from one or more images using a productive adversarial neural network model for image inpainting. can For example, the object/background transforming unit 420 generates an image for a region in which an object is deleted from one or more images through a productive adversarial neural network model for image inpainting, and an image for the region in which an object is deleted. It can be applied to more than one image. In another embodiment, the object/background transforming unit 420 receives a user input for the extracted object, transforms the object based on the received user input, and converts an empty area of one or more images due to the deformation to an image. This can be supplemented by using a productive adversarial neural network model for painting. For example, the object/background transforming unit 420 generates an image for an empty region or a distorted region of one or more images formed according to the transformation of an object through an adversarial neural network model for image inpainting, and An image for an empty area or a distorted area can be applied to one or more images. In another embodiment, the object/background transforming unit 420 may receive a user input for some of the extracted objects and transform some of the extracted objects by using a productive adversarial neural network model for image inpainting. have.

The image generator 430 may generate an image including at least one of a deformed object and a background area. Additionally, the image generator 430 may output the generated image. For example, the image generator 430 may output the generated image through the screen of the user terminal. In an embodiment, when one or more images are moving images, the image generator 430 may generate a moving image in which the background area is out-focused according to the movement of the object. Additionally, the image generator 430 selects another object included in one or more images, sets an area other than the selected other object as a background area, and displays a video in which the set background area is out-focused according to the movement of the selected other object. can create In another embodiment, the image generating unit 430 receives at least one second image, receives the first image generated by the object/background transforming unit 420, and includes at least one second image in the received first image. You can create an image by combining 2 images. In another embodiment, when one or more images are moving images and an object is moved in the moving image, the image generator 430 may generate a moving image in which the moving object is deleted.

5 is a flowchart illustrating an image modification method 500 according to an embodiment of the present disclosure. In an embodiment, the image modification method 500 may be performed by a processor (eg, at least one processor of a user terminal or an information processing system). As shown, the image modification method 500 may be initiated when the processor receives one or more images (S510). The processor may extract a first object included in one or more received images (S520).

Thereafter, the processor may transform at least one of the first object and the first background area ( S530 ). Here, the first background area may refer to an area other than the first object in one or more images. In an embodiment, the processor may perform out-focus on the first background area. Additionally, the processor may adjust the intensity of out-focus. In another embodiment, the processor may generate at least one first image including the first object by deleting the first background area from the one or more received images. In another embodiment, the processor deletes the first object from the one or more received images, and generates, via a productive adversarial neural network model for image inpainting, an image for the region from which the first object has been deleted in the one or more images. can Thereafter, the processor may apply the image for the region from which the first object is deleted to one or more received images. In yet another embodiment, the processor receives a user input for a first object, transforms the first object based on the received user input, and via a productive adversarial neural network model for image inpainting, transforms the first object An image of a blank area or a distorted area of one or more images formed according to . Thereafter, the processor may apply the image for the blank region or the distortion region of the one or more images to the received one or more images. In another embodiment, the processor may receive a user input for some of the first objects, and deform some of the first objects using a productive adversarial neural network model for image inpainting.

Thereafter, the processor may generate an image including at least one of the deformed first object and the first background region ( S540 ). In an embodiment, when one or more received images are moving images, the processor may generate a moving image in which the first background area is out-focused according to the movement of the first object. Additionally, the processor selects a second object included in one or more received images, sets an area other than the selected second object as the second background area, and the second background area is out-focused according to the movement of the second object. You can create videos. In another embodiment, the processor may receive at least one second image and synthesize at least one second image with the first image generated in step S530 . In another embodiment, when one or more received images are moving pictures and the first object is moved in the moving image, the processor may generate a moving image in which the moved first object is deleted. Thereafter, the processor may output the generated image (S550). It can be output to the screen of the user terminal. Or it can be stored in memory.

6 is a diagram illustrating an example of out-focusing a background area that is an area other than some objects 612 and 622 among a plurality of objects included in a video according to an embodiment of the present disclosure. In an embodiment, the processor may receive one or more videos and extract an object included in the received video. Here, the processor may receive a video shot in real time from an input device such as a camera. Alternatively, the processor may receive a pre-stored video and/or a recorded video from a storage device such as a memory. Here, the video may include one or more frame images.

In an embodiment, the processor may extract a first object from the received video and transform at least one of the extracted first object and the first background area. Here, the first object may be selected in response to a user input. Alternatively, the first object may be automatically extracted by an object extraction algorithm or the like. Thereafter, the processor may generate a moving picture including at least one of the deformed first object and the first background area. For example, the processor may extract a first object from some of a plurality of frames included in a moving picture, and may transform at least one of the extracted first object and the first background area. Thereafter, the processor may generate one or more frames including at least one of the deformed first object or the first background region, and generate a video including the generated one or more frames.

In an embodiment, the processor may generate a video in which the first background region is out-focused by out-focusing the first background region and adjusting the out-focus intensity. For example, the processor may generate a video in which the first background area is out-focused according to the movement of the first object. Specifically, the processor extracts the first object from each frame image corresponding to a partial section in the received video and out-focuses the background area that is an area other than the first object, so that the background area is out according to the movement of the first object It is possible to generate a moving picture in which focus is made and the focus of the first object is maintained.

In an embodiment, the processor selects a second object included in the received video and sets an area other than the selected second object as the second background area, so that the second background area is out of focus according to the movement of the second object You can create videos that are Here, the second object may be selected in response to a user input. Alternatively, the second object may be automatically selected by an object extraction algorithm or the like. For example, the processor selects a second object from some of a plurality of frames included in the video, generates one or more frames in which a second background area that is an area other than the selected second object is out-focused, and the generated one or more You can create a video containing frames.

As illustrated, the processor may extract the first male object 612 from each frame image corresponding to the first section from the received video and may out-focus a background area other than the first male object 612 . Accordingly, the processor may generate one or more frames 610 in which the background area is out of focus and the focus to the first male object 612 is maintained according to the movement of the first male object 612 . Thereafter, the processor extracts the second male object 622 from each of the frame images corresponding to the second section in the received video based on the user input for selecting the second male object 622 in the video, and 2 A background area other than the male object 622 may be out-focused. Accordingly, the processor may generate one or more frames 620 in which the background area is out of focus according to the movement of the second male object 622 and the focus to the second male object 622 is maintained.

The processor may generate a video including the frames 610 and 620 generated in this way. Here, the first section and the second section may be continuous sections. In this case, the processor may generate a video in which the focus moves from the first male object 612 to the second male object 622 .

7 is a diagram illustrating an example of synthesizing an image including an object extracted from one or more images 710 with another image according to an embodiment of the present disclosure. The processor extracts one or more objects included in the received one or more images, deletes a background region that is a region other than the extracted object from the one or more images, and at least one first image including the extracted object (eg, an object extract image). Thereafter, the processor may receive at least one second image, and combine the received second image with the generated first image. For example, the processor stores the generated first image in an internal and/or external storage device of the user terminal, and then receives the first image from the storage device according to the user input, and adds the first image to the received first image. 2 Images can be combined. Here, the first image and/or the second image may be moving pictures. In the first image, the extracted object may be enlarged, reduced, duplicated, vertical/left/right symmetry, tilt adjustment, sharpness adjustment, etc. according to user input, or various effects such as black and white effect and emphasis effect are applied to the extracted object. have.

In an embodiment, the processor may generate at least one video including the extracted object by extracting an object included in the received video and deleting a background area that is an area other than the extracted object from the video. Thereafter, the processor may receive at least one image and synthesize the generated video with the received image. Specifically, the processor extracts an object from each of the frame images corresponding to a certain section in the received video, deletes a background area that is an area other than the object, generates one or more frames including only the extracted object, and the generated one or more An object moving picture (eg, a GIF image) may be generated based on the frame.

In another embodiment, the processor extracts an object included in one or more images, deletes a background region that is a region other than the extracted object from the one or more images, and at least one image including the extracted object (eg, a sticker image) ) can be created. Thereafter, the processor may receive at least one moving picture and combine the generated image with the received moving picture. For example, the processor may synthesize an image generated in each frame image corresponding to a predetermined section in the received video.

As illustrated, the processor may generate a first image including only the extracted male object by extracting a male object from the received image 710 and deleting a background area other than the male object. Thereafter, the processor may generate an image 720 including the first image and the second image by receiving the second image and synthesizing the first image with the area around the object included in the second image. Here, the first image may be a reduced, duplicated and/or symmetrical of the extracted male object.

In FIG. 7 , the object included in the first image and the object included in the second image are illustrated as the same, but the present invention is not limited thereto. Also, in FIG. 7 , the processor synthesizes the first image in the area around the object included in the second image, but is not limited thereto. For example, the processor may synthesize the first image in the area where the object exists in the second image.

8 is a diagram illustrating an example of generating an image 800 similar to a multiple exposure shot by synthesizing an image including an object extracted from one or more images with another image according to an embodiment of the present disclosure. In an embodiment, the processor may receive a video including a plurality of frame images. The processor may extract one or more objects from a first frame image among a plurality of frame images included in the received moving picture, and may generate a first image including the extracted object by deleting a background area that is an area other than the extracted object. Thereafter, by synthesizing the second frame image and the first image among the plurality of frame images, the processor may generate an image similar to a multiple exposure shot or a moving image in which a movement trace of an object appears. Here, the first frame image, the second frame image, and/or the first image may be one or more images. Also, the first frame image and/or the second frame image may be a frame image extracted in a specific time unit from among a plurality of frames.

인 자연수)개의 프레임 이미지 중 m(여기서 m은

인 자연수) 번째 프레임을 제외한 나머지 n-1개의 프레임 이미지 각각에서 하나 이상의 객체를 추출하고, 추출한 객체 이외의 영역인 배경 영역을 삭제함으로써 n-1개의 제1 이미지를 생성할 수 있다. 이 후, 프로세서는 m 번째 프레임 이미지와 n-1개의 프레임 이미지를 합성함으로써, 다중 노출 샷과 유사한 이미지 또는 객체의 이동 흔적이 나타나는 동영상을 생성할 수 있다.For example, the processor extracts n (where n is

m (where m is a natural number) of frame images

is a natural number), n-1 first images may be generated by extracting one or more objects from each of the remaining n-1 frame images except for the n-th frame, and deleting a background region that is a region other than the extracted object. Thereafter, by synthesizing the m-th frame image and n-1 frame images, the processor may generate an image similar to a multi-exposure shot or a moving image showing movement traces of an object.

In another embodiment, the processor may receive a plurality of images continuously photographed in a specific time unit. The processor may generate a first image including the extracted object by extracting one or more objects from the first continuous shooting image among the plurality of images and deleting a background area that is a region other than the extracted object. Thereafter, the processor may generate an image similar to a multiple exposure shot or a moving image showing movement traces of an object by synthesizing the second continuously captured image and the first image among the plurality of images. Here, the first continuously shot image, the second continuously shot image, and/or the first image may be one or more images.

As shown, the processor extracts a female object from each of 8 images out of 9 images continuously photographed in a specific time unit, and deletes a background area other than the extracted female object, thereby 1 image can be created. Thereafter, the processor synthesizes the remaining one image (that is, the image including the ice rink background) and the 8 first images among the 9 consecutively shot images, thereby generating an image 800 similar to the multiple exposure shot image. can create In the present disclosure, the image 800 generated according to an embodiment is an image in which a background area exists among a plurality of consecutively photographed images by extracting only an object from some of the plurality of consecutively photographed images, unlike the existing multiple exposure shot image. Since it is synthesized in , the position of each object to be synthesized can be modified through a user input such as a touch input.

In FIG. 8 , the processor synthesizes the object images extracted from each of the sequentially captured images (or frame images) as it is, but is not limited thereto. For example, the processor may synthesize the extracted object image by applying a black-and-white effect. Alternatively, the processor may correct and synthesize the vividness, saturation, brightness, etc. of colors according to the shooting order of the sequentially captured images (or frame images) from which each object image is extracted.

9 is a diagram illustrating an example of deleting an object extracted from one or more images according to an embodiment of the present disclosure. The processor extracts and deletes one or more objects from one or more received images, and may supplement regions in which objects are deleted from one or more images using a productive adversarial neural network model for image inpainting. In an embodiment, the processor extracts and deletes one or more objects from one or more received images, and generates an image for a region in which an object is deleted from one or more images through a productive adversarial neural network model for image inpainting. . For example, the processor may generate an image of the area from which the object is deleted based on a surrounding area of the area from which the object is deleted. Alternatively, when the received one or more images are frame images of a moving picture, the processor may generate an image for a region from which an object is deleted based on the before/after frame images. After that, the processor may apply the image for the area from which the object is deleted to one or more images.

In an embodiment, when one or more received images are moving images, the processor may generate a moving image in which the object is deleted according to the movement of the extracted object. For example, when one or more received images are moving images and an object is moved in the moving image, a moving image may be generated in which the moving object is deleted. Specifically, by extracting and deleting an object from each of a plurality of frames corresponding to a predetermined section in a video, a video in which the object does not appear even if the object moves can be generated. In this case, the object extracted and deleted by the processor may be an object selected through a user input such as a touch, an object automatically recognized as not a main object, and/or an object not included in a preset object DB.

As illustrated, the processor may extract and delete the person object standing on the beach background from the first image 710 . Areas where human objects have been deleted can be filled naturally using a productive adversarial neural network model for image inpainting. To this end, the processor generates an image for the area in which the human object is deleted through a productive adversarial neural network model for image inpainting, and applies the image for the area in which the human object is deleted to the first image 710 . have. Accordingly, the processor may generate and/or output the filled second image 720 such that the human object is deleted and the deleted area is naturally connected to the surrounding background area. According to an embodiment of the present disclosure, it is possible to prevent invasion of privacy caused by an unintentional object being photographed, and to allow users to focus on the main object.

10 is a diagram illustrating an example of deforming an object extracted from one or more images according to an embodiment of the present disclosure. The processor may receive a user input for the object, and may deform the object based on the received user input. The processor may compensate for blank or distorted regions of one or more images due to such deformations using a productive adversarial neural network model for image inpainting. In one embodiment, the processor generates an image for a blank region or a distortion region of one or more images formed according to the deformation of an object, through the productive adversarial neural network model for image inpainting, and the blank region or distortion region of the one or more images may be applied to one or more received images. For example, the processor may naturally supplement the image through image inpainting based on the surrounding area of the blank area that appears as object deformation in the image. Alternatively or additionally, when one or more images are frame images of a moving picture, the processor may naturally supplement the blank area of the image through image inpainting based on frames before and after the corresponding frame. Blank areas may appear in the image.

As illustrated, the user may increase the height of the human object and change the body shape to be slim through a touch input, a click input, and/or a drag input to the object in the output image 1010 . As the body of the human object extracted from the image is deformed to be slim, some of the area where the object existed before the deformation may appear as an empty area or appear distorted in the image. The processor generates an image for an empty area or distortion area formed by deforming the body of a human object to a slender body through a productive adversarial neural network model for image inpainting, and creates an image for the empty area or distortion area as an image (1010) can be applied to That is, the processor supplements these blank or distorted regions by using a productive adversarial neural network model for image inpainting, thereby generating an image 1020 in which the blank or distorted region generated by deforming the extracted object is naturally filled. can create In this case, the distortion of the background due to the correction may not appear unlike the existing photoshop or correction application.

11 is a diagram illustrating an example of deforming some of objects extracted from one or more images according to an embodiment of the present disclosure. In one embodiment, the processor may receive user input for some of the extracted objects from one or more images, and use a productive adversarial neural network model for image inpainting to transform or create some of the extracted objects. . Here, the productive adversarial neural network model for image inpainting may be a model trained to determine a color, a partial shape, eyes, nose, mouth, clothes, hair, etc. that match an object included in an input image.

For example, if the processor automatically recognizes that an object's eyes are closed, the processor uses a productive adversarial neural network model for image inpainting to transform the object's eye region into an eye shape that matches the object, or You can create matching eye shapes and apply them to your image. Alternatively, when the user touches an eye region of an object in the image, the processor uses a productive adversarial neural network model for image inpainting to transform the eye region of the object into an eye shape that matches the object or You can create an eye shape and apply it to an image.

As illustrated, when the human object included in the first image 1110 has its eyes closed, the user may apply a touch input to the eye region of the human object. In response to such a user input, the processor may generate and/or output a natural image including the human object with the eyes open, such as the second image 1120, by transforming the eye region into an eye shape matching the corresponding human object. have. Alternatively, the processor may recommend a plurality of eye shapes suitable for the corresponding object and receive a user input for selecting one of the plurality of recommended eye shapes to transform the eye region of the object into the selected eye shape. .

11 illustrates an example in which the processor transforms the object's eye region into an open eye shape when the object closes its eyes, but is not limited thereto. For example, when the ears of the object extracted from the image cannot be seen properly, the processor may transform the ear region of the object into an ear shape suitable for the object or a shape covered with hair. As another example, in response to a user input for the bangs region of the object, the processor may transform the bangs region of the object into a different style of bangs. As another example, in response to a user input on the under-nose area of the object, the processor may transform the under-nose area of the object into a mustache shape or a mustache-free shape. As another example, in response to a user input on the lip region of the object, the processor may transform the lip region of the object into various lip shapes, such as a smiling lip shape.

In one embodiment, the above-described image transformation method may be provided as a computer program stored in a computer-readable recording medium for execution in a computer. The medium may continuously store a computer executable program, or may be a temporary storage for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributedly on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute various other software, and servers. According to another embodiment, the above-described image transformation method may be operated through a browser (eg, a web browser) that the user terminal accesses without installing a separate program. That is, the user of the user terminal may be provided with an image modification service through a web browser.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those of ordinary skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure are suitable for use in general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the present disclosure. It may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementations, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on computer readable media such as programmable read-only memory), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. It may be implemented as stored instructions. The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to various modifications without departing from the spirit or scope of the disclosure. Accordingly, this disclosure is not intended to be limited to the examples set forth herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Although example implementations may refer to utilizing aspects of the presently disclosed subject matter in the context of one or more standalone computer systems, the subject matter is not so limited, but rather in connection with any computing environment, such as a network or distributed computing environment. may be implemented. Still further, aspects of the presently disclosed subject matter may be implemented in or across a plurality of processing chips or devices, and storage may be similarly affected across the plurality of devices. Such devices may include PCs, network servers, and handheld devices.

Although the present disclosure has been described with reference to some embodiments herein, it should be understood that various modifications and changes can be made without departing from the scope of the present disclosure as understood by those skilled in the art to which the present invention pertains. something to do. Moreover, such modifications and variations are intended to fall within the scope of the claims appended hereto.

110: user
120: user terminal
130: screen
140: object

Claims (10)

An image modification method performed by at least one processor, comprising:
receiving one or more images;
extracting a first object included in the received one or more images;
transforming at least one of the first object and the first background area;
generating an image including at least one of the deformed first object and a first background area; and
outputting the generated image,
The step of transforming at least one of the first object and the first background area includes:
Receiving a user input including a drag input for a part of the first object, wherein the drag input includes information on a specific point of the part of the first object corresponding to a starting point of the drag input and information about the drag input. including information on a specific point located after a part of the first object corresponding to the end point is deformed;
Based on a user input of a portion of the first object, the first object is moved from a specific point of the first object corresponding to the starting point of the drag input to the second object corresponding to the end point of the drag input. 1 transforming a part of an object to a specific point located after being transformed; and
Through a generative adversarial network (GAN) model for image inpainting, generating an image for the distortion region of the image formed according to the deformation of a part of the first object,
The step of generating an image including at least one of the deformed first object and the first background region includes:
applying an image for a distorted region of the generated image to the one or more images;
and the first background area is an area other than the first object in the received one or more images.
According to claim 1,
The step of transforming at least one of the first object and the first background area includes:
performing out-of-focus (OUT-OF-FOCUS) on the first background area; and
adjusting the intensity of the out-focus
Including, an image transformation method.
3. The method of claim 2,
The step of generating an image including at least one of the deformed first object and the first background region includes:
When the one or more received images are moving images, generating a moving image in which the first background area is out-focused according to the movement of the first object;
How to transform an image.
4. The method of claim 3,
generating a moving picture in which the first background area is out-focused according to the movement of the first object when the one or more received images are moving pictures;
selecting a second object included in the received one or more images;
setting an area other than the selected second object as a second background area;
generating a moving picture in which the second background area is out-focused according to the movement of the second object;
How to transform an image.
According to claim 1,
The step of transforming at least one of the first object and the first background area includes:
generating at least one first image including the first object by deleting the first background area from the received one or more images;
The step of generating an image including at least one of the deformed first object and the first background region includes:
receiving at least one second image; and
synthesizing the at least one second image with the generated at least one first image;
Including, an image transformation method.
According to claim 1,
The step of transforming at least one of the first object and the first background area includes:
deleting the first object from the received one or more images; and
generating an image for a region in which the first object is deleted from the one or more images through a productive adversarial neural network model for image inpainting;
The step of generating an image including at least one of the deformed first object and the first background region includes:
applying an image for the area from which the first object is deleted to the received one or more images;
Including, an image transformation method.
7. The method of claim 6,
The step of generating an image including at least one of the deformed first object and the first background region includes:
When the at least one received image is a moving picture and the first object is moved in the moving picture, generating a moving picture in which the moved first object is deleted.
delete delete A computer program stored in a computer-readable recording medium for executing the image modification method according to any one of claims 1 to 7 in a computer.

Images