WO2019066281A1

WO2019066281A1 - Electronic device for providing image related to inputted information, and operating method therefor

Info

Publication number: WO2019066281A1
Application number: PCT/KR2018/010301
Authority: WO
Inventors: 최동범; 이상진; 정승혜; 원종훈; 이해동
Original assignee: 삼성전자주식회사
Priority date: 2017-09-29
Filing date: 2018-09-04
Publication date: 2019-04-04
Also published as: KR102457568B1; US20200273137A1; KR20190038053A; US11341595B2

Abstract

Various embodiments of the present invention relate to an electronic device capable of generating a virtual image by using a user input, and an operating method therefor. The electronic device according to various embodiments of the present invention comprises a communication module and a processor, wherein the processor can be set so as to: receive an input through the electronic device or an external device functionally connected through the communication module; check situation information related to the input, wherein the situation information includes time information and object information; search for at least one image, corresponding to the situation information, among one or more images stored in the electronic device or the external device; provide the found at least one image when the at least one image corresponding to the situation information is found; generating an image corresponding to situation information by using at least one other image, corresponding to a part of the situation information, among the one or more images, when the at least one image corresponding to the situation information is not found; and provide the generated image.

Description

An electronic device for providing an image associated with input information and a method of operating the same

Various embodiments of the present invention are directed to an electronic device capable of generating a virtual image associated with input information and a method of operation thereof.

As portable electronic devices such as smart phones have become increasingly sophisticated, a variety of services are being offered through electronic devices. Various services include basic services such as telephone calls and text transmission, and playback and editing of multimedia such as image / video.

With the development of wireless Internet service, users of electronic devices can experience various services by exchanging information between a plurality of electronic devices connected to a network, as well as by reproducing or editing contents stored in electronic devices.

Information exchanged between electronic devices includes peripheral information (e.g., time information, place information, etc.) of a user or an electronic device.

The electronic device can transmit the photographed image to an external device in order to acquire an image of improved image quality than the image acquired through the image sensor. The electronic device can transmit an image search condition to an external device to search for an image corresponding to the condition.

However, the method of improving the image quality of the photographed image or retrieving the image stored in the external device as described above has a limitation in providing a virtual image that the user wants to see, so that it is difficult to perfectly satisfy the user's needs.

Various embodiments of the present invention relate to an electronic device capable of generating a virtual image and a method of operation thereof.

The technical objects to be achieved by the present invention are not limited to the technical matters mentioned above, and other technical subjects which are not mentioned can be clearly understood by those skilled in the art from the following description. There will be.

An electronic device according to various embodiments of the present invention includes a communication module and a processor, wherein the processor receives input via an external device operatively connected through the electronic device or the communication module, Wherein the status information includes time information and object information and searches at least one image corresponding to the status information among one or more images stored in the electronic device or the external device, Providing at least one image retrieved when a corresponding one of the at least one image is retrieved and, if the at least one image corresponding to the context information is not retrieved, Using at least one other corresponding image, Generating an image corresponding to the information, and sulfur, and is set to provide the generated image.

The method of operation of an electronic device according to various embodiments of the present invention includes receiving an input via an external device functionally connected through the electronic device or a communication module of the electronic device and verifying contextual information associated with the input, Wherein the context information includes time information and object information and acquires at least one image corresponding to a part of the context information among the one or more images stored in the electronic device or the external device, To generate an image corresponding to the situation information, and to provide the generated image.

According to various embodiments, an electronic device may implement a virtual image that a user would like to see by acquiring each object from a plurality of images and combining the acquired objects, using an image generation condition received from a user .

Figure 1 illustrates an environment for a network in accordance with various embodiments of the present invention.

Figure 2 shows a block diagram of an electronic device according to various embodiments of the present invention.

Fig. 3 shows a conceptual diagram for explaining the operation of an electronic device and an external device according to various embodiments of the present invention.

4A is a block diagram of an electronic device in accordance with various embodiments of the present invention.

4B is a block diagram of an external device in accordance with various embodiments of the present invention.

5 is a flow chart for providing an image in which an electronic device according to various embodiments of the present invention satisfies an image generation condition.

Figure 6 is a detailed flow diagram of an operation in which an electronic device according to various embodiments of the present invention retrieves an image that satisfies part of an image generation condition and generates an image that satisfies the image generation condition.

7 and 8 are cross-sectional signaling diagrams of an electronic device and an external device for providing an image satisfying an image generation condition according to various embodiments of the present invention.

Figures 9A and 9B illustrate examples of user interfaces for receiving image creation requests in accordance with various embodiments of the present invention.

10 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to determine an image generation condition.

Figure 11 shows an example of determining an image generation condition based on an image generation request according to various embodiments of the present invention.

12 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to obtain an object image to be included in a request image.

Figure 13 shows information of images and images stored in a first area of memory in accordance with various embodiments of the present invention.

Figure 14 shows information of images and images stored in a second area of memory in accordance with various embodiments of the present invention.

15 is a detailed flowchart of an operation for an electronic device to generate a request image according to various embodiments of the present invention.

16 is a detailed flow diagram of an operation for an electronic device to generate an object image that fails acquisition in accordance with various embodiments of the present invention.

17 illustrates one example in which an electronic device according to various embodiments of the present invention generates a request image.

18 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to generate a request image.

Figure 19 illustrates a user interface for receiving user feedback in accordance with various embodiments of the present invention.

20 is a flow diagram of an operation of storing an image acquired by an external device in a memory in accordance with various embodiments of the present invention.

1 is a block diagram of an electronic device 101 in a network environment 100, in accordance with various embodiments. 1, an electronic device 101 in a network environment 100 communicates with an electronic device 102 via a first network 198 (e.g., near-field wireless communication) or a second network 199 (E. G., Remote wireless communication). &Lt; / RTI > According to one embodiment, the electronic device 101 is capable of communicating with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identity module 196, and an antenna module 197 ). In some embodiments, at least one (e.g., display device 160 or camera module 180) of these components may be omitted or other components may be added to the electronic device 101. In some embodiments, some components, such as, for example, a sensor module 176 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) embedded in a display device 160 Can be integrated.

Processor 120 may be configured to operate at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120 by driving software, e.g., And can perform various data processing and arithmetic operations. Processor 120 loads and processes commands or data received from other components (e.g., sensor module 176 or communication module 190) into volatile memory 132 and processes the resulting data into nonvolatile memory 134. [ Lt; / RTI > According to one embodiment, the processor 120 may operate in conjunction with a main processor 121 (e.g., a central processing unit or an application processor) and, independently, or additionally or alternatively, Or a co-processor 123 (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communications processor) specific to the designated function. Here, the coprocessor 123 may be operated separately from or embedded in the main processor 121.

In such a case, the coprocessor 123 may be used in place of the main processor 121, for example, while the main processor 121 is in an inactive (e.g., sleep) state, At least one component (e.g., display 160, sensor module 176, or communications module 176) of the components of electronic device 101 (e.g., 190) associated with the function or states. According to one embodiment, the coprocessor 123 (e.g., an image signal processor or communications processor) is implemented as a component of some other functionally related component (e.g., camera module 180 or communication module 190) . Memory 130 may store various data used by at least one component (e.g., processor 120 or sensor module 176) of electronic device 101, e.g., software (e.g., program 140) ), And input data or output data for the associated command. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

The program 140 may be software stored in the memory 130 and may include, for example, an operating system 142, a middleware 144,

The input device 150 is an apparatus for receiving a command or data to be used for a component (e.g., processor 120) of the electronic device 101 from the outside (e.g., a user) of the electronic device 101, For example, a microphone, a mouse, or a keyboard may be included.

The sound output device 155 is a device for outputting a sound signal to the outside of the electronic device 101. For example, the sound output device 155 may be a speaker for general use such as a multimedia reproduction or a sound reproduction, . According to one embodiment, the receiver may be formed integrally or separately with the speaker.

Display device 160 may be an apparatus for visually providing information to a user of electronic device 101 and may include, for example, a display, a hologram device, or a projector and control circuitry for controlling the projector. According to one embodiment, the display device 160 may include a touch sensor or a pressure sensor capable of measuring the intensity of the pressure on the touch.

The audio module 170 is capable of bi-directionally converting sound and electrical signals. According to one embodiment, the audio module 170 is configured to acquire sound through the input device 150, or to acquire sound from the sound output device 155, or an external device (e.g., an electronic device) that is wired or wirelessly connected to the electronic device 101 The device 102 (e.g., a speaker or headphone)).

The sensor module 176 may generate an electrical signal or data value corresponding to an internal operating state (e.g., power or temperature) of the electronic device 101, or an external environmental condition. The sensor module 176 may be a gesture sensor, a gyro sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, Or an illuminance sensor.

The interface 177 may support a designated protocol that may be wired or wirelessly connected to an external device (e.g., the electronic device 102). According to one embodiment, the interface 177 may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may be a connector such as an HDMI connector, a USB connector, an SD card connector, or an audio connector (not shown) capable of physically connecting the electronic device 101 and an external device For example, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (e.g., vibrations or movements) or electrical stimuli that the user may perceive through tactile or kinesthetic sensations. The haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture a still image and a moving image. According to one embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

The power management module 188 is a module for managing the power supplied to the electronic device 101, and may be configured as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 is an apparatus for supplying power to at least one component of the electronic device 101 and may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module 190 is responsible for establishing a wired or wireless communication channel between the electronic device 101 and an external device (e.g., electronic device 102, electronic device 104, or server 108) And the like. Communication module 190 may include one or more communication processors that support wired communication or wireless communication, operating independently of processor 120 (e.g., an application processor). According to one embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (E.g., a local area network (LAN) communication module, or a power line communication module), and the corresponding communication module may be used to communicate with a first network 198 (e.g., Bluetooth, WiFi direct, Communication network) or a second network 199 (e.g., a telecommunications network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). The various types of communication modules 190 described above may be implemented as a single chip or may be implemented as separate chips.

According to one embodiment, the wireless communication module 192 may use the user information stored in the subscriber identification module 196 to identify and authenticate the electronic device 101 within the communication network.

The antenna module 197 may include one or more antennas for externally transmitting or receiving signals or power. According to the temporary example, the communication module 190 (for example, the wireless communication module 192) can transmit a signal to or receive a signal from an external device through an antenna suitable for the communication method.

Some of the components are connected to each other via a communication method (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI) (Such as commands or data) can be exchanged between each other.

According to one embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic devices 102 and 104 may be the same or a different kind of device as the electronic device 101. [ According to one embodiment, all or some of the operations performed in the electronic device 101 may be performed in another or a plurality of external devices. According to one embodiment, in the event that the electronic device 101 has to perform some function or service automatically or upon request, the electronic device 101 may be capable of executing the function or service itself, And may request an external device to perform at least some functions associated therewith. The external device that receives the request may execute the requested function or additional function, and may forward the result to the electronic device 101. [ The electronic device 101 can directly or additionally process the received result to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

The electronic device according to the various embodiments disclosed herein can be various types of devices. The electronic device can include, for example, at least one of a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

It should be understood that the various embodiments of the present document and the terms used therein are not intended to limit the techniques described herein to specific embodiments, but rather to include various modifications, equivalents, and / or alternatives of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly dictates otherwise. In this document, the expressions "A or B," "at least one of A and / or B," "A, B or C," or "at least one of A, B, and / Possible combinations. Expressions such as "first", "second", "first" or "second" may be used to qualify the components, regardless of order or importance, and to distinguish one component from another And does not limit the constituent elements. When it is mentioned that some (e.g., first) component is "(functionally or communicatively) connected" or "connected" to another (second) component, May be connected directly to the component, or may be connected through another component (e.g., a third component).

As used herein, the term " module " includes units comprised of hardware, software, or firmware and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. A module may be an integrally constructed component or a minimum unit or part thereof that performs one or more functions. For example, the module may be configured as an application-specific integrated circuit (ASIC).

Various embodiments of the present document may include instructions stored on a machine-readable storage medium (e.g., internal memory 136 or external memory 138) readable by a machine (e.g., a computer) Software (e.g., program 140). The device may include an electronic device (e.g., electronic device 101) in accordance with the disclosed embodiments as an apparatus capable of calling stored instructions from the storage medium and operating according to the called instructions. When the instruction is executed by a processor (e.g., processor 120), the processor may perform the function corresponding to the instruction, either directly or using other components under the control of the processor. The instructions may include code generated or executed by the compiler or interpreter. A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' means that the storage medium does not include a signal and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

According to a temporal example, the method according to various embodiments disclosed herein may be provided in a computer program product. A computer program product can be traded between a seller and a buyer as a product. A computer program product may be distributed in the form of a machine readable storage medium (eg, compact disc read only memory (CD-ROM)) or distributed online through an application store (eg PlayStore ^™ ). In the case of on-line distribution, at least a portion of the computer program product may be temporarily stored, or temporarily created, on a storage medium such as a manufacturer's server, a server of an application store, or a memory of a relay server.

Each of the components (e.g., modules or programs) according to various embodiments may be comprised of a single entity or a plurality of entities, and some of the subcomponents described above may be omitted, or other subcomponents May be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity to perform the same or similar functions performed by each respective component prior to integration. Operations performed by a module, program, or other component, in accordance with various embodiments, may be performed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be performed in a different order, .

Figure 2 is a block diagram 200 of a camera module 180, in accordance with various embodiments. 2, the camera module 180 includes a lens assembly 210, a flash 220, an image sensor 230, an image stabilizer 240, a memory 250 (e.g., a buffer memory), or an image signal processor (260). The lens assembly 210 can collect light emitted from a subject, which is an object of image photographing. The lens assembly 210 may include one or more lenses. According to one embodiment, the camera module 180 may include a plurality of lens assemblies 210. In this case, the camera module 180 may be, for example, a dual camera, a 360 degree camera, or a spherical camera. The plurality of lens assemblies 210 may have the same lens properties (e.g., angle of view, focal length, autofocus, f number, or optical zoom), or at least one lens assembly may have at least one Lt; / RTI > may have different lens properties. The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens. The flash 220 may emit a light source used to enhance the light emitted from the subject. Flash 220 may include one or more light emitting diodes (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs), or xenon lamps.

The image sensor 230 can acquire an image corresponding to the subject by converting the light transmitted from the subject through the lens assembly 210 into an electrical signal. According to one embodiment, the image sensor 230 may include a selected one of the image sensors of different properties, such as, for example, an RGB sensor, a BW (black and white) sensor, an IR sensor, A plurality of image sensors having different attributes, or a plurality of image sensors having different attributes. Each of the image sensors included in the image sensor 230 may be implemented by, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

The image stabilizer 240 is configured to respond to the movement of the camera module 180 or the electronic device 101 including it to at least partially compensate for the negative effects (e.g., image jitter) At least one lens or image sensor 230 included in the assembly 210 may be moved or controlled in a particular direction (e.g., adjusting the read-out timing). According to one embodiment, the image stabilizer 240 may be implemented as an optical image stabilizer, for example, and may include a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 180 Can be used to detect the motion.

The memory 250 may at least temporarily store at least a portion of the image acquired via the image sensor 230 for subsequent image processing operations. For example, if image acquisition according to the shutter is delayed or a plurality of images are acquired at high speed, the acquired original image (e.g., a high resolution image) is stored in the memory 250, and a corresponding copy An image (e.g., a low resolution image) can be previewed via the display device 160. Thereafter, at least a portion of the original image that was stored in the memory 250 may be acquired and processed by, for example, the image signal processor 260 if the specified condition is satisfied (e.g., user input or system command). According to one embodiment, the memory 250 may be comprised of at least a portion of the memory 130, or a separate memory operated independently thereof.

The image signal processor 260 performs image processing (e.g., depth map generation, three-dimensional modeling, panorama generation, feature point extraction, and the like) on an image obtained through the image sensor 230 or an image stored in the memory 250, Image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening) 260 may perform control (e.g., exposure time control, or lead-out timing control, etc.) for at least one of the components (e.g., image sensor 230) included in camera module 180. [ The image processed by the signal processor 260 may be stored back into the memory 250 for further processing or may be stored in the external components of the camera module 180 such as the memory 130, The image signal processor 260 may comprise at least a portion of the processor 120 or may be coupled to the processor 120 and / The image processed by the image signal processor 260 may be processed by the processor 120 as it is or after additional image processing, and then displayed on the display device (not shown) 160 < / RTI >

According to one embodiment, the electronic device 101 may include two or more camera modules 180, each having a different attribute or function. In this case, for example, at least one camera module 180 may be a wide angle camera or a front camera, and at least one other camera module may be a telephoto camera or a rear camera.

3 shows a conceptual diagram for explaining the operation of an electronic device and an external device according to various embodiments of the present invention.

The electronic device 101 may include an image sensor 321, an ISP 323 and a memory 325. The external device 300 may include a recognition module 331, an ISP 333 and a storage 335. The recognition module 331 may be a logical module or a processor of the external device 300. [ The ISP 333 may also be implemented as a processor of the external device 300. For example, the processor of the external device 300 may perform both recognition and image processing. Although not shown, the electronic device 101 may include a communication module (e.g., a communication interface 170 or a communication module 220) capable of transmitting and receiving data with the external device 300. The external device 300 may include a communication module capable of transmitting and receiving data with the electronic device 101.

The image sensor 321 (e.g., camera module 180) may acquire an image for an external object and generate a corresponding raw image 322 (raw image). The image sensor 321 may pass the low image 322 to the ISP 323. In various embodiments of the present invention, the image sensor 321 may generate a small row image 324 and transmit it to the external device 300 via the communication module. In yet another embodiment, a processor of the electronic device 101 other than the image sensor 321 may generate a small row image 324, and the generated small row image 324 may be transmitted to an external electronic device 300). The image sensor 321 can transmit the raw image 322 in a compressed state to the ISP 323 or the external device 300. [ The image sensor 321 may be compressed and stored in a memory within the image sensor 321 for some processing of the raw image 322. The recognition module 331 of the external device 300 can obtain the small row image 324 through the communication module and segment the at least one image region from the small row image 324. [ The recognition module 331 may recognize each of at least one image region separated by the segmentation result. Correction area information 332 including at least one of information associated with a plurality of image areas generated from the recognition module 331, for example, coordinate information of the image area or recognition results, may be generated. The correction area information 332 may be transmitted to the electronic device 101. The ISP 323 can correct the row image 322 using the correction area information 332, and thus the corrected image can be generated. The corrected image may have a format of YUV, for example. The corrected image may be stored in memory 325. [ Or the corrected image may be compressed according to, for example, the JPEG method, and the compressed image may be stored in the memory 325. [

In various embodiments of the present invention, the raw image 322 provided from the image sensor 321 may be transmitted to the external device 300 separately from the small row image 324. Since the row image 322 is larger in capacity than the small row image 324, the small row image 324 is first transmitted to the external device 300 and then the row image 322 is transmitted to the external device 300 . For example, the row image 322 may be transmitted to the external device 300 while the ISP 323 performs the correction for the row image 322. [ The raw image 322 may be uploaded to the external device 300 as generated by the image sensor 321 or may be uploaded with a pre-processed image for which lens distortion correction or noise removal has been performed. The above-described preprocessing may be performed in the external device 300. The external device 300 may perform demosaic processing or image format modification, or preprocessing to increase the image recognition rate. The ISP 333 of the external device 300 can correct the received row image 322. [ The external device 300 may correct the row image 322 using the correction region information 332 that has been generated in the past or may correct the row image 322 using the extended correction region information. The low image 322 may have a higher resolution than the small row image 324 so that the ISP 333 of the external device 300 can obtain more detailed correction region information from the high resolution image. The ISP 333 may generate the extended correction region information by using the generated correction region information and the raw image 322 together. The ISP 333 can acquire a high quality image 334 by correcting the row image 322 using the extended correction region information. The high resolution image 334 may be stored in the storage 335 of the external device 300 and downloaded to the electronic device 101. [

In one embodiment, electronic device 101 is a user device that receives a user request for image generation (hereinafter referred to as an " image generation request ") and sends information related to the image generation request to external device 300 have.

In one embodiment, the information related to the image generation request is information (hereinafter referred to as " image generation condition ") about a condition for an object to be included in an image to be generated (Or determined) information by the electronic device 101 based on the received information. In one embodiment, the image generation condition may be the information in which the image generation request is converted into a plurality of items and their corresponding property values.

In one embodiment, the external device 300 may be an electronic device that stores a plurality of images transmitted from a plurality of electronic devices, including the electronic device 101. For example, the external device 300 may be a cloud server. The plurality of images may be a low image 322 or a small low image 324. [

In one embodiment, the external device 300 receives an image generation condition from the electronic device 101, generates a request image based on the received image generation condition, and provides the request image to the electronic device 101 . The external device 300 can generate a request image using a plurality of images stored in the storage 335 of the external device 300. [

According to various embodiments, at least one configuration described as being performed in the electronic device 101 may be performed in the external device 300, and vice versa. For example, if the electronic device 101 is capable of using a plurality of images stored in the electronic device 101 or a plurality of images stored in the external device 300 connected to the electronic device 101, The request image may also be generated. As another example, the external device 300 other than the electronic device 101 may generate (or determine) an image generation condition based on the image generation request.

In one embodiment, the electronic device 410 may be an apparatus that performs operations to generate a request image based on image creation conditions. For example, the electronic device 410 may be the electronic device 101 shown in FIG.

Referring to FIG. 4A, the electronic device 410 may include a communication module 440, a processor 420, and a memory 430.

The communication module 440 may be electrically / functionally coupled to the processor 420 and may be configured to exchange data with the external device 450 under the control of the processor 420. For example, the electronic device 410 may transmit an image generation request to the external device 450 through the communication module 440 and receive the request image from the external device 450. [

The memory 430 may be electrically and / or functionally coupled to the processor 420 and may be configured to store data generated by the electronic device 410 or data received from the external device 450 under the control of the processor 420 have. For example, the memory 430 may store an image generation condition generated based on an image generation request or an image generation request. That is, the memory 430 may store a history of the image creation request or the image creation condition. As another example, the memory 430 may store the request image provided from the external device 450. [ As another example, the memory 430 may store a user profile that is used by the personal information transformation module 422 to transform personal information. As another example, the memory 430 may store a plurality of images taken by a user or downloaded via a network. At this time, the memory 430 may store at least one piece of information about the stored image, and at least one piece of information may include time information of the stored image, place information, weather information, object (e.g., Or the like. The person information may include at least one of relationship information with the user and / or information on the accuracy of the relationship information.

In one embodiment, the image stored in memory 430 may be in the form of raw data and may be encoded data (e.g., jpg).

In one embodiment, the image stored in the memory 430 may include a static image and a dynamic image.

The processor 420 may include at least one or more of the electronic devices 410 to perform the specified functions of the electronic device 410.

4A, the processor 420 may include an instruction analysis module 421, a personal information conversion module 422, and an ISP (image signal processor) 423.

The command analysis module 421 may determine the context information and image generation conditions based on the image creation request.

In one embodiment, the image creation request is a user request for image creation, and may include at least one description or representation (or user comment) to define the image to be created. For example, an image creation request may be of various types of input (e.g., text input, sound input, etc.) including user comments such as " I want to see a picture of my mother & .

In one embodiment, the context information may be information representing a variety of information that may be contained in the image creation request, the image being defined by a plurality of items and corresponding attribute values. The context information may include at least one of time information, place information, weather information, object information (e.g., person information), background information, and operation information. For example, the command analysis module 421, based on the request to create an image such as " I would like to see a picture of my mom and a girl who went to Seoul Park during the summer vacation of a junior high school, " (Time, person, place) and the corresponding property values (summer vacation in junior high school, mother and me, Seoul Park) can be determined.

In one embodiment, the attribute value may be one specific value or a specific range.

In one embodiment, the command analysis module 421 may include a natural language analysis module (not shown). The command analysis module 421 may use the natural language analysis result of the user comment included in the image generation request in determining the image generation condition based on the image generation request. Natural language analysis is performed by a natural language analysis module and can include lexical analysis, syntactic analysis, and semantic analysis. For example, the command analysis module 421 may use the results of natural language analysis of user comments, such as " I want to see a picture of my mom and sister who went to Seoul Park during the summer vacation of the junior high school, " The attribute value of the object information in the context information can be determined as " Mom and I "

In one embodiment, the image creation condition may be determined based on at least one part of the context information, or at least one description or representation included in the image creation request, as a condition for the object to be included in the request image.

In one embodiment, the command analysis module 421 may determine an image generation condition based on context information. The command analysis module 421 can process the situation information on the basis of the object to determine the image generation condition. For example, if the situation information is "Time - 1997.7.1 ~ 1997.8.31, Person - I, Mom, Location - Seoul Park", the command analysis module 421 reads "Seoul Park, 1997.7.1 ~ 1997.8.31" An image creation condition may be created as a condition for an object to be included in the request image, such as " Between 1997.1.1 to 1997.31.1.3 ", and " Mom from 1997.7.1 to 1997.8.31. &Quot;

In one embodiment, the image generation condition may include a condition regarding an operation on the object. For example, the conditions for image generation may include conditions relating to "I, 1997.7.1 ~ 1997.8.31, and the motion of mother holding hands of 1997.7.1 ~ 1997.8.31".

According to the above description, the image generation condition can be determined. However, when a part of the context information of the determined image generation condition includes personal information, an error occurs in a process to be performed later, that is, a process of determining an object to be included in the requested image or an image including an object to be included in the requested image . Thus, the processor 420 of the electronic device 410 needs to transform the personal information contained in the contextual information.

The personal information conversion module 422 converts the attribute value including the personal information of the user or the person related to the user among the plurality of items included in the image creation condition and the attribute values corresponding thereto, Can be converted to other attribute values that do not contain. Here, the personal information about a specific person may represent information limited only to a specific person, or information whose meaning differs depending on who the user is. For example, if the request to create an image includes a comment of "I want to see a picture of my mom and I went to Seoul Park during the summer vacation in junior high school," "Summer vacation in junior high school third grade" and "mom" The user may include personal information on the user, since the meaning may vary.

In one embodiment, the personal information conversion module 422 may detect that personal information about the user is included in at least one attribute value among a plurality of items included in the context information and corresponding attribute values. For example, the personal information conversion module 422 includes personal information in "Time - Summer Vacation in Third Year of Junior High School" among "Summer Vacation in Time - Junior High School Summer Vacation, People - Mom and Mom and Location - Seoul Park" Can be detected.

In one embodiment, the personal information transformation module 422 may use predetermined criteria in determining that personal information is included in the context information. The personal information conversion module 422 can determine whether the attribute value of a specific item can be converted into a standard unit or a reference unit of the corresponding item from the context information (i.e., a plurality of items and attribute values corresponding thereto) have. If the property value of a specific item can not be converted into a standard unit or a reference unit of the corresponding item, the attribute value may determine that the personal information about the user or the person related to the user is included. For example, the personal information conversion module 422 may determine that the attribute value (summer vacation of the junior high school) of a specific item (time) is a standard unit or a reference unit (i.e., year, month, , Minute, second), it can be determined that the personal information about the user or the person related to the user is included.

In one embodiment, the personal information transformation module 422 may transform an attribute value including personal information about the user into an attribute value that does not contain personal information, based on the user profile stored in the memory 430 .

In one embodiment, the user profile may be information describing a user, and may be reference information used to convert an attribute value including personal information into an attribute value that does not include personal information. For example, the user profile may include one of a user's face photograph or personal description (e.g., date of birth, social security number, home address, education, history, etc.).

For example, the personal information conversion module 422 uses the date of birth of the user included in the user profile to designate the attribute value including the personal information "junior high school third grade summer vacation" as "1997.7.1 to 1997.8.31" It can be converted to an attribute value that does not contain personal information.

When the attribute value including the personal information is converted into the attribute value that does not include the personal information, the personal information conversion module 422 can update the status information based on the converted attribute value. For example, the personal information conversion module 422 stores status information (" Time - Summer Vacation in Third Grade Middle School, Person - I, Mother, Place - Seoul Park " - 1997.7.1 ~ 1997.31 ", person - me, mother, place - Seoul Park") can be changed or renewed.

The ISP 423 may be the ISP 323 of FIG. The ISP 423 may obtain each object image from each of the plurality of images stored in the memory 430 or may acquire a plurality of object images from the external device 450 through the communication module 440. [ The ISP 423 can generate the request image by combining the acquired object images in one image. For example, the electronic device 410 may generate a request image by obtaining a plurality of object images from the external device 450 and synthesizing the plurality of acquired object images. Alternatively, the electronic device 410 may obtain the respective object images from at least one of the plurality of images stored in the memory 430 of the electronic device 410, synthesize the acquired object images, Images can be generated.

In one embodiment, when the ISP 423 obtains an object image based on a plurality of images stored in the memory 430, the ISP 423 may include a plurality of images stored in the memory 430, (Hereinafter referred to as " original image ") including an object image. The original image may be a low image 322 or a small low image 324. [ The ISP 423 can search for at least one original image including the object image among the plurality of images stored in the memory 430 based on the image creation condition. For example, the electronic device 410 may be configured to generate an image based on an image generation condition (" Seoul Park of 1997.7.1 ~ 1997.31 ", " Me of 1997.7.1 ~ (I.e., an image representing an object representing a Seoul park or an image including a subject) including an image of a Seoul park corresponding to 1997.7.1 to 1997.8.31 in the memory 430, An original image including the image of the user corresponding to the number of the users corresponding to the number of the users,

In one embodiment, the ISP 423 may obtain an object image to be included in the request image, from the obtained original images, if the ISP 423 has acquired the original images including the object image to be included in the request image. In one embodiment, the act of obtaining an object image to be included in the request image may include obtaining an image in the original image that represents the object to be included in the request image, by editing the original image. For example, the ISP 423 may acquire the correction area information 332 of the external device 300. [

In one embodiment, the ISP 423 may segment or crop the acquired original image to obtain an object image to be included in the request image from the acquired original images. For example, the ISP 423 may obtain at least one segmented image region from the small row image 324. [

In one embodiment, the ISP 423 determines a portion of the acquired image using the outline of the object to be included in the requested image, and crops or divides the determined portion of the image, Can be obtained. For example, if the ISP 423 has obtained an original image including an image of the object "Seoul Park 1997.7.20" in the memory 430, the ISP 423 may display an object or object (for example, The sign of Seoul Park can be used as a landmark, front gate, bronze statue, etc.) of the contour, "1997.7.20 Seoul Park" object image can be obtained.

In one embodiment, the ISP 423 may obtain the entire area of the image as an object.

In one embodiment, if the ISP 423 determines that the original images containing the object image to be included in the request image are not present in the memory 430, an additional search operation may be performed based on the attributes of the object to be included in the request image have. At this time, the attribute of the object may be an attribute indicating the sensitivity to time. For example, it can be considered that an object or a subject representing a Seoul park is an inanimate object which does not change much with respect to time. The ISP 423 may set the conditions for the time differently and perform additional search operations based on the otherwise set conditions. For example, if there is no original image including a Seoul park corresponding to 1997.7.1 ~ 1997.8.31, the ISP 423 may compare the time interval between 1997.7.1 ~ 1997.31 For example, July 1, 1998 to August 31, 1998).

In an embodiment, if an original image including an object corresponding to an image generation condition is not searched even though additional search is performed, the ISP 423 may directly generate an object corresponding to the image generation condition. For example, the ISP 423 may generate an object corresponding to an image generation condition using a vector drawing technique. In another example, the ISP 423 may create an object corresponding to an image generation condition using a machine learing technique. At this time, the learning data of the machine learning may be a plurality of images stored in the memory 430. For example, the ISP 423 may generate an object corresponding to an image generation condition using a feature extracted commonly from a plurality of images stored in the memory 430. [ If the image creation conditions include the condition of the object for me, mother, Seoul Park, 1997.7.1 ~ 1997.8.31, and the condition for the action that the mother and I are holding hands, If not present, the ISP 423 may create an object with which the mother grasps the hand, based on features commonly extracted from other images held in hand, stored in memory.

In one embodiment, the ISP 423 may generate the request image using the object images obtained or generated. In one embodiment, the ISP 423 may generate the request image by composing the object images that it has acquired or generated. In one embodiment, the ISP 423 may generate the request image by arranging or arranging the object images obtained or generated at different positions of one image. For example, the ISP 423 stores the image of the object " I " obtained from the first original image stored in the memory 430 and the image of the object " Seoul Park " obtained from the second original image stored in the memory 430 Can be placed in different areas of an image to generate a request image. The position at which the acquired object is placed in the request image can be determined based on the position of the acquired object in the original image. For example, the ISP 423 may use the correction area information 332 to correct the raw image 322 and thereby generate a corrected image (i.e., a request image). In another example, the ISP 423 may use the correction area information 332 to generate a new image that is not existing (i.e., a request image).

In another embodiment, the ISP 423 may generate a request image by overlaying one of the object images obtained or generated to another object image. For example, the ISP 423 places an image of the object " Seoul Park on August 1, 1997 " in a specific area of the same image, Park " image. &Lt; / RTI >

In one embodiment, the ISP 423 may process the object images obtained or generated, in consideration of unity or harmony with other object images to be placed, before arranging the object images obtained or generated. In one embodiment, the ISP 423 may adjust properties (e.g., white balance, noise, color saturation, size, etc.) associated with the display of the object images obtained or generated. For example, if the object image is a monochrome image and the object image is a color image, the color image of "1997.7.20" can be processed in black and white. have. In another example, the object image processing may include at least one of white balance adjustment, color adjustment, noise reduction, sharpening, and detail enhancement.

In another embodiment, the ISP 423 may arrange object images obtained or generated, and then perform processing to reduce discomfort between the disposed object images. For example, the ISP 423 may perform a blurring process on a boundary surface (or a boundary line) of each object image in an image in which placement is completed, thereby suppressing a discomfort that may occur at the boundary between different object images . In another example, the boundary (or boundary) at which two or more objects are touched may be subjected to a refinement process to smoothly process the boundary (or boundary). In another example, if there is a difference in depth between the object images or a space-based posterior relationship between the object images in the batch-completed image, based on the PSF characteristics of the lens according to the distance from the focus, A blur or a deconvolution process can be performed on the object image. This process may be performed before the arrangement of the object images and in the preprocessing process.

Processor 420 may provide the user with a request image generated by ISP 423. For example, the processor 420 may control the display (not shown) to output the generated request image to the screen.

Although not shown in FIG. 4A, the electronic device 410 may further include additional configurations. For example, the electronic device 410 may further include an input module (e.g., a keyboard, a mouse, a microphone, etc.) or an output module (e.g., a display).

In various embodiments of the present invention, the operation of generating a request image based on an image generation condition may be performed in the electronic device 410 or the external device 450. Thus, the external device 450 may include at least one module for generating a request image based on an image generation condition. For example, the external device 450 may receive an image generation request from the electronic device 410 and generate a request image based on the received image generation request. The external device 450 may include at least one module for generating a request image based on the received image creation request.

In one embodiment, the external device 450 may be an external device 300.

In one embodiment, the external device 450 may include a communication module 440-1, a processor 420-1, a communication module 440-1, a processor 420-1, ) And the processor 420, respectively. The processor 420-1 may include an ISP 423-1 and the ISP 423-1 may be an ISP 333.

In one embodiment, the external device 450 may include a memory 430-1. The memory 430-1, like the memory 430, can store a plurality of images. In one embodiment, the memory 430-1 may store data generated in the external device 450 or data transmitted (or uploaded) in a plurality of electronic devices, including the electronic device 410. [

In one embodiment, the memory 430-1 may include two regions that are distinct from each other. At this time, the two regions may be the first region 431 and the second region 432. In one embodiment, the first area 431 and the second area 432 may be physically or by access authority.

In one embodiment, the first area 431 of the memory 430-1 may be a storage space designated by a particular person (e.g., a user) or dedicated to a particular person. For example, if the first area 431 is assigned or dedicated to person A, the first area 431 may contain only information (e.g., an image) transmitted by person A (or by an electronic device of person A) Can be stored. For example, when the first area 431 is designated or dedicated to the person A, the first area 431 may include personal information of the person A (e.g., a user profile of the person A) For example, an image including the face of person A). In another example, the first area 431 includes a first area A (not shown), a first area B (not shown), a first area A is designated or dedicated to person A, B is designated or dedicated to person B, the first area B can store information including personal information or personal information of person A, with the consent of person A. [

In one embodiment, the second area 432 of the memory 430-1 may be a storage space that is not designated or dedicated to a particular person. For example, the second area 432 may be a storage area that is not included in the first area 431. For example, the second area 432 may store an image captured by a user not registered in the external device 300, or an image that can not specify the photographer. As another example, the second area 432 may store information including personal information or personal information of a particular person (e.g., a user) under the consent of a particular person.

In one embodiment, the ISP 423-1 may determine the search area of the memory 430-1 based on the attributes of the object to be included in the request image. The attribute of the object to be included in the request image may be an attribute indicating whether the user or the person related to the user includes personal information. For example, if the object image to be included in the request image is " Seoul Park, 1997.7.1 ~ 1997.31 ", " Seoul Park " is independent of the personal information of the person related to the user or user, In the second area 432 of the memory, the original images. In another example, if the object image to be included in the request image is " I (or a user) from 1997.7.1 to 1997.8.31 ", the object includes the user's personal information 1 may retrieve original images containing object images to be included in the request image in the first area 431 of the memory. The ISP 423-1 searches the images stored in the first area 431 or the second area 432 of the memory and acquires original images including an object image to be included in the request image according to the search result, It can be determined that the original images including the objects to be included in the memory 430-1 are not present in the memory 430-1.

In one embodiment, the ISP 423-1 obtains prior consent from a specific person (e.g., a user) specified in the first area 431 before retrieving the object to be included in the request image in the first area 431 of the memory can do.

In one embodiment, the processor 420-1 may control the communication module 440-1 to send the request image to the electronic device 410. [

In operation 510, the processor 420 may receive an image creation request. In one embodiment, the processor 420 may receive an image creation request through an input module (not shown). The image creation request is a user request for image creation, and may be various types of input such as text input, voice input, touch input to a menu included in the user interface provided in the display. For example, the image creation request may be a user input for inputting text such as " I want to see a picture of my mom and sister who went to Seoul Park during the summer vacation of junior high school " in the input window provided on the screen.

In operation 520, the processor 420 may determine image generation conditions. In one embodiment, the processor 420 may determine an image generation condition based on the image generation request received in operation 510. [ The image generation condition may be determined based on the plurality of items and the context information indicating the corresponding attribute value. For example, the processor 420 may be configured to determine whether the image is to be generated based on the image creation request or status information received in operation 510 (see " Seoul Park, 1997.7.1 ~ 1997.31, 1997, Quot; mother " of July 1, 1997 to March 1, 1997), the image creation condition for the object to be included in the request image can be determined. As another example, the processor 420 may determine the image generation conditions ("93 years of age", "95 years of Seoul park") for different objects with different time conditions.

In operation 530, the processor 420 may retrieve an image that satisfies the image creation condition. The processor 420 may search the memory 430 for an image that satisfies the image creation condition or send a search request for the image that satisfies the image creation condition to the external device 450. [

In one embodiment, the image satisfying the image generation condition may represent an image including all the objects having respective specific conditions included in the image generation condition.

If an image satisfying the image creation condition is retrieved, in operation 570, the processor 420 may provide the retrieved image.

If an image satisfying the image creation condition is not found, the processor 420 can check whether the original images satisfying a part of the image creation condition are retrieved. For example, if the image generation conditions are ("Seoul Park of 1997.7.1 ~ 1997.8.31," "I of 1997.7.1 ~ 1997.8.31", "Mom of 1997.7.1 ~ 1997.8.31"), (420) is an original image including an object image of "Seoul Park of 1997.7.1 ~ 1997.8.31, 1997", and an original image including an object image of "I of 1997.7.1 ~ 1997.8.31", 1997.7.1 ~ 1997.8 Quot ;, " mother of .31 ", respectively.

If original images that satisfy some of the image creation conditions are retrieved, the processor 420 may, in operation 550, generate an image that satisfies the image creation conditions. An image satisfying the image generation condition can be described as a request image below.

In one embodiment, the processor 420 may obtain an object image to be included in the request image from the retrieved original image. In one embodiment, the act of obtaining an object image to be included in the request image may mean obtaining an image (object image) representing the object to be included in the request image, by dividing or cropping the original image.

In one embodiment, the processor 420 may generate the request image using the acquired at least one object image. For example, if the processor 420 has acquired a first object image and a second object image to be included in the request image, the request image may be generated by combining the first object image and the second object image in one image . Specifically, the processor 420 may generate a request image by arranging or arranging the first object image and the second object image at different positions of one image.

In operation 560, the processor 420 may provide the generated request image. In one embodiment, the processor 420 may control the display to display on the screen the request image generated in operation 550. [

Although not shown, the processor 420 may receive user feedback on the provided request image. In one embodiment, the processor 420 may receive user feedback via an input module or interface displayed on the screen. For example, processor 420 may provide a request image in operation 560 and then control the display to display a user interface for receiving feedback or evaluation (e.g., a rating) of the user, Lt; / RTI > User evaluation or feedback on the requested image may be used to update attributes (e.g., accuracy, preference, etc.) for the original image of the object image (e.g., first object image or second object image) contained in the request image . For example, if a first object image is obtained from a first original image stored in the memory 430 of the electronic device 410, the processor 420 may determine, from among various attributes of the first original image, The memory 430 may be controlled to update the value of at least one attribute.

According to various embodiments of the present invention, at least some of the operations described in Figure 5 may be performed by the processor 420-1 of the external device.

In one embodiment, when 510 operation is performed by processor 420-1 of an external device, processor 420-1 receives an image generation request from electronic device 410 via communication module 410-1 can do.

In one embodiment, when the 530 operation or the 540 operation is performed by the processor 420-1 of the external device, the processor 420-1 may compare the images stored in the memory 430-1 of the external device 450 , It is possible to retrieve an image satisfying at least a part of the image generation condition.

In one embodiment, when the 530 operation or the 540 operation is performed by the processor 420-1 of the external device, the processor 420-1 determines whether the 530 operation or the 540 operation is performed by the processor 420-1 of the external device 450 based on the attribute of the object to be included in the request image It is possible to determine the search area of the search area 430-1. The attribute of the object may be an attribute indicating whether the user or the person associated with the user includes personal information.

6 is a detailed flow diagram of operation (operation 550) of generating an image that satisfies an image generation condition (operation 540) of retrieving an image that satisfies part of an image generation condition according to various embodiments of the present invention to be.

In operation 610, if an image satisfying a part of the image generation condition is retrieved, the processor 420 can check whether all the retrieved images have been retrieved from the electronic device 410. [

If some of the retrieved images have been retrieved from the memory 430-1 of the external device 450 and not from the memory 430 of the electronic device 410, the processor 420 causes the external device 450 to retrieve some of the retrieved images Original image), or a request for an object image included in the retrieved partial image.

In operation 630, the processor 420 may obtain at least one object image to be included in the request image. For example, if all of the retrieved images are retrieved from the electronic device 410, the processor 420 may obtain at least one object image by segmenting or cropping each retrieved image. In another example, when a request for an object image included in some of the searched images is transmitted to the external device 450, the electronic device receives the object image requested from the external device 450 so that at least one An object image can be acquired. In another example, when receiving some images retrieved from the external device 450, the electronic device may obtain object images by dividing or cropping the received images.

Operation 640 is similar to operation 550, and thus a detailed description thereof will be omitted.

7 is a cross-sectional signaling diagram of an electronic device and an external device for providing an image satisfying an image generation condition according to various embodiments of the present invention.

FIG. 7 illustrates a cross-signal flow based on an embodiment in which the operation of generating a request image based on an image generation condition is performed by an external device 450. FIG.

In operation 710, the processor 420 of the electronic device 410 may receive an image creation request through an input module. 710 operation may correspond to 510 operation.

In operation 720, the processor 420 of the electronic device 410 may control the communication module 440 to send an image creation request received from the user to the external device 450 in 710 operation.

In operation 730, the processor 420-1 of the external device 450 may determine the image creation conditions based on the image creation request received from the electronic device 410 in 720 operation.

In operation 740, the processor 420-1 of the external device 450 may obtain at least one object to be included in the request image.

In one embodiment, the processor 420-1 of the external device 450 may determine the search area of the processor 420-1 of the external device 450 based on the attributes of the object to be included in the request image. The attribute of the object may be an attribute indicating whether the user or the person associated with the user includes personal information. For example, if the object associated with the object image to be included in the request image includes personal information of the person associated with the user or user, the processor 420-1 may search the first area 431 of the memory.

In one embodiment, the processor 420-1 of the external device 450 retrieves, in the search area of the determined memory 430, an original image containing the object to be included in the request image, and based on the retrieved original image, The object image can be obtained.

In operation 750, processor 420-1 of external device 450 may generate a request image. In one embodiment, the processor 420-1 of the external device 450 may generate the request image by compositing the acquired at least one object image into one image.

In operation 760, the processor 420-1 of the external device 450 may send the generated request image to the electronic device 410. In one embodiment, if the request image can not be generated because the original image including the object image to be included in the request image is not retrieved, the processor 420-1 of the external device 450 can not generate the request image And the cause of the failure.

In one embodiment, processor 420-1 of external device 450 may store the generated request image in memory 430-1.

In one embodiment, instead of sending the request image to the electronic device 410, the processor 420-1 of the external device 450 may request the request image from the electronic device 410 based on the recognition of the retrieved original image And to transmit the generated processing information to the electronic device 410. The electronic device 410 may be a personal computer, At this time, the electronic device 410 can generate the request image using the received process information.

The processor 420-1 of the external device 450 may send the processing results to the electronic device 410 after processing for a portion of the request image, instead of sending the request image to the electronic device 410. In one embodiment, Can be transmitted. At this time, the electronic device 410 may, after processing for another portion of the request image, combine with the received result to produce the final result (i.e., the request image).

In operation 770, the processor 420 of the electronic device 410 may provide the received request image to the user. In one embodiment, the processor 420 of the electronic device 410 may receive user feedback or evaluation of the provided request image.

In operation 780, the processor 420 of the electronic device 410 may send the received user feedback or evaluation to the external device 450.

In operation 790, the processor 420-1 of the external device 450 may update the attributes of the image based on the received user feedback or evaluation. For example, processor 420-1 of external device 450 may update an attribute value (e.g., preference) for the original image of the object image that was used to generate the request image based on the received user feedback have.

8 is a cross-sectional signaling diagram of an electronic device and an external device for providing an image satisfying an image generation condition according to various embodiments of the present invention.

Figure 8 illustrates a cross-signal flow based on an embodiment in which the operation of generating a request image based on an image generation condition is performed by the electronic device 410. [

In operation 810, the processor 420 of the electronic device 410 may receive an image creation request. For example, the processor 420 of the electronic device 410 may receive a textual input such as " I would like to see a picture of my mom and sister who went to Seoul Park during the summer vacation of junior high school. &Quot;

In operation 820, the processor 420 of the electronic device 410 may determine image generation conditions. For example, the processor 420 of the electronic device 410 may be configured to receive a request for an image based on context information extracted from an image creation request (e.g., " Seoul Park, 1997.7.1 ~ 1997.31, 1997 ", "Quot;, " me ", and " mother of July 1, 1997 to March 31, 1997 ").

In operation 830, the processor 420 of the electronic device 410 may request the external device 450 for an image (object image) for the determined object. Here, the object image requested to the external device 450 may be a partial area that is divided or cut out from the image (original image) different from the request image.

In one embodiment, the processor 420 of the electronic device 410 may request the object image by sending data associated with the object image request to the external device 450. [

As described above, the data related to the object image request may include information on the object and condition information on the object. In one embodiment, the information about the object may include identification information of the object, and the condition information for the object may include information on various conditions for defining the object, for example, time information, place information, have. For example, in operation 830, the electronic device 410 may request an object image for " Mother 1997.3.1 ~ 1997. 8.31. &Quot;

In one embodiment, the processor 420 of the electronic device 410 may search the memory 430 of the electronic device 410 before requesting the external device 450 to image an object to be included in the request image. For example, if the image of the object to be included in the request image can be retrieved from the memory 430 of the electronic device 410, the processor 420 of the electronic device 410 may request the external device 450 to image the object I can not. In another embodiment, if the image of the object to be included in the request image includes a first object image and a second object image and the first object image can be retrieved from the memory 430 of the

electronic device

410, 410 may request only the second object image to the external device 450.

In operation 840, the processor 420-1 of the external device 450 may obtain the image of the requested object by retrieving from the memory 430-1 of the external device 450. [

In one embodiment, the processor 420-1 of the external device 450 may retrieve the original image from the memory 430-1. In one embodiment, the processor 420-1 of the external device 450 may obtain an image of the requested object by dividing or cropping the image of the requested object in the retrieved original image.

In one embodiment, the processor 420-1 of the external device 450 may include a first area 431 and a second area 432. The processor 420-1 of the external device 450 can determine an area to be searched in the memory 430 based on information (e.g., object identification information) about the object included in the data related to the object image request. For example, when the external device 450 receives data requesting an image for "mother" of "1997.7.1 ~ 1997.8.31", the processor 420-1 of the external device 450 requests It is determined that the object of the image is an object related to the user's personal information and the area to be searched by the memory 430 is determined as the first area 431. [

In operation 850, the processor 420-1 of the external device 450 may send an image of the acquired object to the electronic device 410.

In one embodiment, if the processor 420-1 of the external device 450 did not acquire the requested object image in the 850 operation, then the processor 420-1 may determine that the memory 430- 1) to the electronic device 410 that the requested object image could not be acquired.

In one embodiment, when the processor 420-1 of the external device 450 fails to acquire the object image requested in the 850 operation, the object image that satisfies the time condition of the image creation condition (first time condition) , But may send a message to the electronic device 410 that an object image satisfying the second time condition can be obtained.

In operation 860, the processor 420 of the electronic device 410 may generate a request image using the received object image and provide the generated request image to the user.

In operation 870, the processor 420 of the electronic device 410 may send user feedback to the external device 450 for a request image displayed or provided on an output module (e.g., a display) of the electronic device.

In operation 880, the processor 420-1 of the external device 450 may update at least one image attribute based on the received user feedback. In one embodiment, the processor 420-1 of the external device 450 may update the attributes for the original image. For example, when the image of the requested object is divided or cropped in the first image stored in the memory 430-1 of the external device 450, the processor 420-1 of the external device 450 determines whether the first image The memory 430-1 of the external device 450 can be controlled to change the value of the attribute (e.g., accuracy, preference)

9A, when receiving a user input for executing a specific function or application from the user, the electronic device 410 may display a user interface for receiving an image generation request from the user on the screen. For example, when an electronic device receives a user input (e.g., touching an application icon) to execute a " memorable image creation application ", a user interface for receiving an image creation request from the user may be displayed on the screen .

In one embodiment, the user interface for receiving the image creation request may include a text entry window 910. [ For example, the electronic device 410 can display a text input window 910 for inputting a user's image description together with a message saying " Please describe a desired image ". At this time, the electronic device 410 can receive the image description of the user through the single text input window 910, such as " I want to see a picture that went to Seoul Park with my mom at the third grade summer vacation. &Quot;

9B, the user interface for receiving an image generation request may include a plurality of menus 920 to 940, respectively, corresponding to the status information. For example, the electronic device 410 may display menus 920 to 940 corresponding to time information, place information, and person information on a screen. At this time, when the user touches the menu 920 corresponding to the time information, the electronic device 410 displays an input window 950 for receiving a user description of the time information and a " Can be displayed together. At this time, the electronic device 410 can receive the text " Summer Vacation of the Middle School Third Grade " from the user through the input window 950 for receiving the time information. That is, the electronic device 410 may provide a user interface for correctly receiving each condition for the request image.

10 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to determine an image generation condition. FIG. 10 is a detailed flowchart of operation 520 of FIG.

In operation 1010, the processor 420 of the electronic device 410 may determine context information. In one embodiment, the processor 420 of the electronic device 410 may determine the context information based on the analysis result of the image creation request. In one embodiment, an image creation request may be received from the user before the 1010 operation is performed. In one embodiment, the image creation request may be received from the user via one of the various user interfaces described in FIG. 9A or FIG. 9B.

In one embodiment, the context information may be information representing a variety of information that can define an image, as a plurality of items and their corresponding attribute values. For example, the situation information may include at least one of time information, place information, weather information, object information (e.g., person information), and background information. In one embodiment, the time information may be time related information among the information included in the image creation request. Here, the information related to the time may be information related to the age of the object to be included in the image, or the shooting time.

In one embodiment, the electronic device 410 may receive an image creation request via the user interface described in FIG. 9A. For example, referring to FIG. 11, the electronic device 410 may receive a user's text input 1110 (" I want to see a picture of my mother and I went to Seoul Park during the third grade summer vacation ") through the user interface described in FIG. As an image creation request.

In one embodiment, when the electronic device 410 receives the image creation request via the user interface described in FIG. 9A, the processor 420 of the electronic device 410 may perform a natural language analysis Alternatively, separate analysis results such as machine learning can be utilized. For example, the processor 420 of the electronic device 410 may receive, from the image generation request 1110, status information including time information and two pieces of object information (e.g., 1120).

In one embodiment, when the electronic device 410 receives an image creation request via the user interface described in FIG. 9B, the processor 420 of the electronic device 410 receives the image creation request from the user < RTI ID = Based on the input, context information including time information and object information can be determined. For example, the processor 420 of the electronic device 410 may determine a user input (e.g., a summer vacation in the third year of junior high school) input to the input window 950 corresponding to the time information as the attribute value of the time information .

In operation 1020, the processor 420 of the electronic device 410 may determine whether personal information is included in the determined status information. In one embodiment, the personal information may refer to information limited only to a specific person, or information whose meaning varies depending on who the user is.

If it is determined that personal information is not included in the determined contextual information, then in operation 1040, the processor 420 of the electronic device 410 may determine image creation conditions based on contextual information. In one embodiment, the image creation condition may be a collection of all determined context information. In another embodiment, the image generation condition may be information obtained by converting the determined situation information based on the object.

If it is determined that the personal information is included in the situation information, then in operation 1030, the processor 420 of the electronic device 410 converts at least some of the personal information into information (hereinafter referred to as " public information &Quot;). &Lt; / RTI > The processor 420 of the electronic device 410 may convert at least some of the personal information 1140 of the personal information 1130 from the context information 1120. [

In one embodiment, at least some of the context information that includes the personal information may be converted to public information based on the user profile stored in the memory 430.

In one embodiment, the processor 420 of the electronic device 410 may not convert it into public information, even if it is context information that includes personal information. For example, the processor 420 of the electronic device 410 may not convert person information (e.g., a face image of a person associated with a user or a user) into public information. The reason for this is that the person information is information that should be essentially included in the image (request image) that the user wants to create. If the person information is converted into public information (e.g., mosaic processing) This is because it is difficult.

In one embodiment, the processor 420 of the electronic device 410 may update status information by converting at least some status information including personal information into public information. For example, the processor 420 of the electronic device 410 may convert the time information including the personal information (1140, third-year summer vacation) into public information (1997.7.1 to 1997.8.31).

In operation 1050, the processor 420 of the electronic device 410 may determine an image generation condition based on the updated status information. For example, the processor 420 of the electronic device 410 may determine updated condition information as an image generating condition.

By performing the above-described operations in the above-described order, the electronic device 410 can determine the image generation conditions based on the image generation request received from the user. In another embodiment, the external device 450 may determine the image generation conditions based on the image generation request received from the electronic device 410. [ At this time, the user profile used to convert at least some of the context information including personal information into public information may be stored in the first area 431 of the memory of the external device 450. [

12 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to obtain an object image to be included in a request image. FIG. 12 is a detailed flowchart of operation 630 of FIG.

In operation 1210, the processor 420 of the electronic device 410 may search the memory 430 of the electronic device 410 for the original image containing the object of the first condition, or may retrieve the original image including the object of the first condition To the external device 450. The external device 450 transmits the search request to the external device 450. [ In one embodiment, the first condition may be a condition included in the image generation condition. For example, the processor 420 of the electronic device 410 may send an original image containing an object of a first condition (e.g., " Mother of 1997.7.1 ~ 1997.8.31 ") included in the image creation condition to the electronic device 410 in the memory 430.

The processor 420 of the electronic device 410 determines whether the original image containing the object of the first condition is stored in the memory 430 of the electronic device 410 or in the memory 430-1 of the external device 450 It can be determined whether or not it exists. If the processor 420 of the electronic device 410 determines that the original image including the object of the first condition is not present in the memory 430 of the electronic device 410 or the memory 430-1 of the external device 450 If judged, in operation 1230, the original image including the object of the second condition can be retrieved from the memory 430 of the electronic device 410 or the memory 430-1 of the external device 450. If the processor 420 of the electronic device 410 judges that the original image including the object of the first condition exists in the memory 430 of the electronic device 410 or the memory 430-1 of the external device 450 The processor 420 of the electronic device 410 may acquire the original image in the 1250 operation and obtain the object image of the first condition by dividing or cropping the object image from the acquired original image.

In one embodiment, the second condition may include a first condition, and the object of the first condition and the object of the second condition may represent the same entity. For example, the object of the first condition may be "mother of 1997.7.1 ~ 1997.8.31" and the object of the second condition may be "mother of 1996.7.1 ~ 1998.8.31". That is, when the object image of the specific condition according to the image generation condition can not be obtained, by alleviating the specific condition, the object which is not perfectly matched with the condition presented by the user but is related to the condition (or similar condition) You can acquire an object.

In one embodiment, the second condition may be determined according to an attribute of the object. In one embodiment, the attribute of the object may be an attribute that indicates sensitivity to time. For example, if the object is a " Seoul Park ", and the first condition is associated with a time period of " 1997.7.1 to 1997.8.31 ", then consider the fact that the object is inanimate, The processor 420 may determine the second condition with a relatively wide time interval (1996.7.1 to 1998.8.31). For example, if the object is a "mom" and the first condition is associated with a time period of "1997.7.1 to 1997.8.31", the processor of the electronic device 410 420) can determine the second condition with a relatively narrow time interval (1997.5.1 ~ 1997.10.31).

In operation 1240, the processor 420 of the electronic device 410 determines whether the original image including the object of the second condition exists in the memory 430 of the electronic device 410 or in the memory 430-1 of the external device Or not. If it is determined that the original image including the object of the second condition exists in the memory 430 of the electronic device 410 or the memory 430-1 of the

external device

450, 420 can obtain an object image of a second condition by obtaining an original image including an object of the second condition and dividing or cropping the object image from the acquired original image in 1250 operation.

If it is determined that the original image including the object of the second condition is not present in the memory 430 of the electronic device 410 or the memory 430-1 of the

external device

450, 410 may provide a notification to the user that the request image corresponding to the image creation request can not be generated. The notification that the request image can not be generated may include a reason why the request image can not be generated. For example, the processor 420 may notify the user that the request image can not be generated because the original image including " Mother 1997.7.1 ~ 1997.8.31 " can not be obtained.

In one embodiment, the first area 431 of memory may store data associated with a particular person ' s personal information. For example, the first area 431 of the memory may store images of the user or a person (e.g., a user's family or friend) and image information associated with the user. At this time, the information of the image is obtained through the additional information of the user such as the information that can be obtained from the image metadata such as the shooting time and the shooting location, information that can be obtained by analyzing the image such as object information, accuracy, Or information that can be received.

In one embodiment, the first area 431 of the memory may be divided into a plurality of areas, each of which may be assigned to a user or a person (e.g., a mother) associated with the user, respectively.

In one embodiment, the first area 431 of the memory may be a storage space designated for a particular person (e.g., a user).

In one embodiment, the first area 431 of memory may not be allowed access according to an image creation request of a person different from the specific person, unless the specified person's consent is given.

One image 1301 stored in the first area 431 of the memory is illustrated. The image 1301 may include a plurality of objects including a person 1302 associated with the user or user.

The information 1303 of the image 1301 is illustrated. The information 1303 of the image 1301 may include a plurality of items and corresponding attribute values. The plurality of items may include information such as time 1310, location 1320, accuracy 1330, object information 1340, weather 1350, time sensitivity 1360, preferences 1370, destination information 1380, 1390), and the like, and each item can correspond to each attribute value.

In one embodiment, the information 1303 of the image 1301 may be provided by the user or may be determined based on the metadata of the image or the machine learning learning result in the external device 450. [

In one embodiment, time 1310 and location 1320, weather 1350, and destination information 1380 provide information about the time, location, weather, and shooting purpose (e.g., travel, etc.) Respectively.

In one embodiment, the accuracy 1330 may correspond to information indicating how accurate the object information 1340 is.

In one embodiment, the object information 1340 may correspond to information indicating who the object 1302 contained in the image 1301 is or has a relationship with the user. For example, the object information 1340 may correspond to the identifier C indicating who the object included in the image 1301 is.

In one embodiment, the preferences 1370 may correspond to information indicating how satisfactory the user was when the requested image was synthesized using the image 1301.

In one embodiment, the upload information 1390 may correspond to information indicating how the image 1301 is stored in the first area 431 of the memory.

In one embodiment, the second area 432 of memory may store data that is independent of a person ' s personal information. For example, the second area 432 of the memory may store images and images of facilities (e.g., commercial facilities, public facilities, etc.) together. At this time, the information of the image may include at least one of the information obtained through additional information of the user such as information to be confirmed from the metadata of the image such as a photographing time and a photographing place, accuracy, preference and the like.

In one embodiment, one image 1480 stored in a second area 432 of memory is illustrated. The image 1480 may include a plurality of

objects

1481, 1482 independent of the person associated with the user or user.

In one embodiment, information 1483 of image 1480 is illustrated. The information 1483 of the image 1480 may include a plurality of items and corresponding attribute values. The item included in the information 1483 may be the same item as or different from the item included in the information 1303 disclosed in FIG. For example, time 1410, location 1420, accuracy 1430, object information 1440, weather 1450, time sensitivity 1460, upload information 1470, Item. &Lt; / RTI >

Some of the items included in the information 1303 of the image 1301 disclosed in Fig. 13 may not be included in the information 1483 of the image 1480. [ For example, the destination information 1380 may not be included in the information 1483.

15 is a detailed flowchart of an operation for an electronic device to generate a request image according to various embodiments of the present invention. FIG. 15 is a detailed flowchart of operation 640 of FIG.

In operation 1510, the processor 420 of the electronic device 410 may determine whether it has acquired all object images to be included in the request image. In one embodiment, the processor 420 of the electronic device 410 receives all of the object images to be included in the request image, via at least one of a search of the memory 430 of the electronic device 410 or a request to the external device 450 Is obtained. For example, if all the object images to be included in the request image are the first object image and the second object image, the first object image is obtained through the search of the memory 430 of the electronic device 410, The processor 420 of the electronic device 410 may determine that it has acquired all the object images to be included in the request image if the second object image is obtained from the external device 450 after requesting the second object image with the request .

In operation 1510, if the processor 420 of the electronic device 410 determines in 1520 operation that it has failed to acquire all object images to be included in the requested image, the object image that failed to acquire may be generated. For example, if the processor 420 of the electronic device 410 determines that the first object image has not been acquired (or fails to retrieve the original image including the first object image), the processor 420 of the electronic device 410 You can create it yourself. For example, the processor 420 of the electronic device 410 may generate a first object image using a vector drawing technique. In another example, the processor 420 of the electronic device 410 may directly generate the first object using a machine learning technique.

In one embodiment, the processor 420 of the electronic device 410 may generate an object image that failed acquisition, by interpolating previously stored images in the memory 430. Details will be described later with reference to FIG.

In operation 1510, if the processor 420 of the electronic device 410 determines that it has acquired all of the object images to be included in the request image, or if the object image fails to acquire in the 1520 operation, the processor 420 of the electronic device 410 acquires A request image can be generated using one object images.

In one embodiment, the processor 420 of the electronic device 410 may generate a request image by compositing (or combining) object images obtained (or generated).

In one embodiment, the processor 420 of the electronic device 410 may synthesize object images obtained (or generated) by arranging (or creating) the object images obtained at different positions of one image . For example, the processor 420 of the electronic device 410 may compare a first object image (" me in 1997.7.20 ") with a second object image in the image, such that the first object is seen as being ahead of the second object And a second object image (" Seoul Park, 1997.8.1 ") can be placed in the upper part of the same image so that the first object is seen as being ahead of the second object.

In another embodiment, the processor 420 of the electronic device 410 may arrange the acquired objects so that one of the object images acquired (or created) overlaps another object image.

16 is a detailed flow diagram of an operation for an electronic device to generate an object image that fails acquisition in accordance with various embodiments of the present invention. FIG. 16 is a detailed flowchart of operation 1520 of FIG.

In operation 1610, the processor 420 of the electronic device 410 may determine whether an object image that satisfies the time condition does not exist. In other words, the processor 420 of the electronic device 410 can confirm why the acquisition failed if there is an object image that failed to acquire. For example, if the image generation condition is " a mother of 1997.7.1 to 1997.8.31, " the processor 420 of the electronic device 410 may generate an image including an object image representing " mom " Is not searched, or an image containing an object image representing " mother " is searched, but it can be checked whether or not it does not meet the time condition. The process of the processor 420 of the electronic device 410 to determine whether there is no object satisfying the time condition in operation 1610 may be a second example of the two examples described above.

In operation 1610, if it is determined that there is no object image satisfying the time condition, the processor 420 of the electronic device 410 represents the same entity in the 1620 operation, It is possible to check whether or not the images are present in the memory 430 or the memory 430-1. For example, if it is not possible to retrieve an object image representing " mother of 1997.7.1 ~ 1997.8.31.31 ", the processor 420 of the electronic device 410 may compare the object image representing "Quot; mother of .29 " is present in the memory 430 of the electronic device 410. < RTI ID = 0.0 >

In one embodiment, the processor 420 of the electronic device 410 searches the memory 430 of the electronic device 410, or searches the memory 430 of the external device 450 By further requesting a search, it is possible to determine whether there are images taken before and after.

In another embodiment, the processor 420 of the electronic device 410 receives (at 860) an object image from the external device 450 prior to generating the request image, determining whether images taken before and after As shown in FIG.

If it is determined that there are images taken before and after, the processor 420 of the electronic device 410 may generate an object image that satisfies the time condition based on the images taken before and after, in the 1630 operation .

In one embodiment, the processor 420 of the electronic device 410 may interpolate previously and subsequently photographed images to produce an object image that satisfies the time condition. For example, the processor 420 of the electronic device 410 may grow within a specific time period, based on changes in the object image representing " mother of 1989.3.4 " and object image representing " mother of August 29, Or determine the image change of a person due to aging. And the processor 420 of the electronic device 410 may estimate and generate an image of a person at a particular time or time period (e.g., 1997.7.1 to 1997.8.31) included within a particular time period.

Although not shown, the processor 420 of the electronic device 410 may use one of the images taken before and after it to create an object image that satisfies the time condition. In one embodiment, the processor 420 of the electronic device 410 pre-stores information about changes in the general aging (or growth) pattern of a human face, and uses one of the images taken before and after An object image satisfying a time condition can be generated. For example, the processor 420 of the electronic device 410 may determine, based on the object image representing " mother of 1983.3.4 " and the general aging pattern change information of the human face, Can be estimated and generated.

Although not shown, the processor 420 of the electronic device 410 may, in a 1630 operation, generate an object image that satisfies a time condition, using at least two images photographed at times different from the time conditions. That is, the processor 420 may use at least two images previously photographed based on the temporal condition, or may generate an object image satisfying the temporal condition using at least two images photographed later based on the temporal condition . For example, the processor 420 obtains an object image representing " Mother of 1991. 6.10 " and an object image representing " Mother of 1995.1.20 ", and using the acquired object images, Can be determined. The processor 420 may then estimate and generate an image of the mother at a particular time or time period (e.g., 1997.7.1 to 1997.8.31) based on the determined image change of the person.

Figure 17 illustrates one example of generating a request image in accordance with various embodiments of the present invention. 17 may be performed in the electronic device 410 or the external device 450, but for convenience of description, the embodiment disclosed in FIG. 17 will be described as being performed in the external device 450. FIG.

The processor 420-1 of the external device 450 can search the memory 430-1 for the original image 1703 including the object image to be included in the request image. The memory 430-1 may include a first area 1702 and a second area 1701. The first area 1702 may correspond to the first area 431 and the second area 1701 may correspond to the second area 432. [

In one embodiment, the processor 420-1 retrieves two

original images

1704, 1705 that contain objects related to personal information in a first area 1702 of the memory 430-1, The original image 1703 including the object irrelevant to the personal information can be searched in the second region 1701 of the first region 170-1.

The processor 420-1 of the external device 450 can acquire an object image to be included in the request image, based on the retrieved original image. In one embodiment, the act of obtaining an object image to be included in the request image may include obtaining an object image to be included in the request image by dividing or cropping the original image. For example, the processor 420-1 of the external device 450 may obtain, from each of the

original images

1704 and 1705 retrieved in the first area 1702 of the memory 430-1, An object image 1706 representing the "mother of July" and an object image 1706 representing "July of 1997" to be included in the request image.

In one embodiment, the object image to be included in the request image may refer to the entire retrieved original image. For example, the object to be included in the request image may refer to the original image 1703 itself retrieved in the second area 1701 of the memory 430.

The processor 420-1 of the external device 450 may generate the request image 1709 using the acquired objects.

In one embodiment, the request image can be generated by arranging or composing each of the acquired objects at different positions of one image. For example, the processor 420-1 of the external device 450 can arrange the

object images

1706 and 1707 representing the person in parallel in the lower part in one image.

In another embodiment, the request image can be generated by placing objects obtained to overlap one of the acquired objects with another object. For example, the processor 420-1 of the external device 450 can generate a request image by placing object images representing a person to overlap with one original image representing the background.

18 is a detailed flowchart of an operation for an electronic device according to various embodiments of the present invention to generate a request image. FIG. 18 is a detailed flowchart of

operations

630 and 640 of FIG.

In operation 1810, the processor 420 of the electronic device 410 may determine whether there is more than one original image that contains an object that satisfies the condition. For example, if the image generation condition includes information of an object of a first condition (e.g., mother of 1997.7.1 to 1997.8.31), the processor 420 may determine whether the image 430 is in the memory 430 of the

electronic device

410, 1 < / RTI > condition, and as a result, a plurality of original images including an object satisfying the first condition can be searched.

In operation 1820, the processor 420 of the electronic device 410 may check predetermined conditions. In one embodiment, the predetermined condition may be a setting for how to process a plurality of original images retrieved when a plurality of original images are retrieved for one object.

In operation 1830, the processor 420 of the electronic device 410 may verify that it is set up to generate a plurality of request images.

If set to generate a plurality of request images, the processor 420 of the electronic device 410 may, in 1850 operation, generate a plurality of request images using a plurality of source images. For example, if the object image to be included in the request image includes the first object image and the second object image, a plurality of original images including the first object image are searched, and the original image including the second object image is one The processor 420 of the electronic device 410 determines whether or not the first object image obtained from each of the plurality of original images including the first object image and the first object image obtained from the original image including the second object It is possible to generate multiple request images using 2 object images.

If not configured to generate a plurality of request images, the processor 420 of the electronic device 410 may, in 1840 operation, determine an original image of one of the plurality of original images. In one embodiment, the processor 420 of the electronic device 410 may determine one original image based on attributes (e.g., accuracy, preference, etc.) of the plurality of original images or randomly. For example, the processor 420 of the electronic device 410 may determine the original image having the highest preference among the plurality of original images.

In 1860 operation, the processor 420 of the electronic device 410 may generate one request image. For example, when the object image to be included in the request image includes the first object image and the second object image, a plurality of original images including the first object image, and a single original image including the second object , One request image may be generated using one original image determined from a plurality of original images including the first object image and one original image including the second object.

Referring to FIG. 19, the electronic device 1910 may be an electronic device 410.

In one embodiment, the processor 420 of the electronic device 1910 may provide the user with a request image generated by the electronic device 1910 or a request image generated by the external device 450. [ For example, the processor 420 of the electronic device 1910 may control the display to display the request image 1920 on the screen. At this time, the processor 420 of the electronic device 1910 may control the display to display additional messages. For example, the display of the electronic device 1910 may display a message " Please rate the satisfaction " at the bottom of the request image 1920 displayed on the screen.

In one embodiment, the processor 420 of the electronic device 1910 may provide a user interface with the request image 1920 to receive user feedback (e.g., a star) for the request image. For example, the processor 420 of the electronic device 1910 may provide a user interface 2030 with a requesting image 1920 to receive information about a rating for the requesting image 1920.

In one embodiment, the processor 420 may determine the satisfaction (e.g., preference 1370 of FIG. 13) of the original image used for use in generating the request image 1920. For example, the processor 420 may determine the satisfaction of the original image used to generate the request image 1920 based on the user feedback information for the request image 1920, and determine the satisfaction degree information in the memory 430 Can be saved (or updated).

Although not shown, the processor 420 may provide a user interface for receiving user feedback for each of the object images contained in the request image. The processor 420 may determine the satisfaction of the original image used to generate the request image 1920 based on the user input for the provided user interface.

20 is a flow diagram of an operation of storing an image acquired by an external device in a memory in accordance with various embodiments of the present invention. 20 may be a part of the process of building a database in the memory 430-1 of the external device 450. [

In operation of 2010, the processor 420-1 of the external device 450 may acquire one or more images. For example, the external device 450 may obtain one or more images through at least one of a social network service (SNS), an Internet bulletin board, a messenger, a blog, and a media platform.

In operation 2020, the processor 420-1 of the external device 450 may derive information of the image based on at least one of a user comment, at least one object included in the acquired image, or an upload path. In one embodiment, the information in the image may be a plurality of items for the image and corresponding attribute values. For example, the information of the image may include information about the photographed environment of the image (time, place, weather, etc.), information about the photographic context of the image (photographic purpose), information about the object And the like.

In one embodiment, processor 420-1 of external device 450 may derive information of an image based on user comments. In one embodiment, the processor 420-1 of the external device 450 may derive information of an image using a natural language analysis result or a machine learning learning result. User comments can have various forms depending on the acquisition path of the image. For example, if a user has uploaded a particular image via the SNS and left a hashtag such as "#travel# with mom ", the processor 420-1 of the external device 450 would " Travel, " " image object-user, mother "

In operation 2030, the processor 420-1 of the external device 450 may determine the area of the memory 430-1 in which to store the acquired image, based on the information of the derived image. In one embodiment, the processor 420-1 of the external device 450 stores one of the first area 431 and the second area 432 of the memory 430-1 in an area of memory to store the acquired image .

In one embodiment, based on the acquisition path of the image, the processor 420-1 of the external device 450 may determine the area of the memory 430-1 in which to store the acquired image. For example, when an image is acquired through the SNS account of the person A, the processor 420-1 of the external device 450 can store the acquired image in the first area 431 designated or dedicated to the person A have.

In one embodiment, based on whether the image includes personal information, the processor 420-1 of the external device 450 can determine the area of the memory 430-1 in which to store the acquired image. For example, if all the information derived for the acquired image does not contain personal information, the processor 420-1 of the external device 450 stores the acquired image and information of the derived image in the memory 430-1 In a second area 432 of the second area 432. [ In another embodiment, when some of the information derived for the acquired image includes personal information of the person A, the processor 420-1 of the external device 450 may convert the acquired image and the information of the derived image A in the first area 431 designated or dedicated to the user. Some information (for example, " object of image-user, mother ") including personal information may be set to a mark indicating that personal information is included.

In operation 2040, the processor 420-1 of the external device 450 may store the acquired image in the area of the memory 430-1 determined in operation 2030, together with the information of the derived image. The images stored with the information of the derived image are illustrated in Figs. 13 and 14. Fig.

Claims

In an electronic device,

Communication module; And

The processor comprising:

Receiving an input via the electronic device or an external device functionally connected through the communication module,

Wherein the status information includes time information and object information,

Searching at least one image among the one or more images stored in the electronic device or the external device corresponding to the status information,

Providing the retrieved at least one image if the at least one image corresponding to the context information is retrieved, and

And generates an image corresponding to the context information using at least one other image corresponding to a part of the context information among the one or more images when the at least one image corresponding to the context information is not retrieved And an electronic device configured to provide the generated image.
The method according to claim 1,

And the object information includes place information.
3. The method of claim 2,

At least one other image corresponding to a part of the situation information,

And at least one image including an object corresponding to the time information and the object information.
The method of claim 3,

Wherein the processor is configured to retrieve the at least one other image to generate an image corresponding to the context information using the at least one other image,

Wherein the processor is configured to retrieve an image that satisfies a second time condition that includes the first time condition when an image satisfying a first time condition indicated by the time information is not retrieved, Is set to search for the electronic device.
5. The method of claim 4,

Wherein the processor is configured to generate an image corresponding to the status information by generating an image including the time information and the object corresponding to the object information when the at least one other image is not retrieved.
5. The method of claim 4,

Wherein the processor is configured to generate an image corresponding to the context information by dividing or cropping an area corresponding to the object included in the at least one other image.
The method according to claim 6,

Wherein the processor is configured to generate an image corresponding to the context information by arranging the divided or cropped regions in one image.
The method of claim 3,

At least one other image corresponding to a part of the situation information,

A first image corresponding to a previous time of a time interval corresponding to the time information, and a second image corresponding to a time after the time interval,

Wherein the processor is configured to generate an image corresponding to the context information by interpolating the first image and the second image.
The method according to claim 1,

Wherein the processor is configured to check the status information by converting the personal information into public information when the status information includes private information.
The method according to claim 1,

Wherein the processor is further configured to receive user feedback on the provided image.
8. The method of claim 7,

Wherein the processor is further configured to prioritize attributes associated with screen display of at least one of the divided or cropped regions before displaying the divided or cropped regions in the one image based on attributes associated with screen display of the remaining regions An electronic device set to change.
8. The method of claim 7,

Wherein the processor is configured to perform processing for disposing the divided or cropped regions in the one image, and then suppressing discomfort between the arranged regions.
A method of operating an electronic device,

Receiving an input via the electronic device or an external device functionally connected through the communication module of the electronic device,

Checking status information associated with the input, the status information including time information and object information,

Obtaining at least one image corresponding to a portion of the context information of the one or more images stored in the electronic device or the external device;

Generating an image corresponding to the context information using the acquired at least one image, and providing the generated image.
14. The method of claim 13,

Wherein the step of arranging the divided or cropped regions into one image includes the step of arranging the divided or cropped regions in a single image before the divided or cropped regions are arranged in the one image, Based on an attribute associated with the display of the remaining regions.
14. The method of claim 13,

The operation of arranging the divided or cropped regions in one image includes an operation of arranging the divided or cropped regions in the one image and then performing a process for suppressing an uncomfortable feeling between the arranged regions How to.