KR20230115819A - Electronic device for generating panoramic image and method for operating thereof - Google Patents

Abstract

According to various embodiments, an electronic device, for generating a panoramic image with improved image quality, comprises a camera, communication circuit, and at least one processor, wherein the at least one processor is configured to: acquire a first image using the camera; receive a second image from an external electronic device in communication with the electronic device, via the communication circuit; identify first information about a pose of the external electronic device, based on the at least one second image, wherein the first information about the pose includes information about at least one of a position or an orientation; determine whether a change of the pose of the external electronic device is required for generating a panoramic image, based on at least one first feature point identified from the first image and at least one second feature point identified from the second image; if a change of the pose is determined to be required, transmit second information for changing the pose to the external electronic device, wherein the second information includes information for displaying a graphical object representing a change in at least one of the position or the orientation; after the transmission of the second information, receive a third image from the external electronic device; and generate a panoramic image, based on the first image and the third image. Various other embodiments are possible.

Description

Electronic device for generating a panoramic image and its operating method

Various embodiments of the present disclosure relate to an electronic device for generating a panoramic image and an operating method thereof.

With the recent rapid development of electronic devices, electronic devices capable of wireless voice communication and information exchange have become necessities of life. In the early days of the spread of the electronic device, it was recognized that it was simply portable and capable of wireless communication. As with the remote control using short-distance communication, the application range is increasing.

In particular, users take images using digital cameras installed in electronic devices, and use the captured images not only for recording but also for information transmission.

The electronic device may acquire an image including a foreground in front of the camera by performing photographing using a camera included in the electronic device. At this time, due to the limitations of the angle of view of the camera provided in the electronic device, it is difficult to photograph (single shooting) an image that includes all of the foreground viewed by the user at once, so the electronic device is capable of capturing a plurality of images while the electronic device is moving. A function of capturing images and synthesizing a plurality of captured images to generate a panoramic image including a wider foreground than a single capturing may be provided. However, in order to generate a panoramic image, a ghosting effect representing an afterimage of the movement of an object may be reflected in the panoramic image due to the movement of the object while the electronic device is moving, and thus the quality of the image may be degraded.

According to various embodiments, an electronic device and an operating method thereof may generate a panoramic image with improved image quality by generating a panoramic image based on a plurality of images simultaneously photographed by a plurality of electronic devices.

According to various embodiments of the present disclosure, an electronic device and an operating method thereof may provide a pose (eg, direction and/or location) of another sub device to generate a panoramic image in a host device based on feature points included in captured images. ), it is possible to create a panoramic image with improved quality by allowing photographing to be performed at an exact location for generating a panoramic image.

According to various embodiments, an electronic device includes a camera, a communication circuit, and at least one processor, wherein the at least one processor obtains a first image using the camera, and through the communication circuit, the at least one processor Receiving a second image from an external electronic device communicatively connected to the electronic device, identifying first information about a pose of the external electronic device based on the at least one second image, the first information about the pose information on at least one of a position or a direction, and to generate a panoramic image, based on at least one first feature point identified from the first image and at least one second feature point identified from the second image; It is determined whether the external electronic device needs to change the pose, and if it is determined that the pose needs to be changed, second information for changing the pose is transmitted to the external electronic device, and the second information is includes information for displaying a graphic object indicating a change in at least one of a location and the direction; receives a third image from the external electronic device after transmitting the second information; An electronic device configured to generate the panoramic image based on the image may be provided.

According to various embodiments, in an operating method of an electronic device, obtaining a first image using the camera, and receiving a second image from an external electronic device communicatively connected to the electronic device through the communication circuit and identifying first information about a pose of the external electronic device based on the at least one second image, wherein the first information about the pose includes information about at least one of a location or a direction. and whether the pose of the external electronic device needs to be changed based on at least one first feature point identified from the first image and at least one second feature point identified from the second image to generate a panoramic image. and, when it is determined that the pose needs to be changed, transmitting second information for changing the pose to the external electronic device, wherein the second information is selected from among the location and the direction. information for displaying a graphic object representing at least one change, receiving a third image from the external electronic device after transmitting the second information, and based on the first image and the third image Thus, an operation method including an operation of generating the panoramic image may be provided.

According to various embodiments, the solution to the problem is not limited to the above-described solution, and solutions not mentioned can be provided to those skilled in the art from this specification and the accompanying drawings. will be clearly understood.

According to various embodiments, an electronic device for generating a panoramic image with improved image quality by generating a panoramic image based on a plurality of images simultaneously photographed by a plurality of electronic devices and an operating method thereof may be provided. .

According to various embodiments of the present disclosure, based on feature points included in captured images, a guide for changing a pose (eg, direction and/or position) of another sub device to generate a panoramic image in the host device is provided. An electronic device and an operating method thereof may be provided for generating a panoramic image with improved quality by allowing photographing to be performed at an accurate location for generating a panoramic image by providing the electronic device.

1 is a block diagram of an electronic device in a network environment, according to various embodiments.
2 is a diagram for explaining an example of an operation of generating a panoramic image of electronic devices (eg, a first electronic device, a second electronic device, and a third electronic device) according to various embodiments.
3 is a diagram for explaining an example of a panoramic image generated based on images taken by electronic devices (eg, a first electronic device, a second electronic device, and a third electronic device) according to various embodiments; am.
4 is a diagram illustrating an example of a configuration of an electronic device according to various embodiments.
5 is a diagram for explaining an example of an operation of generating information for guiding a pose in order to generate a panoramic image of an electronic device according to various embodiments.
6A is a diagram for explaining an example of a pose (eg, a position and a direction) of an electronic device according to various embodiments.
6B is a diagram for explaining an example of feature points of images according to various embodiments.
7 is a flowchart illustrating an example of an operation of electronic devices (eg, a host device and a sub device) according to various embodiments.
8 is a diagram for explaining an example of an operation of generating a panoramic image of electronic devices (eg, a host device and a sub device) according to various embodiments.
9 is a flowchart illustrating an example of an operation of a host device according to various embodiments.
10A is a diagram for explaining an example of an operation of displaying images received from sub devices on a display of a host device according to various embodiments.
10B is a diagram for explaining another example of an operation of displaying images received from sub devices on a display of a host device according to various embodiments.
11 is a flowchart illustrating an example of an operation of a host device according to various embodiments.
12 is a diagram for explaining an example of an operation in which a host device and a sub device are determined and a communication connection between the host device and the sub device is established based on execution of an application for generating a panoramic image according to various embodiments.
13 is a flowchart illustrating an example of an operation of a host device according to various embodiments.
14 is a flowchart illustrating an example of an operation of a host device according to various embodiments.
15A illustrates an example of an operation of identifying locations of devices (eg, a host device and a sub device) based on feature points of a plurality of images of an electronic device (eg, a host device) according to various embodiments. It is a drawing for
15B is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments.
15C is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments.
15D is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments.
16 is a flowchart illustrating an example of an operation of a host device according to various embodiments.
17A is a diagram for explaining an example of an operation of providing direction guide information for changing a direction of a sub device of an electronic device (eg, a host device) according to various embodiments.
17B is a diagram for explaining an example of an operation of providing direction guide information for changing a direction of a sub device of an electronic device (eg, a host device) according to various embodiments.
17C is a diagram for explaining an example of an operation of identifying a direction to be changed of a sub device based on an overlapping degree of images of an electronic device (eg, a host device) according to various embodiments.

1 is a block diagram of an electronic device 101 within a network environment 100, according to various embodiments. Referring to FIG. 1 , in a network environment 100, an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or through a second network 199. It may communicate with at least one of the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or the antenna module 197 may be included. In some embodiments, in the electronic device 101, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added. In some embodiments, some of these components (eg, sensor module 176, camera module 180, or antenna module 197) are integrated into a single component (eg, display module 160). It can be.

The processor 120, for example, executes software (eg, the program 140) to cause at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 . According to one embodiment, the processor 120 may include a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, when the electronic device 101 includes the main processor 121 and the auxiliary processor 123, the auxiliary processor 123 may use less power than the main processor 121 or be set to be specialized for a designated function. can The secondary processor 123 may be implemented separately from or as part of the main processor 121 .

The secondary processor 123 may, for example, take the place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, running an application). ) state, together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to one embodiment, the auxiliary processor 123 (eg, image signal processor or communication processor) may be implemented as part of other functionally related components (eg, camera module 180 or communication module 190). there is. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself where the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited. The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples. The artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, program 140) and commands related thereto. The memory 130 may include volatile memory 132 or non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used for a component (eg, the processor 120) of the electronic device 101 from an outside of the electronic device 101 (eg, a user). The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. A receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor set to detect a touch or a pressure sensor set to measure the intensity of force generated by the touch.

The audio module 170 may convert sound into an electrical signal or vice versa. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a bio sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more specified protocols that may be used to directly or wirelessly connect the electronic device 101 to an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or movement) or electrical stimuli that a user may perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to one embodiment, the power management module 188 may be implemented as at least part of a power management integrated circuit (PMIC), for example.

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). Establishment and communication through the established communication channel may be supported. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : A local area network (LAN) communication module or a power line communication module). Among these communication modules, a corresponding communication module is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, a legacy communication module). It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, a LAN or a WAN). These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology). NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported. The wireless communication module 192 may support various requirements defined for the electronic device 101, an external electronic device (eg, the electronic device 104), or a network system (eg, the second network 199). According to one embodiment, the wireless communication module 192 is a peak data rate for eMBB realization (eg, 20 Gbps or more), a loss coverage for mMTC realization (eg, 164 dB or less), or a U-plane latency for URLLC realization (eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 197 may transmit or receive signals or power to the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is selected from the plurality of antennas by the communication module 190, for example. can be chosen A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module 197 in addition to the radiator.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead of executing the function or service by itself. Alternatively or additionally, one or more external electronic devices may be requested to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 101 . The electronic device 101 may provide the result as at least part of a response to the request as it is or additionally processed. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199 . The electronic device 101 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, an electronic device, or a home appliance. An electronic device according to an embodiment of the present document is not limited to the aforementioned devices.

Various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, like reference numbers may be used for like or related elements. The singular form of a noun corresponding to an item may include one item or a plurality of items, unless the relevant context clearly dictates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A Each of the phrases such as "at least one of , B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "secondary" may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited. A (e.g., first) component is said to be "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicatively." When mentioned, it means that the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeably interchangeable with terms such as, for example, logic, logical blocks, parts, or circuits. can be used as A module may be an integral part or the smallest unit of a part or part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of this document provide one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101). It may be implemented as software (eg, the program 140) including them. For example, a processor (eg, the processor 120 ) of a device (eg, the electronic device 101 ) may call at least one command among one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium is a tangible device and does not contain a signal (e.g. electromagnetic wave), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium readable by a device such as a manufacturer's server, an application store server, or a relay server's memory.

According to various embodiments, each component (eg, module or program) of the above-described components may include a single object or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component among the plurality of components prior to the integration. . According to various embodiments, the actions performed by a module, program, or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the actions are executed in a different order, or omitted. or one or more other actions may be added.

2 and 3, electronic devices 200 (eg, a first electronic device 201, a second electronic device 202a, and a third electronic device 202b) according to various embodiments ) will be described as an example of the operation. Since the electronic devices 200 may be implemented like the electronic device 101 described in FIG. 1 , a description overlapping with that of FIG. 1 will be omitted.

2 illustrates an operation of generating a panoramic image of electronic devices 200 (eg, a first electronic device 201, a second electronic device 202a, and a third electronic device 202b) according to various embodiments. It is a drawing for explaining an example of. Hereinafter, FIG. 2 will be described with reference to FIG. 3 .

3 is based on images captured by electronic devices 200 (eg, a first electronic device 201, a second electronic device 202a, and a third electronic device 202b) according to various embodiments. It is a diagram for explaining an example of a panoramic image generated by doing.

According to various embodiments, as shown in FIG. 2 , the first electronic device 201 includes a plurality of electronic devices 200 (eg, the first electronic device 201 and the second electronic device 202a). Panoramic image 300 based on a plurality of images (eg, the first image 301, the second image 302a, and the third image 302b) taken by the , and the third electronic device 202b. ) can be created. For example, the first electronic device 201 acquires the first image 301 using a camera (not shown) provided in the first electronic device 201, and other external electronic devices (eg, 2 Images captured by other external electronic devices (eg, the second electronic device 202a and the third electronic device 202b) from the electronic device 202a and the third electronic device 202b (eg : The second image 302a and the third image 302b) may be received. As shown in FIG. 3 , the first electronic device 201 synthesizes a plurality of images (eg, a first image 301, a second image 302a, and a third image 302b) to form a panoramic image. (300) can be created. The panoramic image 300 may be an image realized by attaching a plurality of images in a long way so as to be able to see the foreground at a glance. The plurality of devices 201, 202a, and 203b may be included in a specific group for generating a panoramic image. At this time, in the specific group, a device (eg, the second electronic device 201a and the third electronic device 202b) receives images from other external electronic devices and generates a panoramic image 300 (eg, the second electronic device 201a and the third electronic device 202b). 1 Electronic device 201 may be defined as a host device (or main device). In addition, in the specific group, devices (eg, the second electronic device 202a and the third electronic device 202b) transmitting images to the host device (eg, the first electronic device 201) are sub devices ( or guest device).

Meanwhile, although the panoramic image 300 has been described as an example, a plurality of electronic devices (eg, a first electronic device 201, a second electronic device 202, and a third electronic device 203) are used instead of the panoramic image 300. ))), content that can be generated based on the synthesis of images or videos taken in the video (eg, composite video) may be provided. That is, various embodiments described in this document may be applied to an operation of generating other types of content (eg, synthetic video) instead of the panoramic image 300 .

According to various embodiments, the host device (eg, the first electronic device 201) generates a panorama image with the sub-devices (eg, the second electronic device 202 and the third electronic device 203), Information (eg, guide information) for guiding poses (eg, positions or directions) of devices (eg, the second electronic device 202 and the third electronic device 203) may be provided. The sub-devices (eg, the second electronic device 202 and the third electronic device 203) determine a pose (eg, position) based on guide information received from the host device (eg, the first electronic device 201). , or direction) may be provided with content (eg, screen, audio) for changing. The user of the sub-devices (eg, the second electronic device 202 and the third electronic device 203) can select the sub-devices (eg, the second electronic device 202 and the third electronic device 203) based on the content. Change the pose (eg, move the position or rotate in the corresponding direction), and sub-devices (eg, the second electronic device 202 and the third electronic device 203) take pictures in a state where the pose is changed. The image may be acquired and transmitted to the host device (eg, the first electronic device 201). Accordingly, the host device (eg, the first electronic device 201) may generate a panoramic image 300 based on newly acquired images. Since the poses of the sub devices (eg, the second electronic device 202 and the third electronic device 203) do not meet the conditions for generating the panoramic image 300, they must be connected to generate the panoramic image 300. Images (eg, the first image 301 and the second image 302a, or the first image 301 and the third image 302b) may not be connected to each other. Accordingly, the host device (eg, the first electronic device 201) may provide information for changing a pose to the sub devices (eg, the second electronic device 202 and the third electronic device 203). , which will be described in detail below.

Hereinafter, examples of the configuration of the electronic device 400 according to various embodiments will be described. The aforementioned electronic devices 200 may be implemented like the electronic device 400 described below. However, in the case of a sub device, it may be implemented not to include some components (eg, modules 460).

4 is a diagram illustrating an example of a configuration of an electronic device 200 according to various embodiments. Hereinafter, FIG. 5 will be described with reference to FIGS. 5, 6A, and 6B.

5 is a diagram for explaining an example of an operation of generating information for guiding a pose in order to generate a panoramic image of an electronic device according to various embodiments. 6A is a diagram for explaining an example of a pose (eg, position and direction) of an electronic device according to various embodiments. 6B is a diagram for explaining an example of feature points of images according to various embodiments.

According to various embodiments, the electronic device 400 includes a camera 410, a display 420, a communication circuit 430, a processor 440, and a plurality of modules 460 (eg, a panorama generation determination module 461 ), a pose guide information acquisition module 462, and a memory including a panorama generation module 463). Meanwhile, not limited to the components shown in FIG. 4 , the electronic device 400 may be implemented to include more components or fewer components. For example, the electronic device 400 may be implemented to further include the components of the electronic device 101 described in FIG. 1 . Hereinafter, an example of a configuration of the electronic device 400 according to various embodiments will be described.

According to various embodiments, the camera 410 may be implemented to capture an image. For example, since the camera 410 may be implemented like the camera module 180 of FIG. 1 , duplicate descriptions will be omitted. Since the camera 410 is implemented to capture an image at a specific angle of view, a view of a field of view greater than or equal to the specific angle of view cannot be captured. Accordingly, the electronic device 400 may generate a panoramic image of a view of a specific angle or higher by synthesizing images captured by external electronic devices and images captured by the corresponding electronic device 400 using the camera 410. there is.

According to various embodiments, the display 420 may be implemented to display a screen. For example, since the display 420 may be implemented like the display module 160 of FIG. 1 , duplicate descriptions will be omitted. When the electronic device 400 is a sub device, the electronic device 400 guides the change of the pose (eg, position and/or direction) of the electronic device 400 based on the information for changing the pose received from the host device. A graphic user interface (GUI) for this may be displayed on the display 402 .

According to various embodiments, the communication circuit 430 may establish a communication connection with an external electronic device and transmit and/or receive predetermined information and/or data. For example, since the communication circuit 430 may be implemented like the communication module 190 of FIG. 1 , duplicate descriptions will be omitted. For example, when the electronic device 400 is a host device, the electronic device 400 may receive an image captured by the sub device from the sub device through the communication circuit 430 . Also, for example, when the electronic device 400 is a host device, the electronic device 400 transmits guide information for changing a pose (eg, position and/or direction) to a sub device through the communication circuit 430. can Also, for example, when the electronic device 400 is a sub device, the electronic device 400 may transmit a captured image to the host device through the communication circuit 430 .

Hereinafter, examples of the processor 440 and programs (eg, modules 460) executed by the processor 440 according to various embodiments will be described.

The processor 440 according to various embodiments includes at least one of an application processor (AP), a central processing unit (CPU), a graphic processing unit (GPU), a display processing unit (DPU), or a neural processing unit (NPU). can do.

According to various embodiments, modules 460 implemented (or stored) in the electronic device 400 (eg, a panorama generation determination module 461, a pose guide information acquisition module 462, and a panorama generation module ( 463) is an application, program, computer code, instructions, routine, process, software, firmware, or It may be implemented in the form of a combination of at least two or more of these. For example, when the modules 460 (eg, the panorama generation determination module 461, the pose guide information acquisition module 462, and the panorama generation module 463) are executed, the processor 440 corresponds to each. action can be performed. Therefore, description that a specific module performs an operation below may be understood as the fact that the processor 440 performs an operation corresponding to the specific module as the specific module is executed. Meanwhile, at least some of the modules 460 (eg, the panorama generation determination module 461, the pose guide information acquisition module 462, and the panorama generation module 463) may be implemented as at least part of one program. For example, it may be implemented as at least a part of functions of an application (eg, a camera application) for generating a panoramic image. Meanwhile, at least some of the modules 460 (eg, the panorama generation determination module 461, the pose guide information acquisition module 462, and the panorama generation module 463) may be implemented in hardware form (eg, a processing circuit). may be

According to various embodiments, the panorama generation determination module 461 may determine whether a plurality of images acquired for panoramic image generation satisfy a condition for panoramic image generation. For example, the panorama generation determination module 461 may determine whether a condition for generating a panorama is satisfied based on comparison of information (eg, coordinates of feature points) of feature points extracted for each of a plurality of images. . For example, referring to FIG. 5 , the panorama generation determination module 460 may include a keypoint identification module 510 for obtaining information (eg, coordinates of keypoints) of keypoints for each image. there is. 6B shows the feature points analyzed from the image. For convenience of explanation, feature points are displayed on a part of the image, but the feature points are not limited to those shown in the drawing, and feature points may also be identified from the remaining area where the feature points are not displayed. Referring to FIG. 6B , the feature points of the images 630a and 630b may correspond to features in the image. For example, feature points of the images 630a and 630b may represent outlines and/or corners defining the shapes of objects (eg, buildings, objects, people, and animals) in the images 630a and 630b. For example, the feature point identification module 510 compares the pixel values of the image and identifies the outline and/or corner of the object based on identifying a portion where the pixel value difference is large, and locates the identified outline. Coordinates of corresponding feature points may be determined. Since the difference between the pixel values inside and outside the outline of the object is large, a feature point may be determined based on the difference in size of the pixel values. Meanwhile, the feature point identification algorithm may be implemented in various ways other than the algorithm for comparing the pixel value difference, and since this is a well-known technique, a detailed description thereof will be omitted.

According to various embodiments, the condition for generating the panorama may include a condition associated with a location (location condition) and/or a condition associated with a direction (direction condition). For example, if the condition is a position-related condition, the condition may be set to be satisfied or unsatisfied depending on whether the position of the sub device identified based on the characteristic point corresponds to a preset position. Also, for example, if the condition is a condition related to a pose, the condition may be set to be satisfied or unsatisfied according to the number of corresponding feature points between images. For example, the condition may be set so that the condition is satisfied when the number of corresponding feature points of the images is included in a specified range, and the condition is not satisfied when the number of corresponding feature points corresponding to the images is not included in the specified range. can The size of the region where the images overlap can be indicated. That is, if the number of feature points corresponding to the images is included in the specified range, it may mean that the size of the region where the images overlap is appropriate for generating a panoramic image. When the above conditions are satisfied, the panorama generation determination module 461 may inform the panorama generation module 463 that the conditions are satisfied. The panorama generation module 463 may accordingly generate a panoramic image based on a plurality of images. The panorama generation determination module 461 may inform the pose guide information acquisition module 462 that the conditions are not satisfied when the above conditions are not satisfied. The pose guide information acquisition module 462 may generate guide information for changing poses (eg, positions and/or directions) of external devices (eg, sub-devices) accordingly.

According to various embodiments, the pose guide information acquisition module 462 may generate information for guiding a pose (eg, position and/or direction) of a sub device. For example, referring to 601 of FIG. 6A, the location means the two-dimensional coordinates (x, y) of each of the devices 611, 612, and 613 when observed in a direction substantially perpendicular to the ground (eg, For example, referring to 602 of FIG. 6A, the direction is based on the x-axis of the center line C (or the center line of the camera 410) of the FOV of the camera of the sub device 620. It may mean a rotation angle, a rotation angle based on the y-axis, and a rotation angle based on the z-axis. The rotation angles may also be defined as angles in a roll direction, a yaw direction, and a pitch direction. The pose (eg : Information for guiding position and/or direction) information for displaying a graphic user interface (or graphic object) for changing a position on a display in a sub device (location guide information) and/or for changing a direction Information (direction guide information) for displaying a graphic user interface (or graphic object) may be included For example, information for guiding the pose (eg, position and/or direction) may include the position and/or direction It may include a graphic user interface (or graphic object) for changing. Accordingly, the sub device may display the graphic user interface (or graphic object) included in the information. In another example, the pose ( Example: the information for guiding the location and/or direction) may include identification information representing the target location and/or target direction. Accordingly, the sub device identifies the target location and/or target direction based on the identification information. and may create and display a graphic user interface (or graphic object) for changing to a target location and/or target direction. For example: location and/or direction), an image that satisfies the conditions for creating a panoramic image is taken by the sub device, and as a result, a plurality of images collected by the host device are the conditions for creating a panoramic image. A panoramic image may be generated as it is determined to satisfy .

According to various embodiments, the pose guide information acquisition module 462 identifies the location (eg, coordinates) of the sub device based on the location (eg, coordinates) of the host device based on the comparison of feature points of the images, and identifies the location (eg, coordinates) of the sub device. Information (eg, location guide information) for guiding the location of the sub device to generate a panoramic image may be obtained based on the determined location. For example, referring to FIG. 5 , the pose guide information acquisition module 462 is based on the device location identification module 521 implemented to identify locations of devices (eg, host devices and sub devices) and the identified locations. It may include a location guide information acquisition module 522 configured to identify a location to be changed and generate location guide information for changing the location of the sub device to the location to be changed. An example of an operation of generating location guide information will be described in detail with reference to FIGS. 14 and 15 .

According to various embodiments, the pose guide information acquisition module 462 may include information for guiding the direction of the sub-device to generate a panoramic image based on the overlapping degree of the images identified based on the comparison of feature points of the images ( Example: direction guide information) can be acquired. For example, referring to FIG. 5 , the pose guide information acquisition module 462 includes a device direction identification module 531 configured to obtain information on directions for each device (eg, a host device and a sub device) and a direction to be changed. It may include a direction guide information acquisition module 532 implemented to identify and generate location guide information for changing the direction of the sub device in the direction to be changed. An example of an operation of generating direction guide information will be described in detail with reference to FIGS. 16 and 17 .

According to various embodiments, the panorama generating module 463 may generate a panoramic image by merging a plurality of images when a condition for generating a panoramic image is satisfied. For example, the panorama generating module 463 may generate one panoramic image by merging images including overlapping regions. The panorama generation module 463 may perform various graphic processing (eg, color change, image cropping) in addition to image generation. Since the operation of generating a panoramic image is a well-known technique, a detailed description thereof will be omitted.

Hereinafter, examples of operations of electronic devices (eg, a host device and a sub device) according to various embodiments will be described.

According to various embodiments, the host device may acquire an image captured by the sub device from an image captured by the host device and a sub device to generate a panoramic image. The host device may provide information for changing the pose (eg, position and/or direction) of the sub device when the image received from the sub device does not satisfy the conditions for generating a panoramic image. The sub device may provide the user of the sub device with information for changing a pose (eg, position and/or direction) based on the provided information.

7 is a flowchart 700 for explaining an example of an operation of electronic devices (eg, a host device and a sub device) according to various embodiments. According to various embodiments, the operations illustrated in FIG. 7 may be performed in various orders without being limited to the illustrated order. Also, according to various embodiments, more operations than the operations shown in FIG. 7 or at least one operation less may be performed. Hereinafter, FIG. 7 will be further described with reference to FIG. 8 .

8 is a diagram for explaining an example of an operation of generating a panoramic image of electronic devices (eg, a host device and a sub device) according to various embodiments.

According to various embodiments, the electronic device 700a (eg, a host device) may establish a communication connection with the first external electronic device 700b (eg, a sub device) in operation 701 . For example, referring to FIG. 8 , an electronic device 700a that is a host device and sub devices (eg, a first external electronic device 700b and a second external electronic device 700c) may establish a direct communication connection with each other. there is. The communication connection may be performed based on a device to device (D2D) communication connection method such as Bluetooth or Wi-Fi direct. Alternatively, the communication connection is not limited to the description, and the communication connection is between the electronic device 700a, which is a host device, and the sub devices (eg, the first external electronic device 700b and the second external electronic device 700c) are relays (eg, APs). It may be performed based on a communication connection method such as Wi-Fi that performs a communication connection through (access point)). An example of a specific operation of determining a host device and a sub device and establishing a communication connection will be described later with reference to FIGS. 11 and 12 .

According to various embodiments, the electronic device 700a (eg, the host device) may acquire the first image 800a using the camera 410 in operation 703 .

According to various embodiments, the first external electronic device 700b (eg, a sub device) obtains a second image 800b using a camera (not shown) in operation 705 and transmits the second image electronically in operation 707. It may transmit to the device 700a (eg, host device). Meanwhile, referring to FIG. 8 without being limited thereto, the second external electronic device 700c (eg, a sub device) also captures an image 800c, and transmits the captured image 800c to the electronic device 700a (eg, a sub device). host device).

According to various embodiments, the electronic device 700a (eg, a host device) determines whether a pose change of the first external electronic device 700b (eg, a sub device) is required based on the second image in operation 709 and In operation 711, third information for changing a pose may be transmitted. For example, referring to 801 of FIG. 8 , a first image 800a captured by the electronic device 700a (eg, a host device) and a photographed by a first external electronic device 700b (eg, a sub device) Since the second image 800b includes the same partial area 800, it may be images synthesized to generate a panoramic image. As described above with reference to FIG. 5 , the electronic device 700a (eg, a host device) sets a condition (eg, a position condition and/or a direction condition) for generating a panoramic image based on the second image to be synthesized with the first image. ) is satisfied. The electronic device 700a (eg, host device) generates pose guide information for changing a pose (eg, position and/or direction) when a condition (eg, position condition and/or orientation condition) is not satisfied. and the generated pose guide information may be transmitted to the first external electronic device 700b (eg, a sub device). An example of the satisfaction of the condition and an operation of generating the pose guide information will be described later with reference to FIGS. 14 to 17 . Meanwhile, referring to 801 of FIG. 8 , conditions for generating a panoramic image based on an image captured by the second external electronic device 700c to be synthesized with the first image (eg, position condition and/or direction condition) can be judged to be satisfied. In this case, the electronic device 700a (eg, the host device) transmits information indicating that the condition is satisfied to the second external electronic device 700c, and the second external electronic device 700c based on the received information A notification indicating that the condition is satisfied can be displayed.

According to various embodiments, the first external electronic device 700b (eg, a sub device) may display a screen for changing a pose based on the third information in operation 713 . For example, the screen may include at least one graphic object for changing location and/or direction. Examples of the displayed graphic object will be described later with reference to FIGS. 14 to 17 .

According to various embodiments, the first external electronic device 700b (eg, a sub device) obtains a third image using a camera (not shown) in operation 715 and transmits the third image to the electronic device (operation 717). 700a) (eg host device). For example, referring to 802 of FIG. 8 , the user of the first external electronic device 700b (eg, sub device) uses the first external electronic device 700b (eg, sub device) according to a graphic object displayed on the screen. The position and / or direction of the can be changed. After the location and/or direction of the first external electronic device 700b (eg, a sub device) is changed, the third image 800d is captured, and the captured third image 800d is transferred to the first external electronic device ( 700b) (eg, sub device).

According to various embodiments, the electronic device 700a (eg, a host device) may generate the panoramic image based on the first image 800a and the third image 800d in operation 719 . The electronic device 700a (eg, a host device) may re-determine whether a condition for generating a panoramic image (eg, a position condition and/or a direction condition) is satisfied based on the third image 800d. . In an embodiment, the electronic device 700a (eg, a host device) automatically generates a plurality of images (eg, a first image ( A panoramic image may be generated based on 800a), the third image 800d, and the image 800c). In another embodiment, the electronic device 700a (eg, a host device) provides a notification when a condition for generating a panoramic image is satisfied, and provides a user's shooting input (eg, an icon for capturing a panoramic image (not shown). ) selection), a panoramic image may be generated based on a plurality of images (eg, the first image 800a, the third image 800d, and the image 800c).

Hereinafter, an example of an operation of an electronic device (eg, a host device) according to various embodiments will be described.

According to various embodiments, the host device is configured to change the pose (eg, position and/or direction) of the sub device when it is determined that the conditions for generating a panoramic image are not satisfied based on the image received from the sub device. can provide information. At this time, the host device may display the images received from the sub device on the display so that the user of the host device can check the image captured by the sub device.

9 is a flowchart 900 for explaining an example of an operation of a host device according to various embodiments. According to various embodiments, the operations shown in FIG. 9 are not limited to the order shown and may be performed in various orders. Also, according to various embodiments, more operations than the operations shown in FIG. 9 may be performed, or at least one operation less than that shown in FIG. 9 may be performed. Hereinafter, FIG. 9 will be described with reference to FIGS. 10A and 10B.

10A is a diagram for explaining an example of an operation of displaying images received from sub devices on a display of a host device according to various embodiments. 10B is a diagram for explaining another example of an operation of displaying images received from sub devices on a display of a host device according to various embodiments.

According to various embodiments, the electronic device (eg, the host device 1000a) captures a first image (eg, the image 1010a of FIG. 10A or the image 1060a of FIG. 10B) using the camera 410 in operation 901. ) is obtained, and in operation 903, at least one second image (eg, the image of FIG. 10A ) is obtained from at least one external electronic device (eg, the first sub-device 1000b and the second sub-device 1000c) connected through communication. 1010b and 1010c, or images 1060b and 1060c of FIG. 10B may be received.

According to various embodiments, an electronic device (eg, the host device 1000a), referring to FIGS. 10A and 10B , displays a plurality of acquired images (eg, a first image and at least one second image) in a preview form. can be displayed on the display. For example, the electronic device (eg, the host device 1000a) displays a first image (eg, the image 1010a of FIG. 10A ) on a lower area of an execution screen of an application (eg, a camera application) for generating a panorama image to be described later. , or the image 1060a of FIG. 10B and at least one second image (eg, the images 1010b and 1010c of FIG. 10A or the images 1060b and 1060c of FIG. 10B) may be displayed. Also, the electronic device (eg, the host device 1000a) may display panoramic images 1020 and 1070 in which a plurality of images are synthesized on an upper region of an execution screen of an application (eg, a camera application). At this time, the electronic device (eg, the host device 1000a) identifies the location of each of the identified sub-devices 1000b and 1000c based on the images, and images 1010a, 1060a), the images 1010b and 1060bc of the first sub-device 1000b located on the left are displayed in the left area, and the images 1010a and 1060a captured by the host device 1000a are displayed on the right. Images 1010c and 1060c of the located second sub device 1000c may be displayed in the right area.

According to various embodiments, the electronic device (eg, the host device 1000a) displays the first image (the image 1010a in FIG. 10A or the image 1060a in FIG. 10B) and the at least one second image in operation 905. Based on images (eg, images 1010b and 1010c of FIG. 10A or images 1060b and 1060c of FIG. 10B), at least one external electronic device (eg, the first sub-device 1000b), and the second sub-device 1000b. It is possible to determine whether a change of the pose of the second sub-device (1000c) is necessary. For example, the electronic device (eg, the host device 1000a) captures a first image (eg, the image 1010a of FIG. 10A or the image of FIG. 10B ) captured by the electronic device (eg, the host device 1000a). (1060a)) and a second image (eg, images 1010b and 1010c of FIG. 10a, or images 1060b and 1060c of FIG. 10b) to be synthesized. Based on the location and/or overlapping degree of the images, it may be determined whether a condition for generating a panoramic image (eg, a location condition and/or a direction condition) is satisfied.

According to various embodiments, an electronic device (eg, the host device 1000a) may display different execution screens according to whether a condition (eg, a location condition and/or an orientation condition) for generating a panoramic image is satisfied. there is.

For example, referring to FIG. 10A , when the location condition among the conditions for generating a panoramic image for an external electronic device (eg, the second sub-device 1000c) is not satisfied, the electronic device (eg, the host device 1000a) Displays the image 1010c captured by the second external electronic device (eg, the sub device 1000c) on the lower area of the execution screen, but does not perform synthesis based on the image 1010c on the upper area of the execution screen. A panoramic image 1020 obtained by synthesizing the images 1010a and 1010b satisfying the remaining location conditions may be displayed. Accordingly, the area 1030 for the second external electronic device (eg, the sub device 1000c) may display content other than an image (eg, a single color (eg, black) image or a specific pattern image). there is.

For example, referring to FIG. 10B , when the direction condition among the conditions for generating a panoramic image for an external electronic device (eg, the second sub-device 1000c) is not satisfied, the electronic device (eg, the host device 1000a) ) displays an image 1060b captured by an external electronic device (eg, the first sub-device 1000b) in the lower area of the execution screen, and displays images 1060a, 1060b, and 1060c in the upper area of the execution screen. displaying the synthesized panoramic image 1070 as the based compositing is performed, but displaying a highlighting object 1080 (e.g., the outline of the region is outlined in red) to indicate areas that are not oriented on the panoramic image 1070. can do. The area may correspond to the image 1060b captured by the external electronic device (eg, the first sub-device 1000b). As will be described later, when the direction of the first sub device 1000b is changed according to the provision of guide information and the direction condition is satisfied, the electronic device (eg, the host device 1000a) displays images 1060 in the upper area of the execution screen. , 1060b, 1060d), the synthesized panoramic image 1070 is displayed, but the object 1080 on the panoramic image 1070 (for example, the outline of the region is displayed in red) to indicate that the direction condition is satisfied. ) can be replaced to display another object 1090 (eg, the outline of the region is displayed in green).

According to various embodiments, when an electronic device (eg, the host device 1000a) needs to change a pose of an external electronic device (eg, the first sub-device 1000b and the second sub-device 1000c) (907 -Yes), in operation 909, the third information for changing the pose is transmitted to at least one external electronic device (eg, the first sub-device 1000b and the second sub-device 1000c), and in operation 911, at least At least one third image 1060d is received from one external electronic device (eg, the first sub-device 1000b and the second sub-device 1000c), and the first image and the at least one third image 1060d are received in operation 913. Based on the third image of , the panoramic image may be generated. The third information for changing the pose will be described later in detail with reference to FIGS. 14 to 17 .

According to various embodiments, when the electronic device (eg, the host device 1000a) does not need to change the pose of the external electronic device (907 - No), in operation 915, the first image and at least one second image are displayed. Based on this, a panoramic image can be created.

Hereinafter, an example of an operation of an electronic device (eg, a host device) according to various embodiments will be described.

According to various embodiments, a host device and a sub device may be determined based on execution of an application for generating a panoramic image, and a communication connection between the host device and the sub device may be established.

11 is a flowchart 1100 for describing an example of an operation of a host device according to various embodiments. According to various embodiments, the operations shown in FIG. 11 are not limited to the order shown and may be performed in various orders. Also, according to various embodiments, more operations than the operations shown in FIG. 11 or at least one operation less may be performed. Hereinafter, FIG. 11 will be described with reference to FIG. 12 .

12 is a diagram for explaining an example of an operation of determining a host device and a sub device and establishing a communication connection between the host device and the sub device based on execution of an application for generating a panoramic image according to various embodiments.

According to various embodiments, the electronic device (eg, the host device 1200a) may execute an application for generating a panoramic image in operation 1101 and select a photographing mode based on the application executed in operation 1103. For example, referring to 1201 of FIG. 12 , the electronic device (eg, the host device 1200a) may execute an application for generating a panoramic image in response to selecting an icon for executing an application for generating a panoramic image. there is. The application for generating the panoramic image is implemented to provide a function for generating a panoramic image as one function and to provide functions other than the one function (eg, single shooting function, continuous shooting function, slow motion shooting function) It may be a camera application, but it may also be an application implemented to generate only a panoramic image without being limited to what has been described and/or shown. Meanwhile, the application (e.g., camera application) may be executed in a different pre-set manner (e.g., by selecting an icon placed on a callable panel or by a gesture) rather than by selecting an icon arranged on the home screen. there is. The electronic device (eg, the host device 1200a) displays an execution screen of a camera application, and the execution screen of the camera application selects a plurality of functions (or a plurality of modes) as shown in 1202 of FIG. 12 . It may include a plurality of objects for For example, the plurality of objects may include a first object for selecting a multi-panorama image generating function. The multi-panoramic image generating function may be defined as a function of generating a panoramic image based on a plurality of images captured by the host device and the sub device.

According to various embodiments, the electronic device (eg, the host device 1200a) may determine whether the shooting mode corresponds to the panorama multi-device mode in operation 1105 . For example, referring to 1202 of FIG. 12 , the electronic device (eg, the host device 1200a) may determine whether a first object for selecting a multi-panorama image generating function is selected by the user.

According to various embodiments, the electronic device (eg, the host device 1200a) may establish a communication connection with at least one external electronic device in operation 1107 when the shooting mode corresponds to the panorama multi-device mode (1105-yes). . For example, when the first object is selected, the electronic device (eg, the host device 1200a) identifies that the electronic device (eg, the host device 1200a) is set as a host device for generating a panoramic image, and As shown in 1203 of 12, an execution screen of an application (eg, a camera application) including a code (eg, a QR code 1231) for communication connection may be displayed. The QR code 1231 may include information (eg, SSID, IP address, identification information) for setting the aforementioned D2D communication. The user of the external electronic device 1200b may capture the master QR code 1231 displayed on the electronic device (eg, the host device 1200a) using the external electronic device 1200b as shown in 1203 of FIG. 12 . there is. Based on recognizing the QR code 1231, the external electronic device 1200b may recognize that it is set as a sub device. The external electronic device 1200b requests a communication connection to the electronic device (eg, the host device 1200a) based on the QR code 1231, and the electronic device (eg, the host device 1200a) based on the request Thus, a communication connection with the external electronic device 1200b may be established. As a result, the host device 1200a displaying the QR code 1231 and the sub device 1200b capturing the QR code 1231 can create a specific group for generating a panoramic image (or as a specific group). can be classified). Meanwhile, not limited to what has been described and/or illustrated, the electronic device 1200a set as a host device automatically scans the surrounding external electronic devices 1200b based on the D2D communication method, and the scanned external electronic device ( 1200b) and a communication connection may be established.

According to various embodiments, the electronic device (eg, the host device 1200a) generates a panoramic image based on an image captured by the electronic device and at least one image received from at least one external electronic device in operation 1109. can For example, as shown in 1204 of FIG. 12 , the electronic device (eg, the host device 1200a) displays an execution screen for generating a panoramic image as described above with reference to FIGS. 9 to 10 based on the communication connection. can Operation 1109 for generating a panoramic image of the electronic device (eg, the host device 1200a) may be performed in the same manner as the operation of the above-described electronic device (eg, the host device 700a or 1010a). omit

Hereinafter, an example of an operation of an electronic device (eg, a host device) according to various embodiments will be described.

According to various embodiments, an electronic device (eg, a host device) determines parameters (eg, exposure time, etc.) of cameras of the devices (eg, a host device and a sub-device) before generating a panoramic image, so that the devices (eg, an exposure time, etc.) Example: It is possible to control visual characteristics of captured images for each host device and each sub device to correspond to each other.

13 is a flowchart 1300 for describing an example of an operation of a host device according to various embodiments. According to various embodiments, the operations shown in FIG. 13 are not limited to the order shown and may be performed in various orders. Also, according to various embodiments, more operations than the operations shown in FIG. 13 or at least one operation less than those shown in FIG. 13 may be performed.

According to various embodiments, the electronic device 400 (eg, a host device) establishes a communication connection with at least one external electronic device in operation 1301, acquires a first temporary image using a camera in operation 1303, and in operation 1305 In operation, a second temporary image may be received from an external electronic device connected through communication. The temporary image may be defined as an image obtained before the electronic device 400 (eg, the host device) sets values of parameters associated with the camera. As described above, the communication connection may be performed based on a camera application for generating a panoramic image, and the electronic device 400 (eg, a host device) uses the camera 410 to generate an image (eg, a first temporary image). image), and may receive images (eg, a second temporary image) from sub devices based on a communication connection.

According to various embodiments, the electronic device 400 (eg, the host device) may select an image from among the first temporary image and the second temporary image in operation 1307 . In an embodiment, the electronic device 400 (eg, a host device) may select a specific temporary image from among a plurality of temporary images (eg, a first temporary image and a second temporary image) based on a user's input. . For example, the electronic device 400 (eg, a host device) displays a plurality of temporary images (eg, a first temporary image and a second temporary image) acquired on an execution screen of an application (eg, a camera application). can be displayed The execution screen including the plurality of temporary images (eg, the first temporary image and the second temporary image) is an execution screen for generating a panoramic image (eg, the execution screen of FIGS. 10A and 10B , and/or FIG. The execution screen of 1204 of 12) may be an execution screen displayed before. The electronic device 400 (eg, the host device) may identify an image that the user selects (or touches) from among the plurality of temporary images (eg, the first temporary image and the second temporary image). In another embodiment, the electronic device 400 (eg, a host device) controls visual properties of a plurality of temporary images (eg, a first temporary image and a second temporary image) without (or automatically) a user's input. Based on the value analysis, a specific image may be selected from among a plurality of temporary images (eg, a first temporary image and a second temporary image). For example, the visual properties may include brightness, resolution, color, contrast, tone, and/or saturation of an image, without being limited thereto, and camera parameters (e.g., international standard organization (ISO) sensitivity, It may further include various types of visual attributes determined based on values of shutter speed and aperture. For example, the electronic device 400 (eg, the host device) analyzes the histogram of temporary images (eg, the first temporary image and the second temporary image), and selects a temporary image having relatively better properties. You can choose.

According to various embodiments, the electronic device 400 (eg, host device) determines whether the selected image is the first temporary image in operation 1309, and if the selected image is the first temporary image (1309-yes), operation 1311 may transmit a value of a parameter associated with a camera of the electronic device 400 to an external electronic device. For example, the electronic device 400 (eg, the host device) captures a plurality of temporary images (eg, a first temporary image and a second temporary image) by the electronic device 400 (eg, the host device). When an image (eg, the first temporary image) is selected, values of parameters related to the camera (eg, ISO sensitivity, shutter speed, and aperture) of the electronic device 400 (eg, the host device) are transferred to other sub-devices. can transmit Accordingly, the sub device sets the camera parameter value to the parameter value related to the camera (eg ISO sensitivity, shutter speed, aperture) of the electronic device 400 (eg host device) received, and captures an image. can

According to various embodiments, when the selected image is not the first temporary image (1309 - No), the electronic device 400 (eg, the host device) determines whether the selected image is a second temporary image in operation 1313; When the image selected in operation 1315 is the second temporary image, a parameter value associated with at least one camera of the external electronic device may be received from the external electronic device. For example, the electronic device 400 (eg, a host device) selects an image captured by another sub device (eg, a second temporary image) among a plurality of temporary images (eg, a first temporary image and a second temporary image). ) is selected, parameters related to the camera (eg, ISO sensitivity, shutter speed, aperture) may be requested from other sub-devices. The electronic device 400 (eg, host device) receives values of parameters related to the camera (eg, ISO sensitivity, shutter speed, and aperture) from other sub-devices, and transmits the values of parameters related to the camera (eg, ISO speed, shutter speed, An image may be captured by setting a parameter value of a camera to a parameter value related to an aperture).

According to various embodiments, in operation 1317, the electronic device 400 (eg, the host device) may generate a panoramic image based on an image captured by the electronic device and an image received from an external electronic device. For example, the electronic device 400 (eg, a host device) may perform an operation of acquiring a plurality of images after camera parameters are reset and generating a panoramic image based on the acquisition of the plurality of images. . Operation 1317 for generating a panoramic image of the electronic device 400 (eg, the host device) may be performed in the same way as the operation of the above-described electronic device (eg, the host device 700a or 1010a). omit

Meanwhile, the above-described operation of setting the values of parameters of the camera may be performed periodically and/or non-periodically while the operation of generating the panoramic image is performed. That is, while the group including the host device and the sub device is maintained, the above-described operation of setting the parameter value of the camera may be periodically performed. Alternatively, when a new sub-device joins a group while a group including a host device and a sub-device is maintained, or an existing sub-device leaves the group (eg, camera application termination, sub-device turn-off), An operation of setting a value of a parameter of one camera may be performed.

Hereinafter, an example of an operation of an electronic device (eg, a host device) according to various embodiments will be described.

According to various embodiments, an electronic device (eg, a host device) identifies locations (eg, two-dimensional coordinates) of devices (eg, a host device and a sub device) based on feature points of a plurality of acquired images, and , It is possible to determine whether the location of the sub device needs to be changed based on the reference location generated based on the location of the host device. When the location of a sub-device needs to be changed, the electronic device (eg, the host device) may transmit location guide information for changing the location to the corresponding sub-device.

14 is a flowchart 1400 for describing an example of an operation of a host device according to various embodiments. According to various embodiments, the operations shown in FIG. 14 may be performed in various orders without being limited to the order shown. Also, according to various embodiments, more operations than the operations shown in FIG. 14 or at least one operation less may be performed. 14 will be described below with reference to FIGS. 15A to 15D.

15A illustrates an example of an operation of identifying locations of devices (eg, a host device and a sub device) based on feature points of a plurality of images of an electronic device (eg, a host device) according to various embodiments. It is a drawing for 15B is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments. 15C is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments. 15D is a diagram for explaining an example of an operation of providing location guide information for changing a location of a sub device of an electronic device (eg, a host device) according to various embodiments.

According to various embodiments, the electronic device 1500a (eg, a host device) acquires a first image 1510a using a camera in operation 1401 and, in operation 1403, communicates with the external electronic devices 1500b and 1500c (eg, : At least one second image 1510b or 1510c may be received from the sub device).

According to various embodiments, in operation 1405, the electronic device 1500a (eg, a host device) selects first key-points (or feature points) associated with the first image and at least one second key-point associated with the second image. Based on the points (or feature points), a first location of the electronic device 1500a (eg, a host device) may be identified and a second location of the external electronic devices 1500b and 1500c (eg, sub devices) may be identified. can For example, the electronic device 1500a (eg, the host device) compares the image 1510a captured by the electronic device 1500a (eg, the host device) with feature points of the images 1510b and 1510c to be synthesized. Based on this, the position 1520a (eg, 2D coordinates) of the electronic device 1500a (eg, the host device) and the positions 1520b and 1520c (eg, 2D coordinates) of the sub-devices 1500b and 1500c coordinates) can be identified. For example, the position 1510a (eg, two-dimensional coordinates) of the electronic device 1500a (eg, the host device) when the electronic device 1500a (eg, the host device) is observed in a direction substantially perpendicular to the ground. can decide As shown in 1501 of FIG. 15A, the electronic device 1500a (eg, a host device) receives information (eg, information on feature points indicating the outline and/or corner of an object for each of the received images 1510a, 1510b, and 1510c). Coordinates of feature points) may be identified, and differences (d1, d2) may be identified based on a result of comparing the identified feature points. For example, referring to 1501 of FIG. 15A , the electronic device 1500a (eg, a host device) may include two images (eg, an image 1510a and an image 1510b) at least partially overlapping (ie, to be synthesized), or Based on the comparison of the coordinates of the feature points of the image 1510a and the image 1510c, the feature points of each image (eg, the image 1510a and the image 1510b, or the image 1510a and the image 1510c) (eg, between the feature points 1511a and 1511b or the feature points 1511a and 1511c) in the x-axis direction and the difference (y1) in the y-axis direction may be identified. Here, the x-axis and the y-axis are axes defining the ground, and the z-axis may be an axis perpendicular to the ground. The electronic device 1500a (eg, the host device) determines the difference (x1, y1) in the x-axis direction and the y-axis direction identified based on the comparison result and the pre-determined value of the electronic device 1500a (eg, the host device). Based on the position 1520a, relative positions 1520b and 1520c (eg, 2D coordinates) of the other sub-devices 1500b and 1500c may be identified as shown in 1502 of FIG. 15A. For example, the electronic device 1500a (eg host device) calculates the difference (x1) in the x-axis direction to the x-axis coordinate of the electronic device 1500a (eg host device) as shown in [Equation 1] below. Based on subtracting the difference (y1) in the y-axis direction from the y-axis coordinate of the electronic device 1500a (eg, the host device) as shown in [Equation 2] below, the compared image 1510b or 1510c It is possible to identify the positions 1520b and 1520c (eg, 2-dimensional coordinates) of the sub-devices 1500b and 1500c that have taken the picture.

[Equation 1]

, α is the proportionality constant

[Equation 2]

, β is the proportionality constant

On the other hand, [Equation 1] and [Equation 2] are only examples and are not limited to what has been described, and the electronic device 1500a (eg, the host device) determines the positional difference between the feature points (d1, d2) of the sub device. location can be identified. [Equation 1] and [Equation 2] may be implemented to include more equations for calculating the x-axis coordinate and/or the y-axis coordinate of the sub device, in addition to the above description.

Meanwhile, although it has been described that the electronic device 1500a (eg, host device) determines the location of the other sub-devices 1500b and 1500c based on the position of the electronic device 1500a (eg, host device), it is not limited thereto. It is also possible to perform an operation of identifying the location of another adjacent sub device based on the location of the sub device. For example, although not shown, when a third sub-device (not shown) is located on the left side of the first sub-device 1500b and an image is captured by the third sub-device (not shown) accordingly, the third sub-device 1500b (not shown) may overlap the image and at least a part of the image 1510b captured by the first sub-device 1500b. The electronic device 1500a (eg, the host device) reflects the identified difference to the identified position 1520b of the first sub-device 1500b based on the comparison result of the feature points of the images, and the third sub-device (not shown). position) can be identified. In other words, the electronic device 1500a (eg, the host device) may identify the location of the sub-devices based on differences in feature points of images to be synthesized (or at least partially overlapping) regardless of the number of sub-devices. there is.

According to various embodiments, in operation 1407, the electronic device 1500a (eg, a host device) selects between a reference location identified based on the first location, an area where the external electronic device will be located, and the first location or the second location. At least one can be compared. For example, the electronic device 1500a (eg, host device) determines the locations 1520b and 1520c of other sub-devices 1500b and 1500c based on the location 1520a of the electronic device 1500a (eg, host device). A reference position 1500 to be compared with may be determined, and the determined reference position 1500 may be compared with the positions 1520b and 1520c of the identified sub-devices 1500b and 1500c. For example, referring to FIGS. 15B, 15C, and 15D , the reference position 1500 is on the diameter of a circle 1500 centered on the position 1500a of the electronic device 1500a (eg, a host device). It may be a preset position (1550a, 1550b, 1550c). When images are captured by the plurality of devices 1500a, 1500b, and 1500c in a state in which the plurality of devices 1500a, 1500b, and 1500c are disposed on the circle 1500, enabling the creation of a panoramic image Images can be taken. That is, when photographing is performed while being disposed on the circle 1500, at least a portion of images photographed between the adjacent devices may overlap each other. Meanwhile, the radius (or diameter) of the circle 1500 may be set in proportion to the number of devices (eg, host devices and sub devices) in the group. On the other hand, without being limited to what is shown and/or described, in addition to the circle 1500, various shapes (eg, a semicircle, a cylinder, an ellipse, a rectangle) determined based on a position 1520a of the electronic device 1500a (eg, a host device) , a rectangle, a polygon such as a rhombus) may be determined. The electronic device 1500a (eg host device) determines the coordinates of the circle 1500 (eg a predetermined position on the diameter) and the position (eg x coordinates and/or y coordinates) are different, it may be determined that the positions of the sub-devices 1500b and 1500c need to be changed.

According to various embodiments, the electronic device 400 (eg, a host device) may provide information for changing the first location or the second location based on the comparison result in operation 1409 . For example, when the reference position 1500 of the electronic device 400 (eg, a host device) and the positions 1520b and 1520c of the sub-devices 1500b and 1500c are different from each other, FIGS. 15B, 15C, and As shown in FIG. 15D, information for changing the location of the sub-devices 1500b and 1500c may be provided to the sub-devices 1500b and 1500c. Referring to FIGS. 15B, 15C, and 15D, the information includes first graphic objects O2 and O3 representing a reference position 1500 where sub-devices are to be located, and locations of sub-devices 1500b and 1500c. Information for displaying the second graphic objects P2 and P3 may be included. On the other hand, as shown, the information may further include information for displaying a third graphic object (eg, an arrow) indicating the direction of the sub-devices 1500b and 1500c. The first graphic object includes points 1550a, 1550b, and 1550c where the sub devices 1500b and 1500c are to be positioned, and the corresponding points 1550a, 1550b, and 1550c may be displayed in a highlighted state. Each user of the sub-devices 1500b and 1500c may move the sub-devices 1500b and 1500c according to the positions of the first graphic objects O2 and O3 and the second graphic objects P2 and P3. According to the sequential movement, the electronic device 400 (eg, the host device) updates the information for changing the location and provides it to the sub devices 1500b and 1500c, as shown in FIGS. 15B, 15C, and 15D. And, the sub-devices 1500b and 1500c may also update (eg, change their location) and display the first graphic objects O2 and O3 and/or the second graphic objects P2 and P3 based on the updated information. there is. As a result, the user of the sub-devices 1500b and 1500c can arrange the sub-devices 1500b and 1500c on the reference position 1500.

Meanwhile, as shown in FIG. 15B, in order to move the position of the electronic device 400 (eg host device) to the reference position 1500, a first graphic object ( O1) and the second graphic object P1 may be displayed.

According to various embodiments, the electronic device (eg, the host device 1500a) generates a panoramic image based on an image captured by the electronic device and at least one image received from at least one external electronic device in operation 1411. can An image obtained after an electronic device (eg, the host device 1500a) is identified (or completed) as having arranged (or aligned) devices (eg, the host device and sub-devices) on the reference position 1500. Based on them, an operation of generating a panoramic image may be performed. Operation 1411 for generating a panoramic image of the electronic device (eg, the host device 1500a) may be performed in the same manner as the operation of the above-described electronic device (eg, the host device 700a or 1010a). omit

Hereinafter, an example of an operation of an electronic device (eg, a host device) according to various embodiments will be described.

According to various embodiments of the present disclosure, an electronic device (eg, a host device) identifies an overlapping degree of captured images for each device (eg, a host device and a sub device) based on feature points of a plurality of acquired images, and identifies the image. Based on the overlapped degree, it may be determined whether the direction of the sub device (eg, the angle between the x-direction, y-direction, and/or z-direction axis and the center line of the camera of the sub device) is changed. An electronic device (e.g., a host device) is required to change the direction of a sub-device (e.g., the angle between an axis in the x-direction, y-direction, and/or z-direction and the center line of a camera of the sub-device) to the sub-device. Direction guide information for change may be provided. Here, the x-axis and the y-axis are axes defining the ground, and the z-axis may be an axis perpendicular to the ground.

Meanwhile, the operation of providing direction guide information of the electronic device (eg, the host device) may be sequentially performed after the above-described operation of providing location guide information is completed. In other words, after the images captured by the electronic device and the sub-device overlap each other at least partially, an operation of providing direction guide information of the electronic device (eg, the host device) may be performed.

16 is a flowchart 1600 for describing an example of an operation of a host device according to various embodiments. According to various embodiments, the operations illustrated in FIG. 16 may be performed in various orders without being limited to the illustrated order. Also, according to various embodiments, more operations than the operations shown in FIG. 16 or at least one operation less may be performed. 16 will be described below with reference to FIGS. 17A to 17C.

17A is a diagram for explaining an example of an operation of providing direction guide information for changing a direction of a sub device of an electronic device (eg, a host device) according to various embodiments. 17B is a diagram for explaining an example of an operation of providing direction guide information for changing a direction of a sub device of an electronic device (eg, a host device) according to various embodiments. 17C is a diagram for explaining an example of an operation of identifying a direction to be changed of a sub device based on an overlapping degree of images of an electronic device (eg, a host device) according to various embodiments.

According to various embodiments, the electronic device 1700a (eg, a host device) obtains a first image using a camera in operation 1601 and, in operation 1603, obtains a first image from an external electronic device (eg, a sub device) connected to communication with at least One second image and information on the first direction of the external electronic devices 1700b and 1700c (eg, sub devices) may be received. For example, each of the devices (eg, the host device 1700a and the sub-devices 1700b and 1700c) is based on a rotation direction identified using a sensor (not shown) (eg, an acceleration sensor and a tilt sensor), A first angle between the center line of the camera and the axis in the x direction, a second angle between the center line of the camera and the axis in the y direction, and/or a third angle between the center line of the camera and the axis in the z direction may be identified. The electronic device 1700a (eg, a host device) transmits information about directions identified by the sub-devices 1700b and 1700c (eg, images 1710b and 1710c) received from the communication-connected sub-devices 1700b and 1700c. : The first angle, the second angle, and/or the third angle) may be received.

According to various embodiments, the electronic device (eg, the host device 1500a), in operation 1605, based on first key-points associated with the first image and second key-points associated with at least one second image, , it is possible to identify the degree of overlap between the first image and the second image. For example, referring to FIG. 17A , the overlapped area between an image 1710a captured by an electronic device (eg, the host device 1500a) and an image 1710c captured by the second sub-device 1700c When the size 1711 is larger or smaller than a preset range, it may be inappropriate to combine the images 1710a and 1710c to create a panoramic image. Therefore, as shown in FIG. 18, the electronic device (eg, the host device 1500a) displays the horizontal axis (eg, the images 1710a, 1710b or the images 1710a, 1710c) to be composited with each other. x-axis) and/or overlapping areas (ah, av) on the vertical axis (eg, y-axis).

For example, referring to 1701 of FIG. 17C, the electronic device (eg, the host device 1500a) sets feature points of the first image 1710a and the second image 1710c in a horizontal direction (eg, the x-axis direction). Based on matching (eg, matching feature points in the x-axis direction) with , it is possible to identify the size (ah) of overlapping images in the horizontal direction (eg, the x-axis direction).

Also, for example, referring to 1702 of FIG. 17C, the electronic device (eg, the host device 1500a) sets feature points of the first image 1710a and the second image 1710c in a vertical direction (eg, the z-axis direction). ), it is possible to identify an overlapping size (av) of images in a vertical direction (eg, z-axis direction) based on matching (eg, matching feature points in the z-axis direction).

According to various embodiments, the electronic device (eg, the host device 1500a) identifies information about the second direction to be changed based on the information about the overlapping degree and the first direction identified in operation 1607, and in operation 1609 Information on the identified second direction may be transmitted to an external electronic device. For example, the electronic device may determine an angle of rotation (e.g., a yaw angle associated with the z-axis in the case of a horizontal axis, or an x-axis in the case of a vertical axis) to be changed about an axis perpendicular to each axis based on the identified overlap size and a pre-set overlap size. roll angle relative to the axis). The electronic device (eg, the host device 1500a) calculates an angular proportional constant to the difference between the previously set target overlapping size and the identified current overlapping size for each axis (eg, the horizontal axis and the vertical axis) as shown in [Equation 3] below. By reflecting, an angle for each axis to be rotated (eg, a yaw angle associated with the z-axis and/or a roll angle associated with the x-axis) can be identified.

[Equation 3]

In [Equation 3], to is the target overlap size (eg mh, mv), co is the current overlap size (eg ah, av), and the angular proportionality constant is the angle associated with the corresponding axis corresponding to the unit overlap size ( e.g. in the case of the x-axis-y-axis plane, associated with the z-axis). For example, the angular proportionality constant is a variation of an angle associated with each axis (eg, a yaw angle associated with the z-axis or a roll angle associated with the x-axis) received from the sub-device 1700c. It can be calculated based on dividing by the amount of change in size (eg, the difference between ah and bh, or the difference between av and bv). The amount of change of the angle and the amount of change of the overlapped size may be calculated as the electronic device 1700a receives information on a direction and an image from the sub device 1700c in real time.

The electronic device (for example, the host device 1500a) provides first information for displaying the first graphic object 1701a corresponding to the current rotation angle in the first position and the rotation direction to be changed (for example, based on the identified z-axis). Rotation guide information including second information for displaying the second graphic object 1701b corresponding to the yaw angle and/or the roll angle based on the x-axis) at the second position is generated, and the generated rotation guide information is It can be transmitted to an external electronic device (eg, the second sub device 1700c). For example, the electronic device (eg, the host device 1500a) receives a current direction (eg, the current yaw angle, roll angle, or pitch angle) received from the external electronic device (eg, the second sub-device 1700c) at least. 1) may identify (or obtain) a first graphic object 1701a having a first position corresponding to the first position. Also, for example, the electronic device (eg, the host device 1500a) may change the direction to be changed (eg, [Equation 3]) to the current direction (eg, at least one of a current yaw angle, a roll angle, or a pitch angle). The second graphic object 1701b having a second position corresponding to the identified direction may be identified by reflecting the yaw angle associated with the z-axis and/or the roll angle associated with the x-axis calculated according to . Based on the rotation guide information, the second sub-device 1700c may display a first graphic object 1701a corresponding to the current direction and a second graphic object 1701b corresponding to the direction to be changed. The user of the second sub-device 1700c may rotate the second sub-device 1700c so that the position of the first graphic object 1701a corresponds to the position of the second graphic object 1701b. As a result, referring to FIG. 17B, when the position of the first graphic object 1701a corresponds to the position of the second graphic object 1701b, the overlapped size 1712 between the images to be synthesized 1710a and 1710c A condition for generating a panoramic image may be satisfied.

According to various embodiments, an electronic device (eg, the host device 1500a) may generate a panoramic image based on an image captured by the electronic device and at least one image received from at least one external electronic device in operation 1611. can The electronic device (eg, the host device 1500a) may perform an operation of generating a panoramic image based on the acquired images after it is identified that the change in the direction of the sub device is completed. Operation 1611 for generating a panoramic image of the electronic device (eg, the host device 1500a) may be performed in the same manner as the operation of the above-described electronic device (eg, the host device 700a or 1010a). omit

According to various embodiments, in an electronic device (eg, the first electronic device 201 of FIG. 2 ), a camera (eg, the camera 410 of FIG. 4 ), a communication circuit (eg, the communication circuit of FIG. 4 ( 430)), and at least one processor (eg, the processor 440 of FIG. 4), and the at least one processor (eg, the processor 440 of FIG. 4) includes the camera (eg, the camera (eg, the camera of FIG. 4) 410)) to obtain a first image, and communicate with the electronic device (eg, the first electronic device 201 of FIG. 2) through the communication circuit (eg, the communication circuit 430 of FIG. 4). Receiving a second image from a connected external electronic device, identifying first information about a pose of the external electronic device based on the at least one second image, wherein the first information about the pose is a location or a direction information on at least one of the external electronic device, based on at least one first feature point identified from the first image and at least one second feature point identified from the second image, in order to generate a panoramic image; It is determined whether the change of the pose is necessary, and when it is determined that the change of the pose is necessary, second information for changing the pose is transmitted to the external electronic device, and the second information is the location or the direction includes information for displaying a graphic object representing a change of at least one of, receives a third image from the external electronic device after transmitting the second information, and based on the first image and the third image , An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to generate the panoramic image may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) is configured to establish a communication connection with at least one electronic device based on execution of an application for generating the panoramic image. The at least one electronic device includes the external electronic device, and the electronic device (eg, the first electronic device 201 of FIG. 2 ) and the at least one electronic device connected through communication are included in a specific group, , The electronic device (eg, the first electronic device 201 of FIG. 2 ) within the specific group is set as a host device, and the at least one electronic device is set as a sub device (eg, the electronic device (eg, the first electronic device 201 in FIG. 2 ). The first electronic device 201 of may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) is set to display a code for establishing a communication connection with the at least one electronic device on the execution screen of the application. An electronic device (eg, the first electronic device 201 of FIG. 2 ) may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) identifies an event for generating the panoramic image using the application, and the electronic device (eg, the processor 440 of FIG. 4 ) An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to set the first electronic device 201 of FIG. 2 as a host device may be provided.

According to various embodiments, when the at least one processor (eg, the processor 440 of FIG. 4 ) determines that a change in position during the pose is required based on the first image and the second image, the An execution screen including a first area including a first image and a second image and a second area including a first panoramic image generated based on the first image is displayed, and the first image and the second area are displayed. When it is determined that a change in direction during the pause is necessary based on 2 images, the first region including the first image and the second image and the first region generated based on the first image and the second image An electronic device configured to display the execution screen including the second area including 2 panoramic images, and visually highlighting an area corresponding to the at least one second image of the second panoramic image (eg: The first electronic device 201 of FIG. 2 may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) acquires a first temporary image using the camera (eg, the camera 410 of FIG. 4 ), and the external A second temporary image is received from an electronic device, a specific temporary image is selected from among the first temporary image and the second temporary image, and when the selected specific temporary image is the first temporary image, the external electronic device transmits the camera A first value of a parameter (e.g., the camera 410 of FIG. 4) is transmitted to the external electronic device, and the parameter includes at least one of shutter speed, aperture, or ISO sensitivity. The second image captured based on the first value of the parameter of the camera (eg, the camera 410 of FIG. 4 ) is received, and when the selected specific temporary image is the second temporary image, from the external electronic device. A second value of the parameter of the camera (eg, camera 410 of FIG. 4 ) is received from the external electronic device, and a first value of the parameter of the camera (eg, camera 410 of FIG. 4 ) An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to obtain the first image using the camera (eg, the camera 410 of FIG. 4 ) based on the electronic device may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) may generate at least one first feature point identified from the first image and at least one second feature point identified from the second image. Based on the characteristic points, a first location of the electronic device (eg, the first electronic device 201 of FIG. 2 ) and a second location of the external electronic device are identified, and based on the first location, a reference location is determined. and, based on comparing the identified reference position and the second position, it is determined whether or not the position of the external electronic device needs to be changed, and when it is determined that the position needs to be changed, the second position An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to transmit information for displaying at least one graphic object for guiding the object to the reference position to the external electronic device may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) determines the first location of the electronic device (eg, the first electronic device 201 of FIG. 2 ), Identifying the second position of the external electronic device based on identifying a difference between the position of the at least one first feature point and the position of the at least one second feature point, and reflecting the identified difference to the first position An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to do this may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) is centered on the first location of the electronic device (eg, the first electronic device 201 of FIG. 2 ) and , An electronic device configured to determine a circle having a specific radius as the reference position and transmit information for displaying the at least one graphic object for guiding the second position to a position on the circle to the external electronic device. (eg, the first electronic device 201 of FIG. 2) may be provided.

According to various embodiments, the direction includes at least one of a roll angle, a yaw angle, and a pitch angle of a center line of a camera (eg, the camera 410 of FIG. 4 ) of the external electronic device, and the at least one A processor (eg, the processor 440 of FIG. 4 ) determines the first image and the second feature based on at least one first feature point identified from the first image and at least one second feature point identified from the second image. Identifying the size of the area where the second image overlaps, determining whether or not the direction of the external electronic device needs to be changed based on comparing the identified size with a preset size, and determining whether the location needs to be changed If it is determined that it is, an electronic device configured to transmit information for displaying at least one graphic object for changing at least one of a roll angle, a yaw angle, or a pitch angle to the external electronic device (e.g., FIG. 2 ). 1 electronic device 201) may be provided.

According to various embodiments, the at least one processor (eg, the processor 440 of FIG. 4 ) based on the difference between the identified size and the preset size, as the value of the direction to be changed, the roll angle; An electronic device (eg, the first electronic device 201 of FIG. 2 ) in which at least one of the yaw angle and the pitch angle is set to identify an angle to be changed may be provided.

According to various embodiments, the at least one device having a location corresponding to the direction identified based on receiving first information on a current direction from the external electronic device and reflecting a value of the direction to be changed to the current direction. An electronic device (eg, the first electronic device 201 of FIG. 2 ) configured to obtain a graphic object of may be provided.

According to various embodiments, in a method of operating an electronic device (eg, the first electronic device 201 of FIG. 2 ), a first image is obtained using the camera (eg, the camera 410 of FIG. 4 ). receiving a second image from an external electronic device communicatively connected to the electronic device (eg, the first electronic device 201 of FIG. 2 ) through the communication circuit (eg, the communication circuit 430 of FIG. 4 ) and identifying first information about a pose of the external electronic device based on the at least one second image, wherein the first information about the pose is information about at least one of a position and a direction In order to generate a panoramic image, the pose of the external electronic device is changed based on at least one first feature point identified from the first image and at least one second feature point identified from the second image. An operation of determining whether the change of the pose is necessary, and an operation of transmitting second information for changing the pose to the external electronic device when it is determined that the change of the pose is necessary, wherein the second information includes the location or the direction an operation of receiving a third image from the external electronic device after transmitting the second information, and including information for displaying a graphic object representing a change of at least one of Based on the above, an operation method including an operation of generating the panoramic image may be provided.

According to various embodiments, an operation of establishing a communication connection with at least one electronic device based on execution of an application for generating the panoramic image, wherein the at least one electronic device includes the external electronic device. and the communicatively connected electronic device (eg, the first electronic device 201 of FIG. 2 ) and the at least one electronic device are included in a specific group, and within the specific group, the electronic device (eg, the first electronic device 201 of FIG. 2 ) An operating method may be provided in which the first electronic device 201 is set as a host device and the at least one electronic device is set as a sub device.

According to various embodiments, an operating method configured to display a code for establishing a communication connection with the at least one electronic device on an execution screen of the application may be provided.

According to various embodiments, setting the electronic device (eg, the first electronic device 201 of FIG. 2 ) as a host device based on identifying an event for generating the panoramic image using the application. A method of operation, including an operation, may be provided.

According to various embodiments, when it is determined that the location of the pose needs to be changed based on the first image and the second image, the first area including the first image and the second image and the second image. An operation of displaying an execution screen including a second area including a first panoramic image generated based on 1 image, and determining that a change in direction during the pause is necessary based on the first image and the second image , the execution including the first area including the first image and the second image and the second area including a second panoramic image generated based on the first image and the second image. A method of visually highlighting a region corresponding to the at least one second image of the second panoramic image, including displaying a screen, may be provided.

According to various embodiments, an operation of obtaining a first temporary image by using the camera (eg, the camera 410 of FIG. 4 ), an operation of receiving a second temporary image from the external electronic device, and an operation of receiving the first temporary image from the external electronic device. An operation of selecting a specific temporary image from among an image and the second temporary image, and when the selected specific temporary image is the first temporary image, parameters of the camera (eg, the camera 410 of FIG. 4 ) by the external electronic device. and transmitting a first value to the external electronic device, wherein the parameter includes at least one of a shutter speed, an aperture, or an ISO sensitivity, from the external electronic device to the camera (eg, the camera 410 of FIG. 4 ). )), receiving the second image captured based on the first value of the parameter, and when the selected specific temporary image is the second temporary image, the camera (eg, the camera of FIG. 4 ) from the external electronic device. operation of receiving the second value of the parameter of (410)) from the external electronic device, and the camera (eg, camera 410 of FIG. 4) based on the first value of the parameter; An operation method including an operation of acquiring the first image using the camera 410 of FIG. 4 may be provided.

According to various embodiments, based on at least one first feature point identified from the first image and at least one second feature point identified from the second image, the electronic device (eg, the first electronic device of FIG. 2 ) Identifying a first location of the device 201 and a second location of the external electronic device, identifying a reference location based on the first location, and comparing the identified reference location and the second location. Based on this, an operation of determining whether the location of the external electronic device needs to be changed, and if it is determined that the location needs to be changed, at least one graphic object for guiding the second location to the reference location An operation method including an operation of transmitting information for displaying to the external electronic device may be provided.

According to various embodiments, the operation of determining the first position of the electronic device (eg, the first electronic device 201 of FIG. 2 ), the position of the at least one first feature point and the at least one second feature point An operating method may be provided, including an operation of identifying a difference in position of a feature point, and an operation of identifying the second position of the external electronic device based on reflecting the identified difference to the first position.

Claims (20)

In electronic devices,
camera;
communication circuit;
at least one processor, wherein the at least one processor:
Obtaining a first image using the camera;
Receiving a second image from an external electronic device communicatively connected to the electronic device through the communication circuit;
Based on the at least one second image, first information about a pose of the external electronic device is identified, the first information about the pose includes information about at least one of a location or a direction,
To generate a panoramic image, it is determined whether the pose of the external electronic device needs to be changed based on at least one first feature point identified from the first image and at least one second feature point identified from the second image. do,
When it is determined that the pose needs to be changed, second information for changing the pose is transmitted to the external electronic device, and the second information displays a graphic object representing a change in at least one of the location and the direction. contains information for
After transmitting the second information, receiving a third image from the external electronic device;
Based on the first image and the third image, set to generate the panoramic image,
electronic device.
According to claim 1,
The at least one processor is:
Based on the execution of the application for generating the panoramic image, a communication connection is established with at least one external electronic device, wherein the at least one external electronic device includes the external electronic device;
The communicatively connected electronic device and the at least one external electronic device are included in a specific group, the electronic device within the specific group is set as a host device, and the at least one external electronic device is set as a sub device.
electronic device.
According to claim 2,
The at least one processor is:
Set to display a code for establishing a communication connection with the at least one external electronic device on the execution screen of the application,
electronic device.
According to claim 2,
The at least one processor is:
Set to set the electronic device as a host device based on identifying an event for generating the panoramic image using the application,
electronic device.
According to claim 1,
The at least one processor is:
When it is determined that the position of the pose needs to be changed based on the first image and the second image, a first area including the first image and the second image and a first image generated based on the first image Displaying an execution screen including a second area including a first panoramic image;
When it is determined that a change in direction during the pause is necessary based on the first image and the second image, the first area including the first image and the second image and the first image and the second image set to display the execution screen including the second area including a second panoramic image generated based on an image, wherein an area corresponding to the at least one second image of the second panoramic image is visually highlighted; felled,
electronic device.
According to claim 1,
The at least one processor is:
Obtaining a first temporary image using the camera;
Receiving a second temporary image from the external electronic device;
Selecting a specific temporary image from among the first temporary image and the second temporary image;
When the selected specific temporary image is the first temporary image:
Transmitting a first value of a parameter of the camera to the external electronic device to the external electronic device, wherein the parameter includes at least one of shutter speed, aperture, or ISO sensitivity.
Receiving the second image captured based on the first value of the parameter of the camera from the external electronic device;
When the selected specific temporary image is the second temporary image:
Receiving a second value of the parameter of the camera from the external electronic device,
Set to acquire the first image using the camera based on the first value of the parameter of the camera,
electronic device.
According to claim 1,
The at least one processor is:
Based on at least one first feature point identified from the first image and at least one second feature point identified from the second image, a first location of the electronic device and a second location of the external electronic device are identified; ,
Based on the first position, identify a reference position;
Based on the comparison between the identified reference position and the second position, it is determined whether the position of the external electronic device needs to be changed;
When it is determined that the location needs to be changed, information for displaying at least one graphic object for guiding the second location to the reference location is transmitted to the external electronic device.
electronic device.
According to claim 7,
The at least one processor is:
determine the first location of the electronic device;
identifying a difference between a position of the at least one first feature point and a position of the at least one second feature point;
Set to identify the second location of the external electronic device based on reflecting the identified difference to the first location.
electronic device.
According to claim 7,
The at least one processor is:
determining a circle having a specific radius and having a center at the first position of the electronic device as the reference position;
configured to transmit information for displaying the at least one graphic object for guiding the second position to a position on the circle to the external electronic device;
electronic device.
The method of claim 1 , wherein the direction includes at least one of a roll angle, a yaw angle, and a pitch angle of a center line of a camera of the external electronic device,
The at least one processor is:
Based on at least one first feature point identified from the first image and at least one second feature point identified from the second image, a size of an area where the first image and the second image overlap is identified;
Based on the comparison between the identified size and a preset size, it is determined whether a change in the direction of the external electronic device is required;
When it is determined that the position needs to be changed, information for displaying at least one graphic object for changing at least one of the roll angle, the yaw angle, or the pitch angle is transmitted to the external electronic device,
electronic device.
According to claim 10,
The at least one processor is:
Based on the difference between the identified size and the preset size, at least one of the roll angle, the yaw angle, or the pitch angle is set to identify an angle to be changed as a value of the direction to be changed,
electronic device.
According to claim 11,
Receiving first information about a current direction from the external electronic device;
set to obtain the at least one graphic object having a position corresponding to the direction identified based on reflecting the value of the direction to be changed in the current direction;
electronic device.
In the operating method of the electronic device,
acquiring a first image using the camera;
receiving a second image from an external electronic device communicatively connected to the electronic device through the communication circuit;
and identifying first information about a pose of the external electronic device based on the at least one second image, wherein the first information about the pose includes information about at least one of a position and a direction. do,
To generate a panoramic image, it is determined whether the pose of the external electronic device needs to be changed based on at least one first feature point identified from the first image and at least one second feature point identified from the second image. action;
and transmitting second information for changing the pose to the external electronic device when it is determined that the change of the pose is necessary, wherein the second information indicates a change of at least one of the location and the direction. Including information for display of the graphic object that represents,
receiving a third image from the external electronic device after transmitting the second information; and
Based on the first image and the third image, generating the panoramic image; including,
how it works.
According to claim 13,
Based on the execution of the application for generating the panoramic image, establishing a communication connection with at least one external electronic device; wherein the at least one external electronic device includes the external electronic device;
The communicatively connected electronic device and the at least one external electronic device are included in a specific group, the electronic device within the specific group is set as a host device, and the at least one external electronic device is set as a sub device.
how it works.
15. The method of claim 14,
Set to display a code for establishing a communication connection with the at least one external electronic device on the execution screen of the application,
how it works.
15. The method of claim 14,
Based on identifying an event for generating the panoramic image using the application, setting the electronic device as a host device; including,
how it works.
According to claim 13,
When it is determined that the position of the pose needs to be changed based on the first image and the second image, a first area including the first image and the second image and a first image generated based on the first image displaying an execution screen including a second region including a first panoramic image; and
When it is determined that a change in direction during the pause is necessary based on the first image and the second image, the first area including the first image and the second image and the first image and the second image An operation of displaying the execution screen including the second area including the second panoramic image generated based on the image; including, an area corresponding to the at least one second image of the second panoramic image visually highlighted,
how it works.
According to claim 13,
obtaining a first temporary image using the camera;
receiving a second temporary image from the external electronic device;
selecting a specific temporary image from among the first temporary image and the second temporary image;
When the selected specific temporary image is the first temporary image:
Transmitting a first value of a parameter of the camera to the external electronic device to the external electronic device, wherein the parameter includes at least one of shutter speed, aperture, or ISO sensitivity.
receiving the second image captured based on the first value of the parameter of the camera from the external electronic device;
When the selected specific temporary image is the second temporary image:
receiving a second value of the parameter of the camera from the external electronic device; and
Acquiring the first image using the camera based on the first value of the parameter of the camera; including,
how it works.
According to claim 13,
Identifying a first location of the electronic device and a second location of the external electronic device based on at least one first feature point identified from the first image and at least one second feature point identified from the second image movement;
identifying a reference location based on the first location;
determining whether the location of the external electronic device needs to be changed based on comparing the identified reference location and the second location; and
When it is determined that the location needs to be changed, transmitting information for displaying at least one graphic object for guiding the second location to the reference location to the external electronic device;
how it works.
According to claim 19,
determining the first location of the electronic device;
identifying a difference between a position of the at least one first feature point and a position of the at least one second feature point; and
Based on reflecting the identified difference to the first location, identifying the second location of the external electronic device; including,
how it works.

Images