KR20220067034A - Method, apparatus and storage medium for providing slow shutter - Google Patents

Abstract

The present invention relates to a method, apparatus, and storage medium for providing a slow shutter to expand photographing ability of a user. According to various embodiments of the present invention, an electronic device comprises a memory, an image sensor, and at least one processor operatively coupled to the memory and the image sensor. The memory stores instructions to allow the at least one processor, during execution, to acquire a plurality of images including first images having a first size and at least one second image having a second size larger than the first size through the image sensor; acquire a first synthesized image on the basis of the first images; acquire a first moving object area from the first synthesized image on the basis of synthesized area information including at least one of moving object position information and background position information; identify a second moving object area in the at least one second image on the basis of the synthesized area information; and replace the second moving object area with the first moving object area to acquire a second synthesized image.

Description

METHOD, APPARATUS AND STORAGE MEDIUM FOR PROVIDING SLOW SHUTTER

Various embodiments relate to a method and apparatus for providing a slow shutter.

A Compact System Camera (CSC) can adjust parameters such as shutter speed, International Organization for Standardization (ISO), and aperture of the camera for proper exposure photography. A method of photographing for a long time and expressing the trajectory of a moving object or the flow of an object by adjusting parameters may be referred to as a slow shutter.

To implement the slow shutter function, the CSC can perform one read-out after exposure for a long time by adjusting the aperture.

Conventional slow shutter, shake control for a long time (required to use a tripod), and need know-how on precise exposure control to obtain a desired effect in various environments, it was a shooting area for experts.

In order to obtain the desired slow shutter effect, the photographer had to adjust the shutter speed and aperture from time to time depending on the shooting environment, and he had to continuously shoot.

According to various embodiments, the slow shutter may be provided using an electronic device such as a smart phone that does not have a variable aperture.

According to various embodiments, an electronic device includes a memory, an image sensor, and at least one processor operatively connected to the memory and the image sensor, wherein the memory, when executed, causes the at least one processor to cause , acquiring a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through the image sensor, and based on the first images to obtain a first composite image, and according to the composite area information including at least one of the moving object location information and the background location information, to obtain a first moving object area from the first composite image, and based on the composite area information , identifying a second moving object area in the at least one second image and replacing the second moving object area with the first moving object area to obtain a second composite image.

According to various embodiments, a method for providing a slow shutter includes a plurality of first images having a first size and at least one second image having a second size greater than the first size through an image sensor. from the first composite image based on composite area information including at least one of an operation of acquiring images of obtaining a first movable body area, identifying a second movable body area in the at least one second image based on the composite area information, and replacing the second movable body area with the first movable body area; and acquiring a second composite image.

According to various embodiments, in a non-transitory storage medium storing instructions, the instructions are configured to cause the at least one processor to perform at least one operation when the instructions are executed by the at least one processor. The operation may include acquiring a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through an image sensor, the first images obtaining a first composite image based on , acquiring a first moving body region from the first composite image based on composite region information including at least one of moving object position information and background position information; identifying a second moving object region in the at least one second image based on the information, and obtaining a second composite image by replacing the second moving object region with the first moving object region have.

According to various embodiments, since a slow shutter function is provided through shooting for a short period of time, it is possible to provide a user with an enlarged photographic ability.

According to various embodiments, since the slow shutter function is provided based on the motion analysis of the object, it is possible to provide an appropriate brightness and a synthesis amount.

1 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;
2A is a block diagram illustrating a camera module, according to various embodiments.
2B is a block diagram illustrating a structure of an image sensor according to various embodiments.
3A and 3B are diagrams for explaining a slow shutter function.
4 is a block diagram of an electronic device according to various embodiments of the present disclosure;
5 is a flowchart illustrating a method for providing a slow shutter according to various embodiments.
6 is a diagram for describing an operation of acquiring images in a slow shutter mode according to various embodiments of the present disclosure;
7 is a diagram illustrating a first image and a second image according to various embodiments of the present disclosure;
8 is a diagram illustrating a configuration of an image frame according to various embodiments of the present disclosure;
9 is a diagram for describing an operation of an image signal processor according to various embodiments of the present disclosure;
10A to 10C are diagrams for explaining an operation of obtaining composite area information according to various embodiments.
11 is a flowchart illustrating a method for providing a slow shutter according to various embodiments.
12A to 12B are diagrams for explaining a slow shutter mode according to various embodiments of the present disclosure;
13 is a view for explaining an operation of a slow shutter according to various embodiments of the present disclosure;
14A to 14C are diagrams for explaining an image synthesizing operation according to various embodiments of the present disclosure;

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 may use less power than the main processor 121 or may be set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the co-processor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used in a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . A sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 includes various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

FIG. 2A is a block diagram 200 illustrating a camera module 201 (eg, the camera module 180 of FIG. 1 ), in accordance with various embodiments. Referring to FIG. 2A , the camera module 201 includes a lens assembly 210 , a flash 220 , an image sensor 230 , an image stabilizer 240 , a memory 250 (eg, a buffer memory), or an image signal processor. (image signal processor: ISP, 260) may be included. The lens assembly 210 may collect light emitted from a subject, which is an image to be captured. The lens assembly 210 may include one or more lenses. According to various embodiments, the camera module 201 may include a plurality of lens assemblies 210 . In this case, the camera module 201 may form, for example, a dual camera, a 360 degree camera, or a spherical camera. Some of the plurality of lens assemblies 210 may have the same lens properties (eg, angle of view, focal length, auto focus, f number, or optical zoom), or at least one lens assembly may be a different lens assembly. It may have one or more lens properties different from the lens properties of . The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.

The flash 220 may emit light used to enhance light emitted or reflected from the subject. According to various embodiments, the flash 220 may include one or more light emitting diodes (eg, a red-green-blue (RGB) LED, a white LED, an infrared LED, or an ultraviolet LED), or a xenon lamp. The image sensor 230 may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly 210 into an electrical signal. According to various embodiments, the image sensor 230 may include, for example, one image sensor selected from image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor; It may include a plurality of image sensors having the same property or a plurality of image sensors having different properties. Each image sensor included in the image sensor 230 may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

In response to the movement of the camera module 201 or the electronic device 201 including the same, the image stabilizer 240 moves at least one lens or the image sensor 230 included in the lens assembly 210 in a specific direction or Operation characteristics of the image sensor 230 may be controlled (eg, read-out timing may be adjusted, etc.). This makes it possible to compensate for at least some of the negative effects of the movement on the image being taken. According to various embodiments, the image stabilizer 240 may include a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 201 , according to various embodiments. city) may be used to detect such a movement of the camera module 201 or the electronic device 201 . According to various embodiments, the image stabilizer 240 may be implemented as, for example, an optical image stabilizer. The memory 250 may temporarily store at least a portion of the image acquired through the image sensor 230 for the next image processing operation. For example, when image acquisition is delayed according to the shutter or a plurality of images are acquired at high speed, the acquired original image (eg, Bayer-patterned image or high-resolution image) is stored in the memory 250 and , a copy image corresponding thereto (eg, a low-resolution image) may be previewed through the display device 260 . Thereafter, when a specified condition is satisfied (eg, a user input or a system command), at least a portion of the original image stored in the memory 250 may be obtained and processed by, for example, the image signal processor 260 . According to various embodiments, the memory 250 may be configured as at least a part of the memory 230 or as a separate memory operated independently of the memory 230 .

The image signal processor 260 may perform one or more image processing on an image acquired through the image sensor 230 or an image stored in the memory 250 . The one or more image processes may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment, blurring ( blurring, sharpening, or softening. Additionally or alternatively, the image signal processor 260 may include at least one of the components included in the camera module 201 (eg, an image sensor). 230), for example, exposure time control, readout timing control, etc. The image processed by the image signal processor 260 is stored back in the memory 250 for further processing. or as an external component of the camera module 201 (eg, the memory 230, the display device 260, the electronic device 202, the electronic device 204, or the server 208). According to examples, the image signal processor 260 may be configured as at least a part of the processor 120 or as a separate processor operated independently of the processor 120. The image signal processor 260 may include the processor 120 ) and a separate processor, at least one image processed by the image signal processor 260 may be displayed through the display device 260 as it is by the processor 120 or after additional image processing. .

According to various embodiments, the electronic device 101 may include a plurality of camera modules 201 each having different properties or functions. In this case, for example, at least one of the plurality of camera modules 201 may be a wide-angle camera, and at least the other may be a telephoto camera. Similarly, at least one of the plurality of camera modules 201 may be a front camera, and at least the other may be a rear camera.

2B is a block diagram illustrating a structure of an image sensor according to various embodiments.

According to an embodiment, the image sensor 230 may be a component of a camera module (eg, the camera module 180 and the camera module 201 ) included in the electronic device 101 .

Referring to FIG. 2B , the image sensor 230 according to various embodiments includes a pixel array 270 , a row-driver 280 , and a column-readout circuit. ) 290 , a controller 292 , a memory 293 , or an interface 294 .

The pixel array 270 may include a plurality of pixels 271 to 279 . For example, the pixel array 270 may have a structure in which a plurality of pixels 271 to 279 are arranged in an M*N matrix (M and N are natural numbers). The pixel array 270 in which the plurality of pixels 271 to 279 are arranged in an M*N two-dimensional form may have M rows and N columns. The pixel array 270 may include a plurality of photo-sensing devices, for example, photo-sensing devices such as photo diodes or pinned photo diodes. The pixel array 270 may sense light using a plurality of photo-sensing elements, and convert the light into an analog electrical signal to generate an image signal.

The raw driver 280 may drive the pixel array 270 in units of rows. For example, the raw driver 280 controls a transfer control signal for controlling a transfer transistor of the plurality of pixels 271 to 279 included in the pixel array 270 , a reset control signal for controlling the reset transistor, or a selection transistor A selection control signal may be output to the pixel array 270 . The row-driver 280 may determine a row to be read out.

The column-read-out circuit 290 may receive an analog electrical signal generated by the pixel array 270 . For example, the column-read-out circuit 290 may receive an analog electrical signal from a column line selected from among a plurality of columns constituting the pixel array 270 . The column-readout circuit 290 converts an analog electrical signal received from the selected column line into pixel data (or a digital signal) and outputs an analog-to-digital converter (hereinafter, ADC) 291 . may include Meanwhile, the column-read-out circuit 290 receives an analog electrical signal from the pixel array 270 , converts the received analog electrical signal into pixel data using the ADC 291 and outputs the read operation It can be called read out. The column-read-out circuit 290 and the ADC 291 may determine a column to be read out.

The controller 292 may acquire an image frame based on pixel data received from the column-read-out circuit 290 . The controller 292 may output the image frame to the external circuit 295 through the interface 294 . According to an embodiment, the controller 292 generates a transfer control signal for controlling the transfer transistors of the plurality of pixels 271 to 279, a reset control signal for controlling the reset transistor, or a selection control signal for controlling the selection transistor. This may be done, and the generated signal may be provided to the raw driver 280 . Also, the controller 292 may generate a selection control signal for selecting at least one column line among a plurality of column lines constituting the pixel array 270 , and transmit the generated signal to the column-read-out circuit 290 . ) can be provided. For example, the column-read-out circuit 290 may enable some column lines among the plurality of column lines and disable the remaining column lines based on a selection control signal provided from the controller 292 . can do it Also, the controller 292 may be implemented as a processor (eg, the processor 120 ) including a central processing unit (CPU) or an application processor (AP), or as a block or module.

Memory 293 may include volatile and/or non-volatile memory. The memory 293 is a storage device provided inside the image sensor 230 and may include a buffer memory. According to an embodiment, the memory 293 may temporarily store a digital signal output from the column-read-out circuit 290 or the controller 292 . For example, the memory 293 may include at least one image frame obtained based on the light received by the pixel array 270 . Also, the memory 293 may store at least one digital signal received from the external circuit 295 through the interface 294 .

According to an embodiment, the image sensor 230 may directly transmit the read-out image frame to the external circuit 295 through the interface 294 without storing the read-out image frame in the memory 293 .

The interface 294 may include, for example, an interface 177 or a communication module 190 . The interface 294 may connect a component of the image sensor 230 , for example, the controller 292 or the memory 293 to the external circuit 295 in a wired or wireless manner. For example, the interface 294 transmits at least one image frame stored in the memory 293 of the image sensor 230 to the external circuit 295, for example, a memory (eg, memory) of the electronic device 101 (eg, 101). (130)). Also, the external circuit 295 , for example, a control signal of the processor (eg, 120 ) of the electronic device 101 (eg, 101 ) may be transmitted to the controller 292 of the image sensor 230 .

According to an embodiment, the image sensor 230 may communicate with the external circuit 295 through the interface 294 , for example, in a serial communication method. For example, the memory 293 of the image sensor 230 may communicate with the processor (eg, 120) of the electronic device 101 (eg, 101) in an I2C (Inter-Integrated Circuit) method.

According to another embodiment, the image sensor 230 may be connected to the external circuit 295 through the interface 294 , for example, through an interface defined according to a MIPI (Mobile Industry Processor Interface) protocol. For example, the memory 293 of the image sensor 230 may communicate with the processor (eg, 120 ) of the electronic device 101 (eg, 101) according to an interface defined according to the MIPI protocol.

Some or all of the above-described components 270 to 295 may be included in the image sensor 230 as necessary, and each component may be configured in a singular or plural number.

3A and 3B are diagrams for explaining a slow shutter function.

Referring to FIG. 3A , in order to implement the slow shutter function, the CSC may output the first image 311 by adjusting the aperture and performing read-out once after exposure for a long time.

Referring to FIG. 3B , in order to implement the slow shutter function, a camera without a variable aperture performs read-out a plurality of times by applying the same exposure time to output first to n-th images 321 and 323 . can do. The electronic device including the camera may acquire an image obtained by synthesizing the first to nth images 321 and 323 .

4 is a block diagram 400 of an electronic device according to various embodiments.

The electronic device 401 may include an image sensor 410 , a processor 420 , and a memory 430 .

According to various embodiments, it is not limited to the devices shown in FIG. 4 , and the electronic device 401 may be implemented to include more or fewer components. For example, the electronic device 401 may be implemented to further include the components described above with reference to FIGS. 1, 2A, or 2B. Terms such as '~block' used below mean a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

According to an embodiment, the image sensor 410 may be a component of a camera module (eg, the camera module 180 and the camera module 201 ) included in the electronic device 401 .

The image sensor 410 may output an electrical signal corresponding to light received from the outside. For example, the image sensor 410 may include a plurality of pixels each having a photodiode. The photodiode may receive light and may generate an analog electrical signal corresponding to the received light. Analog electrical signals generated from each of the plurality of photodiodes constituting the plurality of pixels may be converted into a plurality of pixel data. In this case, the converted plurality of pixel data may mean pixel values corresponding to each of the plurality of pixels. A set of a plurality of pixel data acquired at a specific time may constitute at least one image frame.

The image sensor 410 may output images according to a frame rate and/or image size corresponding to an operation mode. The image sensor 410 may output preview images having a second size (eg, a pool size) at preset time intervals in the preview mode. The image sensor 410 outputs first images having a first size (eg, sub-size) in the slow shutter mode at the first time interval, and outputs at least one second image having a second size for the first time It is possible to output at a second time interval that is wider than the interval. According to an embodiment, the image sensor 410 may configure a frame rate, a time interval of the first images and/or the at least one second image according to the control information received from the processor 420 or the exposure determination block 427 . , the resolution or the number can be determined/adjusted. The first images having the first size may be referred to as binning images. According to an embodiment, the image sensor 410 outputs preview images through a first path, and outputs first images and/or at least one second image through a second path different from the first path. can

The processor 420 outputs images through a display (eg, the display module 160) according to the operation mode, stores the images in the memory 430, or outputs a synthesized image through the display, The composite image may be stored in the memory 430 . The processor 420 may detect an event requiring a change of the operation mode. Based on the detection of the event, the processor 420 may control at least one component in the electronic device 401 to operate according to the changed operation mode. According to an embodiment, the event is the detection of a scene including a moving body (eg, fluid/fluid, body, liquid or gas) through the image signal processor 423 , the scene detection block 425 , or the like, or the display module 160 . ), reception of a user input through the input module 150 , and the like.

The processor 420 may include an interface 421 , an image signal processor (ISP) 423 , a scene detection block 425 , an exposure determination block 427 , and a slow shutter 429 .

The interface 421 may include, for example, an interface 177 or a communication module 190 . The interface 421 may connect a component of the processor 420 , for example, the image signal processor 423 or the exposure determination block 427 to the image sensor 410 in a wired or wireless manner.

According to an embodiment, the processor 420 may communicate with the image sensor 410 through the interface 421 , for example, using a serial communication method (eg, an I2C (Inter-Integrated Circuit) method).

The image signal processor 423 may receive images from the image sensor 410 through the interface 421 . In the preview mode, the image signal processor 423 may receive preview images having a second size (eg, a full size) from the image sensor 410 at a preset time interval. In the slow shutter mode, the image signal processor 423 receives first images having a first size (eg, sub-size) from the image sensor 410 at a first time interval, and at least one second image having a second size 2 images may be received from the image sensor 410 at a second time interval greater than the first time interval. The image signal processor 423 may store first images having a first size and at least one second image having a second size in the memory 430 . According to an embodiment, the image signal processor 423 may discard images in which the movement of the moving object does not exist among the first images and store only images in which the movement of the moving object exists in the memory 430 .

The image signal processor 423 may perform one or more image processing on the image acquired through the image sensor 410 . The one or more image processing may include, for example, depth map generation, three-dimensional modeling, panorama generation, feature point extraction, image synthesis, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment, blurring ( blurring, sharpening, or softening. The image signal processor 423 may store image-processed images in the memory 430. According to an embodiment, the image signal processor ( 423), based on the first images and/or the at least one second image, may obtain composite area information (or may be referred to as a composite area map) including at least one of the moving object location information and the background location information. The image signal processor 423 may store the composite area information in the memory 430. According to an embodiment, the image signal processor 423 may output the composite area information to the scene detection block 425. .

The scene detection block 425 may receive images from the image signal processor 423 . The scene detection block 425 may identify a moving object region including the moving object in the images, and obtain moving object information such as a moving speed and a moving direction. According to an embodiment, the scene detection block 425 may acquire the moving object information based on the synthesis area information received from the image signal processor 423 . According to another embodiment, the scene detection block 425 may be configured to obtain, based on the first images and/or the at least one second image, composite area information including at least one of the moving object location information and the background location information. can According to an embodiment, the moving object information may include vector values corresponding to a moving speed, a moving direction, and the like. According to an embodiment, the scene detection block 425 may downsize images received from the image signal processor 423 or receive downsized images from the image signal processor 423 . The scene detection block 425 may detect a difference between the images by comparing the downsized images with each other. The scene detection block 425 may determine a moving speed, a moving direction, and the like, based on a difference between the images.

The exposure determination block 427 may receive moving object information from the scene detection block 425 and determine control information related to first images having a first size based on the moving object information. The exposure determination block 427 may output control information related to the first images to the image sensor 410 . According to an embodiment, the exposure determination block 427 may output control information related to at least one second image to the image sensor 410 together with or separately from control information related to the first images. According to an embodiment, the control information related to the first images may include information on at least one of a frame rate, a time interval, a resolution, and the number of the first images. According to an embodiment, the control information related to the at least one second image may include information on at least one of a frame rate, a time interval, a resolution, and the number of the at least one second image.

The slow shutter 429, in the slow shutter mode, obtains first images having a first size stored in the memory 430, obtains a first composite image based on the first images, and obtains a first composite image based on the composite area information. to obtain a first movable body region from the first composite image, identify a second movable body region in at least one second image based on the composite region information, and replace the second movable body region with the first movable body region; A second composite image may be acquired.

According to various embodiments, the electronic device (eg, the electronic device 101 or the electronic device 401 ) includes a memory (eg, the memory 130 , the memory 430 ), and an image sensor (eg, the image sensor 230 ). ), an image sensor 410), and at least one processor (eg, processor 120, processor 420) operatively coupled to the memory and the image sensor, the memory comprising: causing the at least one processor to acquire a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through the image sensor, acquiring a first composite image based on the first images, and obtaining a first moving body region from the first composite image based on composite region information including at least one of mobile location information and background location information; A second composite image may be obtained by identifying a second moving object region in the at least one second image and replacing the second moving object region with the first moving object region based on the composite region information.

According to various embodiments, a frame rate of the first images may be higher than a frame rate of the at least one second image.

According to various embodiments, the second composite image may include a first moving body region of the first composite image and a background region of the at least one second image.

According to various embodiments, when the instructions are executed, the at least one processor may upscale the first composite image and obtain the first moving body region from the upscaled first composite image. .

According to various embodiments, when the instructions are executed, the at least one processor may identify the moving object from at least one of the plurality of images.

According to various embodiments, when the instructions are executed, the at least one processor may identify at least one of the size, speed, or direction of the movement of the moving object.

According to various embodiments, the instructions, when executed, cause the at least one processor to generate the first image to be used for generating the first composite image based on at least one of a movement size, speed, or direction of the moving object. at least one of a frame rate, a time interval, a resolution, or a number of frames.

According to various embodiments, when the instructions are executed, the at least one processor may provide information about at least one of a frame rate, a time interval, a resolution, and a number of the first images to the image sensor. .

According to various embodiments, the image sensor may be configured to provide the first images under the control of the at least one processor.

According to various embodiments, the image sensor may be configured to provide at least a portion of the first images during a period in which a preview image is provided.

According to various embodiments, the at least one second image may have the same size as the preview image.

According to various embodiments, the image sensor may be configured to output an image frame including an image and a header or a footer including information about the image.

According to various embodiments, the at least one processor includes an image signal processor, and the image signal processor changes settings related to image processing according to whether the image to be processed is the first image or the second image. It can be configured to be

According to various embodiments, the image sensor is configured to output an image frame including an image and a header or a footer including information about the image, wherein the at least one processor includes an image signal processor, the image The signal processor is configured to identify whether the image is the first image or the second image based on information about the image included in the header or footer, and the image is the first image or the second image. Depending on whether it is an image or not, it may be configured to change settings related to processing of the image.

According to various embodiments, the instructions, when executed, require the at least one processor to, while displaying a preview image, change to a slow shutter mode requiring acquisition of the second composite image. and identify whether an event is detected and, based on the detection of the slow shutter event, display on the display a graphic object for selecting or changing at least one of a frame rate, resolution, or number of the first images. .

According to various embodiments, the instructions, when executed, require the at least one processor to, while displaying a preview image, change to a slow shutter mode requiring acquisition of the second composite image. and identify whether an event is detected, and display, on the display, a graphic object for selecting or adjusting a value related to blur of the second composite image based on the detection of the slow shutter event.

According to various embodiments, when the instructions are executed, the at least one processor identifies whether slow shutter shooting is selected, and based on the selection of the slow shutter shooting, the first images and the at least one may be configured to acquire a plurality of images including the second image.

According to various embodiments, the instructions, when executed, may be used to generate the first composite image according to the at least one processor switching to a slow shutter mode requiring acquisition of the second composite image. A graphic object for selecting or changing at least one of a frame rate, resolution, or number of the first images may be displayed on a display.

5 is a flowchart 500 illustrating a method for providing a slow shutter in accordance with various embodiments. According to various embodiments, the operations illustrated in FIG. 5 are not limited to the illustrated order and may be performed in various orders. According to various embodiments, more operations than those illustrated in FIG. 5 or at least one fewer operations may be performed.

Referring to FIG. 5 , an electronic device (eg, the electronic device 101 , the electronic device 401 ) or at least one processor (eg, the processor 120 , the processor 420 ) performs at least one of operations 510 to 550 . action can be performed.

In operation 510 , the electronic device uses an image sensor (eg, the image sensor 230 and the image sensor 410 ) to display first images having a first size and at least one image having a second size larger than the first size. A plurality of images including the second image may be acquired.

According to an embodiment, the electronic device identifies a moving object from at least one of the plurality of images, identifies at least one of a movement size, speed, or direction of the moving object, and at least one of a movement size, speed, or direction of the moving object determine at least one of a frame rate, time interval, resolution or number of first images to be used for obtaining the first composite image based on information may be provided to the image sensor. The image sensor may be configured to provide first images under control of the at least one processor. The image sensor may be configured to provide at least a portion of the first images during the period of providing the preview images. The at least one second image may have the same size as the preview image. The at least one second image may be a part of preview images.

According to an embodiment, a frame rate of the first images may be higher than a frame rate of the at least one second image.

According to an embodiment, according to a transition to the slow shutter mode requiring acquisition of the second composite image, the electronic device may configure a frame rate, time interval, resolution, or A graphic object for selecting or changing at least one of the number may be displayed on the display.

In operation 520, the electronic device may acquire a first composite image based on the first images having a first size.

In operation 530, the electronic device may obtain the first moving object area from the first combined image based on the combined area information including at least one of the moving object location information and the background location information.

According to an embodiment, the electronic device may upscale the first composite image and acquire the first moving body region from the upscaled first composite image.

In operation 540, the electronic device may identify a second moving object region from the at least one second image based on the composite region information.

In operation 550, the electronic device may acquire the second composite image by replacing the second moving object area with the first moving object area.

According to an embodiment, the second composite image may include a first moving body region of the first composite image and a background region of at least one second image.

6 is a diagram 600 for explaining an operation of acquiring images in a slow shutter mode according to various embodiments of the present disclosure. The operation illustrated in FIG. 6 may be performed by an electronic device (eg, the electronic device 101 or the electronic device 401) or at least one processor (eg, the processor 120 or the processor 420).

The electronic device may output the preview images 611 and 613 having the second size (eg, the full size) at preset time intervals in the preview mode. In the preview mode, for example, during the first preview section 621 , the electronic device may detect an event requiring a change of the slow shutter mode (ie, a slow shutter event). According to an embodiment, the event is the detection of a scene including a moving object through the image signal processor 423, the scene detection block 425, etc., or the user input through the display module 160, the input module 150, etc. reception, and the like.

In response to detecting the slow shutter event, in the preview mode, for example, during the second preview period 623, the electronic device displays images 631 having a first size (eg, sub-size) smaller than the second size. In step 633 , a moving object area including the moving objects 635a and 635b may be identified, and moving object information such as a moving speed and a moving direction may be acquired. The electronic device may determine control information related to the first images 641 and 643 having the first size based on the moving object information. According to an embodiment, the electronic device may determine control information related to the at least one second image 651 and 653 based on the moving object information. According to an embodiment, the control information may include a frame rate, time interval, resolution, or number of the first images 641 and 643 and/or the at least one second image 651 and 653 .

In the slow shutter mode, for example, during the slow shutter period 625 , the electronic device outputs first images 641 and 643 having a first size at the first time interval, and at least one having a second size The second images 651 and 653 of may be output at a second time interval greater than the first time interval.

The electronic device may perform translational (2D) motion correction on at least one of the second images 651 and 653 in order to compensate for motion caused by hand shake, When the time interval is equal to or greater than the threshold value, rotation, pitch, and yaw axis correction may be performed.

According to an embodiment, the first images 641 and 643 may be output from an image sensor (eg, the image sensor 230 or the image sensor 410 ) for up to 24 ms. Since the first images 641 and 643 of up to 35 frames output for 24 ms may be motion-matched by the image stabilizer in the camera module, the electronic device performs separate motion matching for the first images 641 and 643. may not perform.

7 is a diagram 700 for illustrating a first image and a second image according to various embodiments.

Referring to FIG. 7 , the image sensor (eg, image sensor 230 , image sensor 410 ) may include a processor (eg, processor 120 , processor 420 ) or an exposure determining block (eg, exposure determining block ( ) 427)), by performing image exposure and readout 731 and 733 based on the control information received from The second images 721 and 723 having (eg, a full size) may be output. As the size (and/or resolution) of the first images 711 and 713 decreases, the image sensor may output a greater number of the first images 711 and 713 for a unit time. The processor or scene detection block 425 may determine the size (and/or resolution) of the first images 711 and 713 based on the amount/degree/level of blur required for the composite image. The blur may continuously represent the movement (or the trajectory of the movement) of the moving object.

8 is a diagram illustrating a configuration of an image frame 800 according to various embodiments.

The image frame 800 may include a header 810 , a payload 820 , and a footer 830 .

The image sensor (eg, the image sensor 230 and the image sensor 410 ) may output images having various sizes/resolutions. In order for the processor (eg, the processor 120 , the processor 420 ) to adaptively process an image having a variable size/resolution, the image sensor receives information about the image included in the payload 820 . It may be included in the header 810 and/or the footer 830 .

According to an embodiment, the header 810 may include information 812 on an image in the current image frame, and the footer 830 may include information 832 on an image in the image frame.

9 is a diagram 900 for explaining an operation of an image signal processor according to various embodiments of the present disclosure.

The image sensor 410 may output images having various sizes/resolutions at various frame rates.

When the size/resolution/frame rate of continuously input images is variable, the image signal processor 423 may adaptively process the image. According to an embodiment, when a first image having a first size (eg, sub-size) is input, the image signal processor 423 may be configured to: Auto Focus (AF)/Auto White Balance (AWB)/Auto (AE) Exposure), etc., may not perform exposure and color-related operations, and may perform setting/parameter changes for buffer allocation required when the first image is stored in the memory 430 . The image signal processor 423, when a second image having a second size (eg, full size) is input, exposure and color such as AF (Auto Focus)/AWB (Auto White Balance)/AE (Auto Exposure) A related operation may be performed, and a setting/parameter change may be performed for a buffer allocation required when the second image is stored in the memory 430 . The image sensor 410 outputs an image frame in real time, and the image signal processor 423 refers to the header and/or footer of the current or previous image frame, and according to the setting/parameter set/changed in real time, the current image frame / Images can be processed in real time.

10A to 10C are diagrams for explaining an operation of obtaining composite area information according to various embodiments.

Referring to FIG. 10A , a processor (eg, processor 120, processor 420) (or image signal processor (eg, image signal processor 260, image signal processor 423), or scene detection block 425) ) may receive the image 1010 from an image sensor (eg, the image sensor 230 or the image sensor 410 ).

The processor may identify in the image 1010 a moving object region 1020 including a moving object as shown in FIG. 10B and/or a background region 1030 including a fixed background as shown in FIG. 10C .

The processor may obtain composite area information or composite area maps 1041 and 1043 including location information of the moving object area 1020 and/or location information of the background area 1030 . The processor may store the synthesis region information or the synthesis region maps 1041 and 1043 in a memory (eg, the memory 130 and the memory 430 ). According to an embodiment, the location information of the moving object area 1020 may include location information or indexes of image pixels belonging to the moving object area 1020 , and the location information of the background area 1030 is in the background area 1030 . It may include location information or indexes of image pixels belonging to it.

11 is a flowchart 1100 illustrating a method for providing a slow shutter in accordance with various embodiments. According to various embodiments, the operations illustrated in FIG. 11 are not limited to the illustrated order and may be performed in various orders. According to various embodiments, more operations than the operations illustrated in FIG. 11 may be performed, or at least one fewer operations may be performed.

Referring to FIG. 11 , an electronic device (eg, the electronic device 101 , the electronic device 401 ) or at least one processor (eg, the processor 120 , the processor 420 ) performs at least one of operations 1110 to 1180 . action can be performed.

In operation 1110 , the electronic device may identify whether a slow shutter event requiring a change to the slow shutter mode is detected. Based on the detection of the slow shutter event, the electronic device may control at least one component in the electronic device 401 to operate according to the slow shutter mode. According to an embodiment, the slow shutter event may be detected by a scene including a moving object through the image signal processor 423, the scene detection block 425, or the like, or by a user through the display module 160, the input module 150, or the like. receiving input, and the like. If the slow shutter event is detected, the electronic device may perform step 1120 , and if the slow shutter event is not detected, the electronic device may perform step 1110 . According to an embodiment, while displaying the preview image, the electronic device may identify whether a slow shutter event requiring a change to the slow shutter mode is detected.

In operation 1120, based on the detection of the slow shutter event, the electronic device displays on the display a graphic object for selecting or changing at least one of a frame rate, a resolution, or the number of first images to be used for generating the first composite image. can be displayed in According to an embodiment, the graphic object may indicate the amount/degree/level of blur required for the final composite image. The user can select/adjust the desired amount/degree/level of blur by using a graphic object.

In operation 1130 , the electronic device may identify whether slow shutter photography is selected. According to an embodiment, the selection of slow shutter shooting includes a touch/hover input through a display (eg, the display module 160), a button input through an input module (eg, the input module 150), and a microphone integrated voice It may be implemented by input or the like. When the selection of slow shutter photography is identified, the electronic device may perform step 1140 , and when the selection of slow shutter photography is not identified, the electronic device may perform step 1130 .

Operations 1140 to 1180 may be performed in the same manner as operations 510 to 550 described above, and thus a redundant description will be omitted.

The electronic device (eg, the electronic device 101 , the electronic device 401 ) includes a memory (eg, the memory 130 , the memory 430 ), an image sensor (eg, the image sensor 230 , and the image sensor 410 ). ), and at least one processor (eg, a processor 120 , a processor 420 ) operatively coupled to the memory and the image sensor, wherein the memory, when executed, causes the at least one processor to: ,

According to various embodiments, a method for providing a slow shutter by an electronic device (eg, the electronic device 101 or the electronic device 401 ) includes an image sensor (eg, the image sensor 230 , the image sensor 410 ). )) through an operation of acquiring a plurality of images including first images having a first size and at least one second image having a second size larger than the first size, based on the first images Acquiring a first composite image, on the basis of composite region information including at least one of mobile location information or background location information, acquiring a first mobile region from the first composite image, based on the composite region information Thus, the method may include identifying a second moving object region in the at least one second image, and obtaining a second composite image by replacing the second moving object region with the first moving object region.

According to various embodiments, a frame rate of the first images may be higher than a frame rate of the at least one second image.

According to various embodiments, the second composite image may include a first moving body region of the first composite image and a background region of the at least one second image.

According to various embodiments, the method may further include upscaling the first composite image and acquiring the first moving body region from the upscaled first composite image.

According to various embodiments, the method may further include identifying the moving object in at least one of the plurality of images.

According to various embodiments, the method may further include the operation of identifying at least one of a movement size, speed, or direction of the moving object.

According to various embodiments, the method may include a frame rate, a time interval, a resolution or The method may further include determining at least one of the number.

According to various embodiments, the method may further include providing information about at least one of a frame rate, a time interval, a resolution, and a number of the first images to the image sensor.

According to various embodiments, the image sensor may be configured to provide the first images under the control of the at least one processor.

According to various embodiments, the image sensor may be configured to provide at least a portion of the first images during a period in which a preview image is provided.

According to various embodiments, the at least one second image may have the same size as the preview image.

According to various embodiments, the image sensor may be configured to output an image frame including an image and a header or a footer including information about the image.

According to various embodiments, the electronic device includes an image signal processor, and the image signal processor is configured to change a setting related to image processing according to whether the image to be processed is the first image or the second image. can be

According to various embodiments, the image sensor is configured to output an image frame including an image and a header or footer including information about the image, and the image signal processor is configured to output the image included in the header or footer. based on the information on whether the image is the first image or the second image, and according to whether the image is the first image or the second image, related It can be configured to change settings.

According to various embodiments, the method includes, while displaying a preview image, identifying whether a slow shutter event requiring a change to a slow shutter mode requiring acquisition of the second composite image is detected; and displaying, on a display, a graphic object for selecting or changing at least one of a frame rate, resolution, or number of the first images based on the detection of the slow shutter event.

According to various embodiments, the method includes, while displaying a preview image, identifying whether a slow shutter event requiring a change to a slow shutter mode requiring acquisition of the second composite image is detected; and displaying a graphic object for selecting or adjusting a blur-related value of the second composite image on a display based on the detection of the slow shutter event.

According to various embodiments, the method includes an operation of identifying whether slow shutter shooting is selected, and based on the selection of the slow shutter shooting, a plurality of the first images and the at least one second image The method may further include acquiring images of

According to various embodiments, the method includes: a frame rate, resolution of the first images to be used for generating the first composite image upon switching to a slow shutter mode requiring acquisition of the second composite image Alternatively, the method may further include displaying a graphic object for selecting or changing at least one of the number on the display.

12A to 12B are diagrams 1200a to 1200b for explaining a slow shutter mode according to various embodiments of the present disclosure.

Referring to FIG. 12A , the electronic device 1201 (eg, the electronic device 101 or the electronic device 401 ) displays a camera interface 1210 for capturing an image on a display 1206 (eg, the display module 160 ). ) can be displayed on the The camera interface 1210 may include a capture button 1213 for recording a still/moving image and a preview image 1215 captured by the rear camera of the electronic device 1201 . The electronic device 1201 may detect a slow shutter event requiring a change to the slow shutter mode. According to an embodiment, the slow shutter event is detected by a scene including a moving object through the image signal processor 423, the scene detection block 425, or the like, or a user through the display module 160, the input module 150, or the like. receiving input, and the like.

Referring to FIG. 12B , the electronic device 1201 provides a graphic object for changing at least one of a frame rate, a resolution, or the number of first images to be used for generating a first composite image, based on detection of a slow shutter event 1220 may be displayed on display 1206 . According to an embodiment, the graphic object 1220 may indicate the amount/degree/level of blur (or the amount/time of exposure) required for the final composite image. A user may select/adjust a desired blur amount/degree/level (or exposure amount/time) by using the graphic object 1220 . For example, the user may select a blur amount/degree/level ranging from a maximum blur amount/degree/level to a minimum blur amount/degree/level through a swipe gesture on the graphic object 1220 . According to an embodiment, the electronic device 1201 performs a preview image 1215 of a temporarily synthesized image or a pre-stored image corresponding to the amount/degree/level of blur (or exposure amount/time) selected through the graphic object 1220 . can be displayed instead. For example, the temporary composite image may be discarded if there is no subsequent preset operation (eg, if the capture button 1213 is not selected (for a preset time)).

In response to selection of the photographing button 1213 , the electronic device 1201 may permanently store the temporary composite image or the newly created final composite image in a memory (eg, the memory 130 or the memory 430 ).

According to another embodiment, the electronic device 1201 may automatically select a blur amount/degree/level (or exposure amount/time) required for the final composite image based on the detection of the slow shutter event. The electronic device 1201 may select a blur amount/degree/level (or exposure amount/time) based on the movement size and/or speed of the moving object. The electronic device 1201 increases the blur amount/degree/level (or exposure amount/time) when the size and/or speed of the moving object is large (eg, greater than a threshold value), and increases the amount/time of the movement of the moving object. and/or reduce the blur amount/degree/level (or exposure amount/time) when the speed is small (eg, less than a threshold value).

13 is a view 1300 for explaining an operation of a slow shutter according to various embodiments of the present disclosure.

The slow shutter 429 may include an average block 1311 , an upscaler 1313 , and a fusion block 1315 .

The average block 1311 performs averaging of the first images 1321 having a first size (eg, sub-size) stored in the memory 430 in the time axis direction to obtain the first composite image 1323 . can be obtained For example, a pixel value at each position of the first composite image 1323 may be determined as an average value of pixel values at the position of the first images 1321 .

According to an embodiment, the average block 1311 may acquire the first composite image 1323 by performing average synthesis on the first images 1321 in which the movement size of the moving object is equal to or greater than a preset threshold value.

According to an embodiment, a temporal directional low pass filter (LPF) may be applied to the first images 1321 input to the average block 1311 .

The upscaler 1313 may upscale the first composite image 1323 to a second size (eg, full size).

The fusion block 1315 may acquire the first moving body region from the upscaled first composite image based on the synthesized region map 1330 . The fusion block 1315 identifies a second moving object region in the at least one second image 1325 having a second size stored in the memory 430 based on the synthesized region map 1330 , and the first moving object region By replacing the second moving body region with , a second composite image can be obtained.

According to an embodiment, the slow shutter 429 includes 240 first images 1321 and 10 second images captured for 330 ms by an image sensor (eg, image sensor 230 , image sensor 410 ). Image synthesis may be performed on the image 1325 .

14A to 14C are diagrams for explaining an image synthesizing operation according to various embodiments of the present disclosure;

Referring to FIG. 14A , the electronic device (eg, the electronic device 101 or the electronic device 401 ) (or the slow shutter 429 ) has a first size (eg, sub-size) based on the composite area information/map. ) from the first composite image 1411 upscaled to a second size (eg, pool size), a first moving object region 1415 distinct from the background region 1413 may be obtained.

Referring to FIG. 14B , the electronic device may identify the second moving object area 1425 that is distinguished from the background area 1423 from the second image 1421 having the second size.

Referring to FIG. 14C , the electronic device may acquire a second composite image 1430 by replacing the second movable body area 1425 with the first movable body area 1415 .

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the element from other elements in question, and may refer to elements in other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, a processor (eg, processor 120 ) of a device (eg, electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices ( It can be distributed online (eg download or upload), directly between smartphones (eg smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

401: electronic device, 410: image sensor, 420: processor, 430: memory, 421: interface, 423: image signal processor, 425: scene detection block, 427: exposure determining block, 429: slow shutter

Claims (20)

In an electronic device,
Memory;
image sensor; and
at least one processor operatively coupled to the memory and the image sensor;
The memory, when executed, causes the at least one processor to:
acquiring a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through the image sensor;
obtaining a first composite image based on the first images;
obtaining a first moving body region from the first composite image based on the composite region information including at least one of the moving object position information and the background position information;
identifying a second moving object region in the at least one second image based on the composite region information;
and storing instructions for obtaining a second composite image by replacing the second moving object area with the first moving object area.
According to claim 1,
A frame rate of the first images is higher than a frame rate of the at least one second image.
According to claim 1,
The second composite image includes a first moving body region of the first composite image and a background region of the at least one second image.
4. The method of claim 3,
When the instructions are executed, the at least one processor,
upscaling the first composite image;
and acquiring the first movable body region from the upscaled first composite image.
According to claim 1,
When the instructions are executed, the at least one processor,
identifying a moving object in at least one of the plurality of images;
and determine at least one of a frame rate, a time interval, a resolution, or a number of the first images to be used for acquiring the first composite image, based on at least one of a size, speed, or direction of movement of the moving object.
The method of claim 5 , wherein the instructions, when executed, cause the at least one processor to:
An electronic device configured to provide information on at least one of a frame rate, a time interval, a resolution, and the number of the first images to the image sensor.
According to claim 1,
The image sensor provides at least a portion of the first images during a period of providing a preview image,
The at least one second image has the same size as the preview image.

The method of claim 1 , wherein the instructions, when executed, cause the at least one processor to:
while displaying a preview image, identify whether a slow shutter event requiring a change to a slow shutter mode requiring acquisition of the second composite image is detected;
and display on the display a graphical object for selecting or adjusting a value related to blur of the second composite image based on the detection of the slow shutter event.
The method of claim 8 , wherein the instructions, when executed, cause the at least one processor to:
Identifies whether slow shutter shooting is selected,
The electronic device is configured to acquire a plurality of images including the first images and the at least one second image based on the selection of the slow shutter photographing.
According to claim 1,
The image sensor is configured to output an image frame including an image and a header or footer including information about the image,
The at least one processor includes an image signal processor,
The image signal processor identifies whether the image is the first image or the second image based on information about the image included in the header or footer,
an electronic device configured to change a setting related to processing of the image according to whether the image is the first image or the second image.

A method for providing a slow shutter, comprising:
acquiring a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through an image sensor;
obtaining a first composite image based on the first images;
obtaining a first moving object area from the first combined image based on combined area information including at least one of moving object location information and background location information;
identifying a second moving object region in the at least one second image based on the composite region information; and
and obtaining a second composite image by replacing the second moving body area with the first moving body area.
12. The method of claim 11,
upscaling the first composite image; and
and obtaining the first movable body region from the upscaled first composite image.
12. The method of claim 11,
identifying a moving object in at least one of the plurality of images; and
The method further includes determining at least one of a frame rate, a time interval, a resolution, or a number of the first images to be used for obtaining the first composite image, based on at least one of a size, speed, or direction of the movement of the moving object. How to.
12. The method of claim 11,
while displaying a preview image, identifying whether a slow shutter event requiring a change to a slow shutter mode requiring acquisition of the second composite image is detected; and
based on the detection of the slow shutter event, displaying on a display a graphic object for selecting or adjusting a value related to blur of the second composite image.
12. The method of claim 11,
receiving an image frame including an image and a header or a footer including information about the image;
identifying whether the image is the first image or the second image based on information about the image included in the header or footer; and
and changing a setting of an image signal processor related to processing of the image according to whether the image is the first image or the second image.
A non-transitory storage medium storing instructions, wherein the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor, the at least one operation comprising:
acquiring a plurality of images including first images having a first size and at least one second image having a second size greater than the first size through an image sensor;
obtaining a first composite image based on the first images;
obtaining a first moving object area from the first combined image based on combined area information including at least one of moving object location information and background location information;
identifying a second moving object region in the at least one second image based on the composite region information; and
and acquiring a second composite image by replacing the second moving object area with the first moving object area.
17. The method of claim 16,
upscaling the first composite image; and
and obtaining the first movable body region from the upscaled first composite image.
17. The method of claim 16,
identifying a moving object in at least one of the plurality of images; and
The method further includes determining at least one of a frame rate, a time interval, a resolution, or a number of the first images to be used for obtaining the first composite image, based on at least one of a size, speed, or direction of the movement of the moving object. storage medium.
17. The method of claim 16,
while displaying a preview image, identifying whether a slow shutter event requiring a change to a slow shutter mode requiring acquisition of the second composite image is detected; and
and displaying on a display a graphic object for selecting or adjusting a value related to blur of the second composite image based on the detection of the slow shutter event.
17. The method of claim 16,
receiving an image frame including an image and a header or a footer including information about the image;
identifying whether the image is the first image or the second image based on information about the image included in the header or footer; and
and changing a setting of an image signal processor related to processing of the image according to whether the image is the first image or the second image.

Images