WO2021187876A1

WO2021187876A1 - Self-image capturing method and electronic device therefor

Info

Publication number: WO2021187876A1
Application number: PCT/KR2021/003256
Authority: WO
Inventors: 김규원; 김유식; 장철상; 조형민; 최재웅
Original assignee: 삼성전자 주식회사
Priority date: 2020-03-16
Filing date: 2021-03-16
Publication date: 2021-09-23
Also published as: KR20210115786A

Abstract

Disclosed is an electronic device comprising a camera, a display, a processor operatively connected to the camera and the display, and a memory connected to the processor. The memory can store one or more instructions that allow, when executed, the processor to acquire a first gesture of a user by using the camera, set a frame on the basis of the first gesture, acquire a second gesture of the user by using the camera, set a padding area for an object in the frame on the basis of the second gesture, and record at least a part of the image acquired by the camera according to the frame and the padding area while tracking the object.

Description

Magnetic image capture method and electronic device therefor

Various embodiments disclosed in this document relate to a method for self-image capturing and an electronic device therefor.

A personal device such as a smart phone may be widely used. With the development of technology, functions supported by smart phones are also gradually improved. For example, recent smart phones may support high-resolution video recording. A user can shoot a high-resolution video using a smart phone of the picture. According to the trend of user-created content platforms, interest in content creators is increasing. For example, a content creator may create content using an electronic device such as a smart phone.

As the number of single-person media increases, the number of single-person content creators who directly produce content is increasing. For single content creators, it may not be possible to hire a separate camera director for image capturing (eg video recording). In this case, a content creator of single-person content may directly set a camera and then perform image capturing. However, when the subject moves, the set composition may not be maintained. For example, a content creator may wish to place the content creator in the center of an image. Since there is no camera director capable of separately maintaining the composition, the composition set according to the movement of the content creator may not be maintained.

In this regard, it may be considered to change the composition through post-processing once recording is finished. However, it is difficult to avoid image degradation through post-processing of an image or moving picture already compressed in a certain format. In general, the image acquisition apparatus may perform image quality improvement on an image region provided through a preview. Due to the characteristics of digital image improvement, the effect of image quality improvement may deteriorate as the number of colors in the image increases. Accordingly, when the image acquisition apparatus performs image quality improvement on the entire image, image quality improvement is performed on an unnecessary region to be excluded through post-processing, and the effect of image quality improvement may also be deteriorated. In addition, the image acquisition device may support optical image enhancement. For example, the image acquisition device may support image enhancement based on continuous image capturing and/or image enhancement based on dual cameras. If the composition is changed based on post-processing, such optical image enhancement may not be properly performed.

An electronic device according to an embodiment disclosed in this document includes a camera, a display, a processor operatively connected to the camera and the display, and a memory connected to the processor, wherein the memory includes: Obtaining a first gesture of the user by using the camera, setting a frame based on the first gesture, obtaining a second gesture of the user by using the camera, and obtaining the frame based on the second gesture One or more instructions may be stored to set a padding area for an object within, and to record at least a partial image of an image acquired by the camera according to the frame and the padding area while tracking the object.

A method for acquiring an image of an electronic device according to an embodiment disclosed in this document includes: acquiring a user's first gesture using a camera of the electronic device; and setting a frame based on the first gesture , obtaining a second gesture of the user using the camera, setting a padding area for an object in the frame based on the second gesture, and tracking the frame and the padding while tracking the object The method may include recording at least a partial image of the image acquired by the camera according to the area.

According to various embodiments disclosed in this document, the user of the electronic device may actively determine a photographing composition using a gesture without access to photographing equipment or without additional equipment.

The electronic device may maintain the photographing composition by constantly maintaining the size, ratio, and direction of the subject with respect to the background.

The electronic device may minimize the user's post-processing work by performing image acquisition based on the padding area according to the user setting.

In addition, various effects directly or indirectly identified through this document may be provided.

1 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;

2 illustrates a photographing environment according to an exemplary embodiment.

3 illustrates a photographing environment according to an exemplary embodiment.

4 is a block diagram of an electronic device according to an exemplary embodiment.

5 is a diagram illustrating the setting of a frame according to an embodiment.

6 illustrates a frame change according to a lateral movement of a user according to an exemplary embodiment.

7 illustrates a change in a frame according to a forward movement of a user according to an exemplary embodiment.

8 illustrates a padding area according to an embodiment.

9 illustrates a padding area according to an embodiment.

10 illustrates a method of setting a padding area according to an embodiment.

11 illustrates a method of setting a padding area according to an embodiment.

12 is a flowchart of an image recording method according to an embodiment.

In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of the present invention are included.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments of the present disclosure.

Referring to FIG. 1 , in a network environment 100 , the electronic device 101 communicates with the electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input device 150 , a sound output device 155 , a display device 160 , an audio module 170 , and a sensor module ( 176 , interface 177 , haptic module 179 , camera module 180 , power management module 188 , battery 189 , communication module 190 , subscriber identification module 196 , or antenna module 197 . ) may be included. In some embodiments, at least one of these components (eg, the display device 160 or the camera module 180 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components may be implemented as one integrated circuit. For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while being embedded in the display device 160 (eg, a display).

The processor 120, for example, executes software (eg, the program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to an embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190) to the volatile memory 132 . may be loaded into the volatile memory 132 , process commands or data stored in the volatile memory 132 , and store the resulting data in the non-volatile memory 134 . According to an embodiment, the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and an auxiliary processor 123 (eg, a graphic processing unit or an image signal processor) that can be operated independently or together with the main processor 121 . , a sensor hub processor, or a communication processor). Additionally or alternatively, the auxiliary processor 123 may be configured to use less power than the main processor 121 or to be specialized for a designated function. The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 may be, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display device 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the auxiliary processor 123 (eg, an image signal processor or a communication processor) may be implemented as a part of another functionally related component (eg, the camera module 180 or the communication module 190). have.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input device 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (eg, a stylus pen).

The sound output device 155 may output a sound signal to the outside of the electronic device 101 . The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls. According to an embodiment, the receiver may be implemented separately from or as a part of the speaker.

The display device 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment, the display device 160 may include a touch circuitry configured to sense a touch or a sensor circuit (eg, a pressure sensor) configured to measure the intensity of a force generated by the touch. have.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input device 150 , or an external electronic device (eg, a sound output device 155 ) connected directly or wirelessly with the electronic device 101 . The sound may be output through the electronic device 102 (eg, a speaker or a headphone).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to an embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module may be a first network 198 (eg, a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network 199 (eg, a cellular network, the Internet, or It may communicate with the external electronic device 104 through a computer network (eg, a telecommunication network such as a LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified and authenticated.

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include one antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, RFIC) other than the radiator may be additionally formed as a part of the antenna module 197 .

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external

electronic devices

102 and 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of the operations performed by the electronic device 101 may be performed by one or more of the external

electronic devices

102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. The one or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C” and “A, Each of the phrases "at least one of B, or C" may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other components in question, and may refer to components in other aspects (e.g., importance or order) is not limited. that one (eg first) component is “coupled” or “connected” to another (eg, second) component with or without the terms “functionally” or “communicatively” When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, the internal memory 136 or the external memory 138) readable by a machine (eg, the electronic device 101) may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one of one or more instructions stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the at least one command called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store™) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly, online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, omitted, or , or one or more other operations may be added.

2 illustrates a photographing environment according to an exemplary embodiment.

In the example of FIG. 2 , the user 299 may directly perform image capturing using the electronic device 201 (eg, the electronic device 101 of FIG. 1 ). For example, the user 299 may check an image acquired using the camera 280 of the electronic device 201 through the display 260 of the electronic device 260 . In this example, the electronic device 201 may be assumed to be fixed by a tripod or the like. According to various embodiments of the present document, the electronic device 201 may provide image recording in which the composition set by the user 299 is maintained even in a fixed state. For example, the electronic device 201 may stream an image in which a set composition is maintained in real time.

In the example of FIG. 2 , the electronic device 201 is illustrated as a mobile phone, but embodiments of the present document are not limited thereto. For example, at least some components of the electronic device 201 may be configured as separate external electronic devices.

3 illustrates a photographing environment according to an exemplary embodiment.

In the example of FIG. 3 , the user 299 may perform image recording using the camera 280 . The user 299 may check the image using any electronic device having a display (eg, the TV 360 or the PC 320 ). In the example of FIG. 3 , the camera 280 and the display device (eg, the TV 360 or the PC 320 ) may constitute a photographing system corresponding to the electronic device 201 of FIG. 2 . Hereinafter, various embodiments will be described focusing on the electronic device 201 , but the fact that the display 260 and the camera 280 of the electronic device 201 may be configured as separate devices has been described above with reference to FIG. 3 . like a bar In this case, the operation of the electronic device 201 excluding the output of the image may be referred to as being performed by the camera device (eg, the camera 380 ).

According to an embodiment, the electronic device 201 (eg, the electronic device 101 of FIG. 1 ) includes a processor 220 , a memory 230 , a display 260 , a camera 280 , and/or a communication circuit ( 290) may be included.

The processor 220 (eg, the processor 120 of FIG. 1 ) may be set to control various components of the electronic device 201 . The processor 220 includes a memory 230 , a display 260 , and a camera 280 . , and/or may be operatively connected to the communication circuit 290. The processor 220 may be configured to perform various operations of the electronic device 120 to be described later based on one or more instructions stored in the memory 230. According to an embodiment, the processor 220 may include at least one processor. For example, the processor 220 may include a visual processing unit (VPU), a neural processing unit (NPU), or a central processing unit (CPU). processing unit) and/or at least one of a graphic processing unit (GPU) The processor 220 may include one or a plurality of chips.

The memory 230 (eg, the memory 130 of FIG. 1 ) may store one or more instructions. The memory 230 may store various pieces of information related to the electronic device 201 . For example, the memory 230 may store a moving image and/or a still image obtained by using the camera 280 . The memory 230 may store information on the padding area set by the user (eg, padding area setting information).

The display 260 (eg, the display device 160 of FIG. 1 ) may include a plurality of pixels and display an image using the plurality of pixels. The display 260 may include a touch sensor capable of receiving a touch input. The display 260 may be set to display at least a part of the image acquired by the camera 280 . For example, the processor 220 may provide a real-time image (eg, a preview) of an image acquired using the camera 280 through the display 260 .

The camera 280 (eg, the camera module 180 of FIG. 1 ) may include at least one camera. The camera 280 may include a plurality of lenses and a plurality of image sensors. For example, the camera 280 may include a plurality of camera units.

The communication circuit 290 (eg, the communication module 190 of FIG. 1 ) may provide a communication interface for communicating with another electronic device in a wired or wireless manner. The communication circuit 290 may be a communication interface configured to support short-range wireless communication (eg, Bluetooth, Wi-Fi, NFC, and/or 802.11 series short-range wireless communication). The communication circuit 290 may be a communication interface configured to support long-distance wireless communication (eg, cellular).

According to an embodiment, the electronic device 201 includes a gesture identifier 410 , an upper body detector 420 , a face detector 430 , a padding region setter 440 , and/or an image quality improver 450 . can do. The gesture identifier 410 , the upper body detector 420 , the face detector 430 , the padding region setter 440 , and/or the image quality improver 450 are, for example, dedicated hardware of the electronic device 201 . And it may be a software module implemented by a combination of software or implemented based on instructions stored in the memory 230 . The operation of the gesture identifier 410, the upper body detector 420, the face detector 430, the padding region setter 440, and/or the image quality improver 450, which will be described later, is performed by the electronic device 201 or the processor ( 220) may be referred to as an operation implemented by

The gesture identifier 410 may identify a gesture of an external object (eg, a user). For example, the gesture identifier 410 may identify a gesture based on image analysis and/or reflected wave analysis. According to an embodiment, the gesture identifier 410 may identify a gesture based on image analysis. The gesture identifier 410 may acquire at least one image of an object using the camera 280 . The gesture identifier 410 may identify an object corresponding to the gesture from the acquired image. For example, the gesture identifier 410 may identify a gesture including a movement by analyzing a plurality of consecutive images. In the gesture identification based on image analysis, the gesture identifier 410 may identify an object and a gesture of the object further based on depth information. For example, the gesture identifier 410 may be set to acquire the distance information using the camera 280 (eg, a dual lens camera) and/or a separate sensor (eg, an infrared sensor (not shown)). can According to an embodiment, the gesture identifier 410 may identify a gesture based on reflected wave analysis. For example, the gesture identifier 410 may emit a signal using the communication circuit 290 and/or a circuit configured to emit the signal. The gesture identifier 410 may identify an external object and/or a gesture of the external object by detecting a reflected signal in which the emitted signal is reflected by the external object. For example, the gesture identifier 410 may detect an external object based on the arrival time of the reflected wave. For example, the gesture identifier 410 may identify the movement of the object based on the Doppler effect. According to an embodiment, the gesture identifier 410 may identify an external object and a movement of the external object based on image analysis and reflected wave analysis.

The gesture identifier 410 may identify various gestures. For example, the gesture identifier 410 may identify a static gesture and/or a dynamic gesture. The static gesture may mean maintenance of a specified pose performed by a specified object for a specified time or longer. For example, a static gesture may include a user holding a specified pose using a face, hand, arm, and/or body for a specified amount of time or longer. For example, the user may perform a gesture of designating a corresponding position by maintaining the position of the palm at a predetermined position for a specified time or longer in a state in which the palm is open. For example, the user may perform the gesture by not moving for more than a specified time in a state with folded arms. For example, the user may perform the gesture by maintaining the same facial expression for a specified time or longer. The dynamic gesture may mean a designated movement performed by a designated object. For example, a dynamic gesture may include a user performing a specified action using a face, hand, arm, and/or body. For example, a user may perform a dynamic gesture by repeatedly clenching and opening a finger. For example, a user may perform a dynamic gesture by shaking their hand or shaking their head. For example, a user may perform a dynamic gesture by clapping. In the above-described examples, the gesture is divided into a dynamic gesture and a static gesture, but embodiments of the present document are not limited thereto. For example, a gesture may be classified based on an object performing the gesture (eg, a head, face, hand, arm, and/or body). In addition, the above-described gestures are exemplary, and embodiments of the present document are not limited thereto. In one embodiment, a visual gesture may be combined with an acoustic gesture. In the following examples, a gesture in which a visual gesture and an acoustic element are combined may also be referred to as a gesture. The gesture identifier 410 may identify a gesture based on a convolutional neural network (CNN).

The upper body detector 420 may detect the upper body of an object (eg, a user). For example, “upper body” may mean from the bottom of the head of the object to the top of the leg. As another example, “upper body” may mean above the leg from the head of the object. The upper body detector 420 may identify the upper body based on image analysis. For example, the upper body detector 420 may identify the upper body by identifying a designated shape. The designated shape may include, for example, an omega-shape (eg, a shape corresponding to or resembling the letter “omega”). The upper body detector 420 may detect the upper body based on a convolutional neural network (CNN). For example, information on the upper body identified by the upper body detector 420 may be used by the gesture identifier 410 to identify a gesture.

The face detector 430 may detect a head and/or a face of an object (eg, a user). For example, the face detector 430 may detect an “expression” of the object based on the face of the object. The face detector 430 may detect the head and/or face of the object based on image analysis. For example, the face detector 430 may extract a feature point from an object identified as a head, and identify eyes, nose, mouth, and/or eyebrows based on the feature point. The face detector 430 may detect the upper body based on a convolutional neural network (CNN). For example, information on the face identified by the face detector 430 may be used to identify the gesture of the gesture identifier 410 .

The padding area setter 440 may set a padding area for image acquisition. For example, the padding area setter 440 may store padding area information set by the user in the memory 230 and/or an external electronic device (eg, a server). The padding area information may include upper padding, left padding, right padding, and/or lower padding information for the object. The padding area setter 440 may store and/or manage information on the padding area. For example, the padding area setter 440 may store a plurality of padding area settings and provide the plurality of padding area settings to the user. When a user input for selecting one of a plurality of padding area settings is received, the electronic device 201 may obtain an image based on the selected padding area setting. In the following examples, the padding area may be defined relative to the position and size of the object. The padding area may be referred to as areas other than the object within one frame of the image. Hereinafter, operations of the padding region setter 440 may be referred to by operations of the electronic device 201 described later with reference to FIGS. 5 to 12 .

The image quality improver 450 may perform quality improvement on an image obtained by using the camera 280 . According to an embodiment, the image quality improver 450 may perform quality improvement on all or part of an image using the camera 280 . For example, the image quality improver 450 may perform image quality improvement based on a frame (eg, at least a part of an image) set based on the padding area setting. Image quality improvement may include quality improvement for an image within a frame based on the image within the frame. For example, the image quality improver 450 may improve the image quality by using the image enhancement parameters generated based on the CNN. The image quality improvement may include digital processing-based image quality improvement for the acquired image, as well as changing parameters of the camera 280 for image quality improvement. For example, the image quality improver 450 may change optical characteristics of the camera 280 to improve image quality. For example, the image quality improver 450 may include a lens (eg, a normal lens, a zoom lens, and/or a wide-angle lens), an aperture value, an exposure value, a color temperature setting, and/or a zoom setting (eg, a zoom factor) for image acquisition. ) can be changed.

The configuration of the electronic device 201 illustrated in FIG. 4 is exemplary and embodiments of the present document are not limited thereto. For example, the electronic device 201 may not include at least one of the components illustrated in FIG. 4 . As another example, the electronic device 201 may further include a configuration not shown in FIG. 4 .

5 is a diagram illustrating the setting of a frame according to an embodiment.

In FIG. 5 , for example, it may be assumed that the user 599 uses the electronic device 201 to record self-video. For example, the photographing area 510 may correspond to the entire image that may be acquired by the camera 280 of the electronic device 201 . The frame 520 may correspond to at least a portion of the photographing area 510 . For example, the frame 520 may include, among the photographing areas 510 , an area intended for photographing by the user 599 , an area according to a padding area setting, an area used for image recording, and/or an area used for image transmission. It can be referred to as a region.

According to an embodiment, the frame 520 may be set relative to at least a portion of an object (eg, the user 599 ). For example, the position and size of the frame 520 may be relatively set based on the size and position of the object. In this document, an object is a head 550 (eg, a head identified by the face detector 430 ), an upper body 560 (eg, an upper body detected by the upper body detector 420 ), and/or a user ( 599) may be included. In one example, the object of the frame 520 may be set as the head 550 , the upper body 560 , or a user. In another example, the object of the frame 520 may be set as the head 550 and the upper body 560 . With reference to FIG. 5 , examples of the above-described object are examples, and embodiments of the present document are not limited thereto. Hereinafter, the term “object” may be referred to as a reference object constituting the frame 520 .

According to an embodiment, the electronic device 201 may determine a frame 520 based on the identified object and acquire an image of the frame 520 . For example, the electronic device 201 may identify the object and identify the frame 520 according to a padding area set for the identified object. The electronic device 201 may perform image enhancement only on an image corresponding to the frame 520 in the photographing area 510 . For example, the electronic device 201 may perform zoom-in to acquire an image including the frame 520 in order to improve the image. In this case, the photographing area 510 may be reduced according to zoom-in. For example, in order to improve the image, the electronic device 201 may perform image enhancement (eg, digital processing) only on the image corresponding to the frame 520 based on the image corresponding to the frame 520 . can

According to an embodiment, the electronic device 201 may capture an image using an image corresponding to the frame 520 . For example, the electronic device 201 may store only a portion corresponding to the frame 520 of the entire photographing area 510 in the memory 230 or transmit it to an external electronic device. The electronic device 201 may be set to store both an image corresponding to the entire photographing area 510 and an image 520 corresponding to the frame 520 in the memory 230 . For example, the electronic device 201 may store an image corresponding to the entire photographing area 510 and the image 520 as separate files.

The terms described above in relation to FIG. 5 below. Unless otherwise described, reference may be made to the same meaning as described above in relation to FIG. 5 .

In the example of FIG. 6 , the user 599 may move to the left of the photographing area 510 . In this example, the electronic device 201 may move the frame 520 in the photographing area 510 according to the movement of the object. Since the position and relative size of the object in the frame 520 are maintained, a composition (eg, setting of a padding area) set by the user 599 may be maintained.

According to an embodiment, the electronic device 201 may not be able to maintain the composition set by the user 599 . For example, the electronic device 201 may no longer maintain the composition because the user 599 moves too far to the side of the photographing area 510 . In this case, the electronic device 201 may maintain only the size of the frame 520 without maintaining the position of the object in the frame 520 . For example, when the user 599 moves further to the left of the photographing area 510 , the electronic device 201 may maintain the size and position of the frame 520 in the photographing area 510 .

In the example of FIG. 7 , the user 599 may move toward the electronic device 201 . In this example, as the object advances, the size of the object obtained by the electronic device 201 may increase. In this case, the electronic device 201 may increase the size of the frame 520 in order to maintain the relative size of the object with respect to the frame 520 . Since the position and relative size of the object in the frame 520 are maintained, a composition (eg, padding area setting) set by the user 599 may be maintained. For example, in response to the user 599 moving forward (eg, moving toward the camera 280 ), the electronic device 201 may zoom out the camera 280 . As another example, in response to the user 599 moving backward (eg, moving away from the camera 280 ), the electronic device 201 may zoom in the camera 280 .

According to an embodiment, the electronic device 201 may not be able to maintain the composition set by the user 599 . For example, when the user 599 gets too close to the camera 280 , the electronic device 201 may not be able to maintain the composition any more even after zooming out to the maximum. In this case, the electronic device 201 may be set to maintain a zoomed-out state as much as possible or to transmit a preset image.

8 illustrates a padding area according to an embodiment.

According to an embodiment, the frame 520 may include an object and a padding area set relative to the object. For example, the object may correspond to the head 550 . In this case, the frame 520 may be set as an object (eg, the head 550) and a padding area set for the object. For example, the padding area may include upper padding (PU), left padding (PL), right padding (PR), and lower padding (PD) for the object. The padding area may be defined by upper padding (PU), left padding (PL), right padding (PR), and lower padding (PD). For example, padding can be applied to the boundaries of an identified object (e.g., head and/or upper body) or an identified object area (e.g., an area corresponding to head and/or upper body (e.g., a rectangle)). It may be set as the shortest distance from the boundary to the boundary of the frame.

According to an embodiment, the size of the padding area (eg, the length of the upper padding (PU), the left padding (PL), the right padding (PR), and the lower padding (PD)) is a value (eg, the length of the upper padding) relative to the size of the object. : ratio). For example, when the size of the object is changed, the electronic device 201 may also change the size of the padding area according to the relative value. By changing the padding area, the electronic device 201 may maintain the composition set by the user.

9 illustrates a padding area according to an embodiment.

In the example of FIG. 9 , the object 900 may correspond to the upper body or may correspond to the head and the upper body. For example, the padding area may include upper padding (PU), left padding (PL), and right padding (PR) for the object.

The padding area described above with reference to FIGS. 8 and 9 is exemplary, and embodiments of the present document are not limited thereto. For example, at least one of the upper padding PU, the left padding PL, the right padding PR, and the lower padding PD may be omitted.

10 illustrates a method of setting a padding area according to an embodiment.

In FIG. 10 , a user 599 may perform a first gesture 1001 . For example, the first gesture 1001 may be referred to as a gesture for setting a frame (eg, the frame 520 of FIG. 5 ). The first gesture 1001 may indicate positions of at least two points (eg, upper left and lower right corners or lower left and upper right corners of a frame). For example, the first gesture 1001 may include instructions for a first point indicated by the right hand 1002 of the user 599 and a second point indicated by the left hand 1003 of the user 599 .

According to an embodiment, the electronic device 201 may set the padding area based on the first point and the second point indicated by the first gesture 1001 . For example, the electronic device 201 may set the first frame 1030 such that the first point and the second point form diagonal vertices. For example, when the second gesture 1011 is received, the electronic device 201 may set the first frame 1030 based on the area indicated by the first gesture 1001 . In this case, the electronic device 201 may set the padding area based on the upper padding PU, the first left padding PL1, and the first right padding PR1 for the object.

According to an embodiment, the electronic device 201 may set a padding area based on the image ratio for recording/transmission set in the electronic device 201 and the first gesture 1001 . For example, an area designated by the first gesture 1001 (eg, the first frame 1030 ) may be different from an image ratio for recording/transmission. In this case, the electronic device 201 may set an additional padding area. For example, in the example of FIG. 10 , the first frame 1030 indicated by the first gesture 1001 may have a shorter horizontal length than the image ratio for recording/transmission. For example, the image ratio for recording/transmission may correspond to the second frame 1020 . In this case, the electronic device 201 may further set additional padding on the left and right sides so that the frame corresponds to the image ratio for recording/transmission. For example, the electronic device 201 may set a padding area based on the upper padding PU, the second left padding PL1, and the second right padding PR2 of the object. For another example, in the example of FIG. 10 , the first frame 1030 indicated by the first gesture 1001 may have a shorter vertical length than the image ratio for recording/transmission. In this case, the electronic device 201 may further set additional padding on the upper side and/or lower side so that the frame corresponds to the image ratio for recording/transmission.

According to an embodiment, the electronic device 201 may set a padding area based on the first gesture 1001 and the second gesture 1011 . For example, the electronic device 201 detects an object (eg, a photographing or tracking target) within a frame (eg, the first frame 1030 or the second frame 1020 ) indicated by the first gesture 1001 . object) may be determined based on the second gesture 1011 . For example, the electronic device 201 may set the position and size of the object when the second gesture 1011 is received as the position and size of the object in the frame. The electronic device 201 may set the padding area based on the position of the object, the size of the object, and the frame.

In FIG. 10 , the first gesture 1001 is illustrated as a gesture of opening an arm, but embodiments of the present document are not limited thereto. For example, the first gesture 1001 may be any gesture indicating two points of a frame. The second gesture 1011 is also illustrated as a gesture of crossing arms, but embodiments of the present document are not limited thereto. For example, the second gesture 1011 may include an OK sign using a finger, a gesture of raising an arm above the head to make OK, or a gesture of clapping. The second gesture 1011 may be any designated gesture. For example, the second gesture 1011 may be referred to as a gesture for completing setting of the padding area.

In FIG. 10 , a user performing a first gesture 1001 and a second gesture 1011 is set as a reference of the frame 1030 , but embodiments of the present document are not limited thereto. For example, the first gesture 1001 and the second gesture 1011 are performed by the user, and the object serving as a reference of the frame 1030 may not be the user. For example, the object may be a person other than the user, an animal, or a thing having any mobility. In this case, for example, the user may perform a separate input (eg, a gesture input) for designating an object that is a reference of the frame 1030 . As another example, when the second gesture 1011 is received, the electronic device may display a frame based on an object located at a designated position within the second frame 1020 (eg, the center of the second frame 1020 ). (1030) can be set.

11 illustrates a method of setting a padding area according to an embodiment.

In the example of FIG. 11 , the first gesture (eg, the first gesture 1001 of FIG. 10 ) may include a plurality of sub gestures. For example, the first gesture may include a first sub gesture 1101 and a second sub gesture 1102 . In order to set the padding area, the user 599 points to the first vertex of the frame 1120 using the first sub gesture 1101, and uses the second sub gesture 1102 to set the first vertex of the frame 1120. 2 vertices (eg, a vertex diagonal to the first vertex) may be indicated. For example, the first sub gesture 1101 may include a palm gesture 1111 . For example, the second sub-gesture 1102 may include a palm gesture 1112 .

According to an embodiment, the electronic device 201 may set the padding area based on the first point and the second point indicated by the first gesture. For example, the electronic device 201 may set the frame 1120 so that the first point and the second point form diagonal vertices. For example, when the second gesture 1103 is received, the electronic device 201 may set the frame 1120 based on the area indicated by the first gesture. In this case, the electronic device 201 may set the padding area based on the upper padding PU, the left padding PL, and the right padding PR of the object.

According to an embodiment, the electronic device 201 may set the padding area based on the first gesture (eg, the first sub-gesture 1111 and the second sub-gesture 1112 ) and the second gesture 1103 . . For example, the electronic device 201 determines the position of an object (eg, an object to be photographed or tracked) within the frame 1120 indicated by the first sub gesture 1101 and the second sub gesture 1102 . may be determined based on the second gesture 1103 . For example, the electronic device 201 may set the position and size of the object when the second gesture 1103 is received as the position and size of the object in the frame. The electronic device 201 may set the padding area based on the position of the object, the size of the object, and the frame.

In FIG. 11 , the first sub-gesture 1101 and the second sub-gesture 1102 are illustrated as opening arms, but embodiments of the present document are not limited thereto. For example, each of the first sub-gesture 1101 and the second sub-gesture 1102 may be an arbitrary gesture indicating a point in a frame. The second gesture 1103 is also illustrated as a cross-arm gesture, but embodiments of the present document are not limited thereto. For example, the second gesture 1103 may be referred to as an arbitrary gesture for completing setting of the padding area.

In FIG. 11 , the user performing the second gesture 1103 is set as a reference of the frame 1120 , but embodiments of the present document are not limited thereto. For example, the second gesture 1103 is performed by the user, and the object that is the reference of the frame 1120 may not be the user. For example, the object may be a person other than the user, an animal, or a thing having any mobility. In this case, for example, the user may perform a separate input (eg, a gesture input) for designating an object as a reference of the frame 1120 . As another example, when the second gesture 1103 is received, the electronic device receives the frame 1120 based on an object located at a specified position (eg, the center of the acquired image) in the image acquired using the camera. ) can be set.

12 illustrates a flowchart 1200 of an image recording method according to an embodiment.

The image recording method of FIG. 12 may be performed based on the execution of the camera application of the electronic device 201 or the driving of the camera.

In operation 1205, the electronic device 201 may receive a padding area setting input. For example, the padding area setting input may be any input instructing entry into a setting mode for setting the padding area. The padding area setting input may include, for example, a touch input, a gesture input, and/or a voice input. The electronic device 201 may receive a padding area setting input based on a touch input to the display 260 of the electronic device 201 . The electronic device 201 may receive a padding area setting input by identifying a designated gesture using the camera 280 of the electronic device 201 . When an input for setting a padding area is received, the electronic device 201 may start setting a padding area. For example, the electronic device 201 may set the padding area according to the method described above with reference to FIGS. 10 and 11 .

In operation 1210, the electronic device 201 may obtain a first gesture input. For example, the electronic device 201 may capture an image corresponding to the first gesture input by using the camera 280 of the electronic device 201 and acquire the first gesture based on the captured image. As another example, the electronic device 201 may obtain the first gesture input by using the camera 280 and the microphone of the electronic device 201 . The first gesture input may be any gesture input indicating at least two points. For example, the first gesture input may include one gesture indicating two points (eg, the first gesture 1001 of FIG. 10 ) or a plurality of gestures (eg, the first sub gesture 1101 and the second gesture of FIG. 11 ) 2 sub-gestures 1102).

In operation 1215 , the electronic device 201 determines whether a second gesture input (eg, the second gesture 1002 of FIG. 10 or the second gesture 1103 of FIG. 11 ) is obtained. For example, the second gesture input may be a gesture input indicating completion of setting the padding area. The electronic device 201 may obtain the second gesture input by using the camera 280 and/or the microphone of the electronic device 201 . For example, as the second gesture input, the voice input may be a body gesture. The second gesture input may be a voice input and a body gesture.

In one example, if the second gesture is not received within a specified time after the reception of the first gesture input, the electronic device 201 may determine that the setting of the padding area has failed. In this case, the electronic device 201 may end the setting of the padding area. In one example, when an input for canceling the setting of the padding area is received, the electronic device 201 may return to the state before operation 1210 . For example, an input for canceling the setting of the padding area may be referred to as a third gesture. For example, the third gesture may correspond to an X-shape display using a finger or an arm. When the third gesture is received, the electronic device 201 may be configured to ignore the previously acquired first gesture input and acquire a new first gesture input.

When the second gesture input is not obtained (eg, operation 1215 -N), the electronic device 201 may continuously acquire the first gesture input. For example, when a plurality of first gesture inputs are received, the electronic device 201 may set a padding area based on the most recently received first gesture input before acquiring the second gesture.

When the second gesture input is obtained (eg, operation 1215 -Y), the electronic device 201 may set a padding area based on the acquisition of the second gesture. For example, when the second gesture input is obtained, the electronic device 201 may set the padding area based on the area indicated by the first gesture input. For example, the setting of the padding area may be referred to by the examples described above with reference to FIGS. 5 to 11 . According to an embodiment, the electronic device 201 may set a padding area based on the first gesture input and the second gesture input. For example, the electronic device 201 may set a frame based on the first gesture, set the position and size of an object in the frame based on the second gesture, and set a padding area based on the frame and the object. . As described above, the subject performing the first gesture and the second gesture and the object may be different from each other.

According to an embodiment, the electronic device 201 may store the set padding area in the memory 230 of the electronic device 201 . For example, the saved padding area setting may act as a kind of template. When photographing using the camera 280 , the electronic device 201 may provide the user with the stored padding area setting, and when the user's input for a specific template is received, the electronic device 201 may set the padding area using the corresponding template.

In operation 1225, the electronic device 201 may record the image of the user according to the set padding area. For example, the electronic device 201 may record an image while tracking a found object (eg, a user) according to a determined padding area. For example, the electronic device 201 may also change the size of the padding area according to a change in the size of the object. For example, the electronic device 201 may start recording the image when an input instructing to start recording is received or in response to obtaining the second gesture.

Claims

An electronic device comprising:

camera;

display;

a processor operatively coupled to the camera and the display; and

a memory coupled to the processor, wherein the memory, when executed, causes the processor to:

Obtaining a user's first gesture using the camera,

setting a frame based on the first gesture,

obtaining a second gesture of the user by using the camera;

Set a padding area for the object in the frame based on the second gesture,

and storing one or more instructions to record at least a partial image of an image acquired by the camera according to the frame and the padding area while tracking the object.
The method of claim 1,

and the one or more instructions, when executed, cause the processor to set the padding area based on the size of the object and the location of the object within the frame.
3. The method of claim 2,

and the one or more instructions, when executed, cause the processor to set the padding area based on the size of the object and the location of the object when the second gesture was received.
4. The method of claim 3,

and the one or more instructions, when executed, cause the processor to determine the size of the padding region as a size relative to the size of the object.
The method of claim 1,

The first gesture includes a gesture indicating two vertices of a rectangle that are diagonal to each other,

and the one or more instructions, when executed, cause the processor to set the frame based on the rectangle.
6. The method of claim 5,

The first gesture includes a plurality of sub gestures.
The method of claim 1,

and the one or more instructions, when executed, cause the processor, upon receiving a recording start input, to record at least a partial image of an image acquired by the camera according to the frame and the padding area.
8. The method of claim 7,

The at least some images correspond to images in the frame.
9. The method of claim 8,

and the one or more instructions, when executed, cause the processor to record the at least some image by adjusting the size of the frame according to the size of the image of the object obtained by the camera.
10. The method of claim 9,

and the one or more instructions, when executed, cause the processor to adjust the size and position of the frame such that the size and position of the image of the object is maintained with respect to the padding area.
A method for acquiring an image of an electronic device, comprising:

acquiring a user's first gesture by using the camera of the electronic device;

setting a frame based on the first gesture;

acquiring a second gesture of the user using the camera;

setting a padding area for the object in the frame based on the second gesture; and

and recording at least a partial image of the image acquired by the camera according to the frame and the padding area while tracking the object.
12. The method of claim 11,

The setting of the padding area for the object includes setting the padding area based on the position of the object in the frame and the size of the object.
13. The method of claim 12,

The setting of the padding area based on the position of the object in the frame and the size of the object may include the padding area based on the position of the object and the size of the object when the second gesture is received. A method comprising the action of setting
14. The method of claim 13,

The size of the padding area is determined as a size relative to the size of the object.
12. The method of claim 11,

The first gesture includes a gesture indicating two vertices of a rectangle that are diagonal to each other,

The setting of the frame includes setting the frame based on the rectangle.