KR20210138483A - Method or electronic device providing image shooting guide - Google Patents

Abstract

An embodiment of the present invention relates to an electronic device which may comprise a camera, a distance sensor, a display, and a processor. The processor obtains an image including an object through the camera, obtains distance data for the distance between the object and the electronic device through the distance sensor, determines whether a defocus occurs or not, based on the distance data, detects an area of interest which includes at least a part of the object from the image, crops an area including the area of interest from the image, and magnifies and represents the cropped area along with a message indicating the defocus on the display. Accordingly, a user can change shooting conditions according to what the electronic device proposes to obtain an image without any defocus.

Description

METHOD OR ELECTRONIC DEVICE PROVIDING IMAGE SHOOTING GUIDE

The present disclosure relates to a technique for providing an image capturing guide to a user using a camera and a distance sensor provided in an electronic device.

The minimum focal length refers to the minimum distance between the image sensor and the subject required to focus on the subject. When a subject is located within a minimum focal length from a photographing device such as a camera, a defocused image is photographed due to an optical limitation.

Recently, as the size of an image sensor mounted in a mobile device increases, the minimum focal length of a camera increases. Accordingly, a problem in which a defocused image is captured for a subject located within the minimum focal length is increasing.

In addition, the electronic device according to the prior art uses only low level features such as frequency analysis of the image without considering the unique characteristics of the subject when analyzing the blur included in the image.

An electronic device according to an embodiment of the present document may include a camera, a distance sensor, a display, and an image electrically connected to the camera, the distance sensor, and the display, and including a subject through the camera. obtain, obtain distance data on the distance between the subject and the electronic device through the distance sensor, determine that defocus has occurred based on the distance data, and Detects a region of interest including at least a part of a subject, crops a region including at least the region of interest in the image, and enlarges the cropped region with a message instructing the defocus on the display and may include at least one processor configured to display.

The method of operating an electronic device according to an embodiment of the present document includes: acquiring an image including a subject through a camera included in the electronic device; and the subject and the electronic device through a distance sensor included in the electronic device An operation of acquiring distance data for a distance between the two sides, an operation of determining that defocus has occurred based on the distance data, an operation of detecting a region of interest including at least a part of the subject in the image, and an operation of detecting a region of interest in the image cropping a region including at least the region of interest, and enlarging and displaying the cropped region together with a message instructing the defocus on the display.

An electronic device according to an embodiment of the present document is electrically connected to a camera, a distance sensor, a display, and the camera, the distance sensor, and the display. obtaining an image including a subject through the camera, obtaining distance data on a distance between the subject and the electronic device through the distance sensor, and determining that defocus has occurred based on the distance data, The display may include at least one processor configured to display a message for inducing a distance between the subject and the electronic device to be greater than or equal to a minimum focal length.

According to various embodiments disclosed herein, the user may recognize that the blur included in the image is the blur caused by the defocus generated by photographing a subject located within the minimum focal length. In addition, the electronic device according to the present disclosure may suggest an optimal shooting condition to the user by analyzing the characteristics of the subject, and the user may acquire an image without defocus by changing the shooting condition according to the condition suggested by the electronic device can do.

In addition, since the electronic device according to the present disclosure utilizes not only low-level features but also high-level features such as object detection and texture check when analyzing an image, optimal shooting conditions according to characteristics of a subject can suggest

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. .

1 illustrates an electronic device according to an embodiment.
2 is a block diagram of an electronic device according to an exemplary embodiment.
FIG. 3 is a flowchart illustrating a method of an electronic device providing an image capturing guide while cropping and enlarging an ROI and displaying it on a display, according to an exemplary embodiment.
4 is a flowchart illustrating a method of providing an image capturing guide when a user input for a first visual affordance is received, according to an embodiment.
5 is a flowchart illustrating an operation when a user input for a second visual affordance is received, according to an embodiment.
6 is a flowchart illustrating a method of analyzing image and distance data according to an exemplary embodiment.
7 illustrates an example of providing an image capturing guide according to an exemplary embodiment.
8 illustrates an example of a region of interest that is cropped and enlarged in an image as a distance between a subject and an electronic device increases, according to an embodiment.
9 illustrates an example of a UI for guiding a region of interest to be located in the center of an image, according to an embodiment.
10 is a flowchart illustrating a method of providing an image capturing guide through a message instructing defocus according to an exemplary embodiment.
11 illustrates an example of a message indicating defocus according to an embodiment.
12 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;
13 is a block diagram illustrating a camera module according to various embodiments of the present disclosure;

1 illustrates an electronic device according to an embodiment.

Referring to FIG. 1 , the display 110 may be disposed on the front surface of the electronic device 100 according to an embodiment. In an embodiment, the display 110 may occupy most of the front surface of the electronic device 100 . A display 110 and a bezel 120 region surrounding at least some edges of the display 110 may be disposed on the front surface of the electronic device 100 . In the example of FIG. 1 , the display 110 includes a flat area 111 and a curved area 112 extending from the flat area 111 toward the side of the electronic device 100 . can do. Although the curved area 112 is displayed on only one side (eg, the left side) in FIG. 1 , it may be understood that the curved area is also formed on the opposite side. Also, the electronic device 100 illustrated in FIG. 1 is an example, and various embodiments are possible. For example, the display 110 of the electronic device 100 may include only the flat area 111 without the curved area 112 , or may include the curved area 112 only at one edge instead of both sides. Also, in an embodiment, the curved area may extend to the rear surface of the electronic device 100 , so that the electronic device 100 may include an additional planar area.

In an embodiment, a fingerprint sensor 141 for recognizing a user's fingerprint may be included in the first area 140 of the display 110 . Since the fingerprint sensor 141 is disposed on a layer below the display 110 , the fingerprint sensor 141 may not be recognized by the user or may be difficult to recognize. Also, in addition to the fingerprint sensor 141 , a sensor for additional user/biometric authentication may be disposed in a portion of the display 110 . In another embodiment, a sensor for user/biometric authentication may be disposed in one area of the bezel 120 . For example, the IR sensor for iris authentication may be exposed through one area of the display 110 or may be exposed through one area of the bezel 120 .

In an embodiment, the sensor 143 may be included in at least one area of the bezel 120 of the electronic device 100 or at least one area of the display 110 . The sensor 143 may be disposed at a distance adjacent to a camera module (eg, the front camera 131 and the rear camera 132) or may be formed as a single module with the camera module.

In an embodiment, the front camera 131 may be disposed on the front side of the electronic device 100 . In the embodiment of FIG. 1 , the front camera 131 is shown to be exposed through one area of the display 110 , but in another embodiment, the front camera 131 may be exposed through the bezel 120 .

In one embodiment, the display 110 is on the back of the screen display area (eg, the flat area 111, the curved area 112), a sensor module, a camera module (eg, the front camera 131, the rear camera 132) )), and a light emitting device (eg, LED).

In an embodiment, a camera module may be disposed on the rear surface of at least one of the front surface, the side surface, and/or the rear surface of the electronic device 100 to face the front surface, the side surface, and/or the rear surface. For example, the front camera 131 may not be visually exposed as a screen display area (eg, a flat area 111, a curved area 112), and includes a hidden under display camera (UDC). can do. In an embodiment, the electronic device 100 may include one or more front cameras 131 . For example, the electronic device 100 may include two front cameras, such as a first front camera and a second front camera. In an embodiment, the first front camera and the second front camera may be cameras of the same type having the same specifications (eg, pixels), but the first front camera and the second front camera may be implemented as cameras of different specifications. . The electronic device 100 may support a function related to a dual camera (eg, 3D imaging, auto focus, etc.) through two front cameras.

In an embodiment, a rear camera 132 may be disposed on the rear side of the electronic device 100 . The rear camera 132 may be exposed through the camera area 130 of the rear cover 160 . In an embodiment, the electronic device 100 may include a plurality of rear cameras disposed in the camera area 130 . For example, the electronic device 100 may include two or more rear cameras. For example, the electronic device 100 may include a first rear camera, a second rear camera, and a third rear camera. The first rear camera, the second rear camera, and the third rear camera may have different specifications. For example, whether the first rear camera and the second rear camera and/or the third rear camera support FOV, pixels, aperture, optical zoom/digital zoom, whether image stabilization is supported, and the lens set included with each camera. The type and/or arrangement of the elements may be different from each other. For example, the first rear camera may be a general camera, the second rear camera may be a wide-angle camera (eg, a wide-angle camera), and the third rear camera may be a telephoto camera. In the embodiments of the present document, descriptions of functions or characteristics of the front camera may be applied to the rear camera, and vice versa.

In an embodiment, in the camera area 130 , a distance sensor 145 for detecting a distance between a subject and the electronic device 100 , a sensor for detecting an object, and/or various hardware assisting photographing such as a flash may be additionally disposed.

In an embodiment, the distance sensor 145 may be disposed at a distance adjacent to a camera module (eg, the front camera 131 and the rear camera 132) or may be formed as a single module with the camera module. For example, the distance sensor 145 may operate as at least a part of an infrared (IR) camera (eg, a time of flight (TOF) camera, a structured light camera) or as at least a part of a sensor module. For example, the TOF camera may be operated as at least a part of a sensor module for detecting the distance to the subject.

In an embodiment, at least one physical key may be disposed on a side portion of the electronic device 100 . For example, the first function key 151 for turning on/off the display 110 or turning on/off the power of the electronic device 100 may be disposed on the right edge with respect to the front of the electronic device 100 . have. In an embodiment, the second function key 152 for controlling the volume or screen brightness of the electronic device 100 may be disposed on the left edge with respect to the front surface of the electronic device 100 . In addition to this, additional buttons or keys may be disposed on the front or rear of the electronic device 100 . For example, a physical button or a touch button mapped to a specific function may be disposed in a lower region of the front bezel 120 .

The electronic device 100 illustrated in FIG. 1 corresponds to one example, and the shape of the device to which the technical idea disclosed in this document is applied is not limited. The technical idea disclosed in this document is applicable to various user devices. For example, by adopting the flexible display 110 and the hinge structure, the technical idea disclosed in this document may be applied to a foldable electronic device that can be folded in a horizontal direction or a foldable electronic device in a vertical direction, a tablet, or a notebook computer. . For example, although the electronic device 100 of the illustrated example has a bar-type or plate-type appearance, various embodiments of the present disclosure are not limited thereto. For example, the illustrated electronic device may be a part of a rollable electronic device. A rollable electronic device may be understood as an electronic device capable of bending and deforming the display 110 , so that at least a part thereof may be wound or rolled or accommodated in the electronic device 100 . . The rollable electronic device has a screen display area (eg, a flat area 111, a curved area 112) by unfolding the display 110 or exposing a larger area of the display 110 to the outside according to the user's needs. can be used to expand . The display 110 may also be referred to as a slide-out display or an expandable display.

Hereinafter, various embodiments will be described with reference to the electronic device 100 illustrated in FIG. 1 for convenience of description.

2 is a block diagram of an electronic device according to an exemplary embodiment.

Referring to FIG. 2 , the electronic device 100 may include a camera 210 , a distance sensor 145 , a processor 220 , and a display 110 .

According to an embodiment, the camera 210 may include the rear camera 132 or the front camera 131 illustrated in FIG. 1 .

According to an embodiment, since the camera 210 may focus on a subject located further than the minimum focal length, the processor 220 displays an image in which the subject is clearly expressed when the subject is located further than the minimum focal length. can be obtained Since the camera 210 cannot focus on a subject located closer than the minimum focal length, defocus may occur. For example, when the minimum focal length of the camera 210 is 50 cm, the camera 210 may focus on a subject 70 cm away from the image sensor, but cannot focus on a subject 30 cm away from the image sensor.

According to an embodiment, the distance sensor 145 may detect an object in a non-contact manner or measure a distance to the object using ultrasonic waves. For example, the distance sensor 145 may measure a distance between the subject and the electronic device 100 . The processor 220 may use the distance data acquired through the distance sensor 145 to determine whether the subject is located at a position farther or closer than the minimum focal length of the camera 210 .

According to an embodiment, the processor 220 may be understood to include at least one processor. For example, the processor 220 may include at least one of an application processor (AP), an image signal processor (ISP), and a communication processor (CP). In this respect, the processor 220 may be referred to as at least one processor or one or more processors.

In an embodiment, the processor 220 may execute/control various functions supported by the electronic device 100 . For example, the processor 220 may execute an application by executing a code written in a programming language stored in a memory, and may control various hardware. For example, the processor 220 may execute an application supporting a photographing function stored in the memory. In addition, the processor 220 may set and support an appropriate shooting mode so that the camera 210 may be executed and the camera 210 may perform an operation intended by the user.

According to an embodiment, the processor 220 may analyze the image acquired through the camera 210 and the distance data acquired through the distance sensor 145 . The processor 220 may use low-level features to analyze frequency components in the image when analyzing the image. In addition, the processor 220 may analyze the image using high level features such as object detection, texture check, and extract high level features. Details will be described later with reference to FIG. 6 .

In an embodiment, the display 110 is an execution screen of an application executed by the processor 220 , an image acquired through the camera 210 , or an image cropped and enlarged by the processor 220 . , at least one of a message inducing a user's action, or a visual affordance may be displayed. In addition, the processor 220 may display the image data acquired through the camera 210 on the display 110 in real time.

In an embodiment, the display 110 may be implemented integrally with the touch panel. The display 110 may support a touch function, detect a user input such as a touch using a finger, and transmit it to the processor 220 . The display 110 may be connected to a display driver integrated circuit (DDIC) for driving the display 110 , and the touch panel may be connected to a touch IC that detects touch coordinates and processes a touch-related algorithm. In one embodiment, the display driving circuit and the touch IC may be integrally formed, and in another embodiment, the display driving circuit and the touch IC may be formed separately. The display driving circuit and/or the touch IC may be electrically connected to the processor 220 .

FIG. 3 is a flowchart illustrating a method in which the electronic device 100 provides an image capturing guide while cropping and enlarging a region of interest and displaying it on the display 110, according to an embodiment. The method described in FIG. 3 may be executed by the electronic device 100 or the processor 220 of the electronic device 100 .

According to an embodiment, in operation 310 , the processor 220 acquires an image including the subject through the camera 210 , and a distance between the subject and the electronic device 100 through the distance sensor 145 . data can be obtained.

According to an embodiment, in operation 320 , the processor 220 may determine that defocus has occurred based on the distance data.

According to an embodiment, when the distance between the subject and the electronic device 100 is within the minimum focal length, defocus may occur. At this time, the processor 220 may analyze the distance data acquired through the distance sensor 145 to determine that defocus has occurred. In addition, the processor 220 may analyze the amount of blur included in the image, and may supplementally use it to determine that defocus has occurred. The processor 220 may use the image analysis result to correct an error that may occur in the distance data analysis result. The processor 220 may determine based on the distance data that the blur included in the image is due to defocus caused by shooting at less than the minimum focal length.

According to an embodiment, in operation 330 , the processor 220 may detect an ROI including at least a portion of the subject in the image.

According to an embodiment, the processor 220 may detect an ROI through object detection. The processor 220 may identify a region corresponding to the subject from among the entire image and designate it as a region of interest. The ROI may include the entire subject or a part of the subject. Details will be described later with reference to FIG. 6 .

According to an embodiment, when detecting the ROI, the processor 220 may display a line surrounding the ROI on the display 110 . For example, the processor 220 may display a box surrounding the ROI on the display 110 .

According to an embodiment, in operation 340 , the processor 220 may crop a region including at least the region of interest in the image. The processor 220 may crop only the region of interest or a wider region including the region of interest.

According to an embodiment, in operation 350 , the processor 220 may enlarge and display the cropped area together with a message instructing the defocus on the display 110 .

According to an embodiment, the message includes information that defocus has occurred because the electronic device 100 is located closer than the minimum focal length to the subject, or that the electronic device 100 is moved away from the subject. may include For example, the message may be a content to move the electronic device 100 away from the subject by 30 cm or more.

According to an embodiment, the processor 220 may notify the user of the fact that the defocus has occurred in the ROI in the image through an operation of enlarging the cropped area and displaying it on the display 110 . In addition, the processor 220 may induce the user to move the electronic device 100 away from the subject in order to resolve the defocus by displaying a message instructing the defocus on the display 110 . The user may recognize that the defocus has occurred and that the electronic device 100 needs to be moved away from the subject from the message instructing the defocus displayed on the electronic device 100 and the image that is out of focus on the subject. can The user may move the electronic device 100 away from the subject, and when the distance between the subject and the electronic device 100 is equal to or greater than the minimum focal length, the processor 220 obtains a clear image in focus on the subject. can

According to an embodiment, when detecting the ROI in operation 330 , the processor 220 may display a first visual affordance on the display 110 . When there is a user input for the first visual affordance, the processor 220 crops a region including at least the region of interest in the image, and displays the cropped region along with a message instructing the display 110 to defocus. Can be enlarged to display. Details will be described later with reference to FIG. 4 .

According to an embodiment, when the processor 220 detects the region of interest in operation 330 , even if there is no separate user input, the processor 220 automatically crops the region including at least the region of interest in the image and displays it on the display 110 . The cropped area may be enlarged and displayed together with a message indicating defocus.

4 is a flowchart illustrating a method of providing an image capturing guide when a user input for a first visual affordance is received, according to an exemplary embodiment.

According to an embodiment, in operation 410 , the processor 220 may detect an ROI including at least a portion of the subject in the image. Operation 410 may correspond to operation 330 of FIG. 3 .

According to an embodiment, in operation 420 , the processor 220 may display the first visual affordance on the display. The first visual affordance may be in the form of a soft button displayed on the display 110 .

According to an embodiment, the electronic device 100 may induce the user to photograph the subject at a magnification suitable for photographing the subject. For example, when defocus occurs, the processor 220 may display a first visual affordance capable of adjusting the magnification of the camera lens to a specific magnification on the display.

According to an embodiment, in operation 430 , the processor 220 may determine that a user input for the first visual affordance has been received. For example, the processor 220 may receive a touch input or a voice command for touching the touch screen with a finger or a manipulation device.

According to an embodiment, in operation 440 , when a user input for the first visual affordance is received, the processor 220 may crop a region including at least the region of interest in the image. Operation 440 may correspond to operation 340 of FIG. 3 .

According to an embodiment, in operation 450 , the processor 220 may enlarge and display the cropped area together with a message instructing the defocus on the display 110 . Operation 450 may correspond to operation 350 of FIG. 3 .

According to an embodiment, the processor 220 may display the first visual affordance on the display 110 to display a cropped and enlarged image on the display 110 according to a user's selection. For example, the user may see the cropped and enlarged image displayed on the display 110 by touching the first visual affordance to accurately recognize a situation in which defocus has occurred. As another example, the user may be provided with a cropped and non-magnified image without touching the first visual affordance.

5 is a flowchart illustrating an operation when a user input for a second visual affordance is received, according to an embodiment.

According to an embodiment, in operation 510 , the processor 220 displays the second visual affordance on the display 110 while enlarging the cropped area together with the message instructing the defocus. can The second visual affordance may be in the form of a soft button displayed on the display 110 .

According to an embodiment, the processor 220 detects the region of interest and then automatically instructs defocus when a user input for the first visual affordance displayed on the display 110 is received or as the region of interest is detected. may be displayed on the display 110 by enlarging the cropped area together with the message. In this case, the processor 220 may display the second visual affordance together with the message and the cropped and enlarged image on the display 110 .

According to an embodiment, in operation 520 , the processor 220 may determine that a user input for the second visual affordance is received. For example, the processor 220 may receive a touch input of touching the touch screen with a finger or a manipulation device (eg, an electronic pen).

According to an embodiment, in operation 530 , the processor 220 may display an image that is not cropped or enlarged on the display 110 . When the user input for the second visual affordance is received, the processor 220 performs an operation of cropping a region including at least a region of interest in the image, and an operation of enlarging and displaying the cropped region on the display 110 . can stop The processor 220 may display the image acquired through the camera 210 on the display 110 without cropping or enlarging the image.

According to an embodiment, when a user input for the second visual affordance is received, the processor 220 may stop displaying a message instructing the defocus on the display 110 . For example, when a user input for the second visual affordance is received while the electronic device 100 is positioned farther than the minimum focal length from the subject, the processor 220 displays a message instructing the defocus to the display 110 . may not be displayed in According to another embodiment, even when a user input for the second visual affordance is received, the processor 220 may not stop displaying the message instructing the defocus. For example, even when a user input for the second visual affordance is received, if the distance between the subject and the electronic device 100 is within the minimum focal length and defocus is maintained accordingly, the processor 220 may defocus. A message instructing focus may be continuously displayed on the display 110 .

6 is a flowchart illustrating a method of analyzing image and distance data according to an exemplary embodiment.

According to an embodiment, in operation 610 , the processor 220 acquires an image through the camera 210 and acquires distance data on the distance between the subject and the electronic device 100 through the distance sensor 145 . can Operation 610 may correspond to operation 310 of FIG. 3 .

According to an embodiment, in operation 620 , the processor 220 analyzes the image acquired through the camera 210 to determine whether the blur included in the image is motion blur due to the movement of the subject. can

According to an embodiment, in operation 630, when it is determined that the blur in the image is not motion blur, the processor 220 determines that defocus has occurred based on the distance data, and the blur in the image is caused by defocus. It can be determined whether it is a blur or not. In an embodiment, the processor 220 may analyze the amount of blur included in the image and use the analysis result to determine whether defocus occurs.

According to an embodiment, when the distance between the subject and the electronic device 100 is within the minimum focal length, the processor 220 determines that defocus has occurred based on the distance data acquired through the distance sensor 145 . It can be done, and it can be determined that the blur in the image is the blur caused by the defocus.

According to an embodiment, in operation 640, if the processor 220 determines that the blur included in the image is the blur due to defocus, the ROI including at least a part of the subject is detected through object detection in the image. can

According to an embodiment, the object detection operation of the processor 220 is an operation of recognizing that there is an object in a specific part of the image by analyzing the image acquired through the camera 210, the It may include an operation of determining what an object is (object classification) and an operation of finding an exact location of the object within the image (object localization).

According to an embodiment, in operation 650 , the processor 220 may perform texture check and high-level feature extraction in the image.

According to an embodiment, the processor 220 may determine the subject's level of sharpness by recognizing the texture of the subject through a texture check. For example, since a subject including text or an image needs to be captured clearly, the electronic device 100 needs to adjust the distance between the subject and the electronic device 100 when blur exists in the image of the subject. It can be judged that For another example, when the subject is a blur caused by a surface without a wall or a pattern, the electronic device 100 may determine that it is not necessary to adjust the distance between the subject and the electronic device 100 .

According to an embodiment, the processor 220 may determine the characteristics of the blur included in the image through high-level feature extraction. For example, the processor 220 may determine that the blur included in the image is generated by the user's intention. As another example, the processor 220 may determine that the blur included in the image is a blur due to defocus.

According to an embodiment, operations 620 to 650 may correspond to operations 320 to 330 of FIG. 3 .

7 illustrates an example of providing an image capturing guide according to an exemplary embodiment.

According to an embodiment, reference numerals 710 to 730 illustrate examples of providing the image capturing guide illustrated in FIGS. 3 to 6 .

According to an embodiment, reference numeral 710 denotes a screen displayed on the display 110 when defocus occurs.

According to an embodiment, the processor 220 may analyze the image acquired through the camera 210 to determine whether motion blur is caused by the movement of the subject 711 . As a result of the image analysis, the processor 220 may determine that there is no directionality in the blur around the subject, and determine that the blur of the image is not the motion blur. If the blur of the subject 711 and its surroundings has a predetermined directionality, the processor 220 may determine that the blur of the image is blur caused by the movement of the subject 711 .

According to an embodiment, when it is determined that the blur included in the image is not motion blur, the processor 220 may determine whether the blur included in the image is a blur due to defocus based on distance data. The processor 220 may determine that defocus has occurred because the distance between the subject 711 and the electronic device 100 is within the minimum focal length, and may determine that the blur in the image is blur due to the defocus.

According to an embodiment, reference numeral 720 denotes a screen displayed on the display 110 when a region of interest including at least a part of the subject 711 in an image is detected. The ROI may include a part of the subject 711 .

According to an embodiment, when it is determined that the blur included in the image is due to defocus, the processor 220 may detect the ROI through object detection in the image. The processor 220 may detect the presence of the subject 711 in a specific portion within the image, determine that the subject 711 is a business card, and detect an area including at least a portion of the subject 711 as the ROI.

According to an embodiment, the processor 220 may determine the texture of the subject 711 through a texture check, and determine that the subject 711 should be captured clearly because the subject 711 includes characters. The processor 220 may determine that it is necessary to adjust the distance between the subject 711 and the electronic device 100 because the blur included in the image is not by the user's intention but by the defocus through high-level feature extraction.

According to an embodiment, when detecting the ROI, the processor 220 may display a line 721 surrounding the ROI on the display 110 . The processor 220 may display the line 721 displayed on the display 110 in a color and thickness that is easily recognized in the image.

According to an embodiment, when detecting the region of interest, the processor 220 may display the first visual affordance 725 on the display 110 . The first visual affordance 725 may include a phrase to enlarge the ROI.

According to an embodiment, reference numeral 730 denotes a screen displayed on the display 110 when a user input for the first visual affordance 725 is received.

According to an embodiment, when a user input for the first visual affordance 725 is received, the processor 220 may crop the ROI, enlarge the cropped region, and display it on the display 110 . The processor 220 may also display a message 733 instructing the defocus on the display 110 . The message 733 may include content to move the electronic device 100 away from the subject.

According to an embodiment, the processor 220 may automatically display an enlarged image such as reference number 730 without displaying the first visual affordance 725 on the display 110 after detecting the region of interest in the image. have. In this case, the electronic device 100 may provide the cropped and enlarged image of the ROI through the display 110 even if there is no user input.

According to an embodiment, the processor 220 may display the second visual affordance 735 on the display 110 together with an enlarged image of the cropped area and a message 733 indicating defocus. When a user input for the second visual affordance 735 is received, the processor 220 may stop cropping the ROI and enlarging the cropped region. The processor 220 may display an image that is not cropped or enlarged on the display 110 .

According to an embodiment, if there is a user input for the second visual affordance 735 after the electronic device 100 is moved further than the minimum focal length from the subject, the processor 220 displays an image that is not cropped or enlarged. While displaying 110, the display of the message 733 instructing defocus can be stopped. According to another embodiment, when there is a user input for the second visual affordance 735 in a situation where the electronic device 100 is located within the minimum focal distance from the subject and the defocus is maintained, the processor 220 performs the defocus. The instructing message 733 may be continuously displayed on the display 110 . In this document, visual affordance may be replaced/referred to by visual objects, UI items, icons, menus, indicators, and the like.

8 illustrates an example of a region of interest that is cropped and enlarged in an image as a distance between a subject and an electronic device increases, according to an embodiment.

According to an embodiment of the present disclosure, in FIG. 8 , when the electronic device 100 moves in a direction away from the subject, cropped or unmagnified images 810 , 820 , and 830 and cropped and enlarged ROI images 815 , 825 and 835) are examples.

According to an embodiment, reference numeral 810 denotes an image in which defocus occurs because the electronic device 100 is positioned closer than the minimum focal distance from the subject.

According to an embodiment, reference numeral 815 denotes an image displayed on the display 110 by cropping the region of interest 811 detected by the processor 220 within the image of reference numeral 810 and enlarging the cropped region. do. Since the distance between the subject and the electronic device 100 is less than the minimum focal length, the subject 812 may be defocused, so that a strong blur may appear in the image.

According to an embodiment, reference numeral 820 denotes an image when the electronic device 100 moves in a direction away from the subject, but the distance between the subject and the electronic device 100 is closer than the minimum focal length.

According to an embodiment, reference numeral 825 denotes an image displayed on the display 110 by cropping the region of interest 821 detected by the processor 220 within the image of reference numeral 820 and enlarging the cropped region. do. Since the distance between the subject and the electronic device 100 is greater than in the case of reference number 810, the subject 822 is less defocused than in the case of reference number 815, so that a weak blur may appear in the image.

According to an embodiment, reference numeral 830 denotes an image when the distance between the subject and the electronic device 100 is equal to or greater than the minimum focal length.

According to an embodiment, reference numeral 835 denotes an image displayed on the display 110 by cropping the region of interest 831 detected by the processor 220 in the image of reference numeral 830 and enlarging the cropped region. do. Since the electronic device 100 is located farther than the minimum focal length from the subject, the subject 832 may be in focus and thus a clear image may be obtained.

According to an embodiment, while the user moves the electronic device 100 in a direction away from the subject, the sizes of the subjects 812 , 822 , and 832 displayed on the display 110 may be kept constant. As the electronic device 100 moves away from the subject, the sizes of the regions of interest 811. 821 and 831 among the images 810, 820, and 830 acquired through the camera 210 decrease, but the electronic device 100 The ROIs 811 , 821 , and 831 are detected among the images 810 , 820 , and 830 and cropped, and the cropped region is enlarged and displayed on the display 110 . Accordingly, the sizes of the images 815 , 825 , and 835 of the region of interest displayed on the display 110 may be kept constant.

9 illustrates an example of a UI for guiding a region of interest to be located in the center of an image, according to an embodiment.

According to an embodiment, the processor 220 may detect an ROI including at least a part of the subject in the image. In this case, the region of interest may not be located in the center of the image, but may be located at the edge. The processor 220 may display a UI 910 such as an arrow on the display 110 . The user may adjust the position of the electronic device 100 through the UI 910 displayed on the display 110 so that the ROI is located in the center of the image.

10 is a flowchart illustrating a method of providing an image capturing guide through a message instructing defocus according to an exemplary embodiment. The method described in FIG. 10 may be executed by the electronic device 100 or the processor 220 of the electronic device 100 .

According to an embodiment, in operation 1010 , the processor 220 obtains an image through the camera 210 and obtains distance data on a distance between the subject and the electronic device 100 through the distance sensor 145 . can do. Operation 1010 may correspond to operation 310 of FIG. 3 .

According to an embodiment, in operation 1020 , the processor 220 may determine that defocus has occurred based on the distance data.

According to an embodiment, when the distance between the subject and the electronic device 100 is within the minimum focal length, defocus may occur. At this time, the processor 220 may analyze the distance data acquired through the distance sensor 145 to determine that defocus has occurred. In addition, the processor 220 may analyze the amount of blur included in the image, and may supplementally use it to determine that defocus has occurred. The processor 220 may use the image analysis result to correct an error that may occur in the distance data analysis result. The processor 220 may determine based on the distance data that the blur included in the image is due to defocus caused by shooting at less than the minimum focal length.

According to an embodiment, in operation 1030 , the processor 220 may display a message inducing the distance between the subject and the electronic device 100 to be equal to or greater than the minimum focal length on the display 110 . The user may recognize that defocus can be eliminated by moving the electronic device 100 away from the subject through the message displayed on the display 110 .

According to an embodiment, the processor 220 may additionally use at least one output device of a speaker or a light emitting device (eg, an LED lamp) to notify that defocus has occurred while displaying the message on the display 110 . can According to another embodiment, the processor 220 may use at least one output device among a speaker and a light emitting device instead of displaying the message on the display 110 to notify the user that defocus has occurred.

According to an embodiment, in operation 1040 , the processor 220 may determine that the distance between the subject and the electronic device 100 is equal to or greater than the minimum focal length. After recognizing that defocus has occurred through the message displayed on the display 110 , the user may move the electronic device 100 away from the subject. The electronic device 100 may analyze the distance data to determine that the distance between the subject and the electronic device 100 is equal to or greater than the minimum focal length, and thus defocus does not occur.

According to an embodiment, in operation 1050 , the processor 220 may remove the message from the display 110 in response to the distance between the subject and the electronic device 100 being equal to or greater than the minimum focal length. Since defocus does not occur when the user positions the electronic device 100 further than the minimum focal length from the subject, the processor 220 may not display the message on the display 110 .

11 illustrates an example of a message indicating defocus according to an embodiment.

According to an embodiment, the processor 220 may determine that defocus has occurred based on the distance data acquired through the distance sensor 145 . The processor 220 may display on the display 110 a message 1110 for inducing that the distance between the subject and the electronic device 100 is equal to or greater than the minimum focal length from the subject.

According to an embodiment, the processor 220 may display a message 1110 to move the electronic device 100 away from the subject on the display 110 . According to another embodiment, the processor 220 may suggest an accurate distance to move the electronic device 100 away from the subject through the message 1110 .

According to an embodiment, while the message 1110 is displayed on the display 110 , the processor 220 outputs at least one of a speaker or a light emitting device (eg, an LED lamp) to notify that defocus has occurred. additionally available.

12 is a block diagram of an electronic device 1201 in a network environment 1200, according to various embodiments of the present disclosure. Referring to FIG. 12 , in a network environment 1200 , the electronic device 1201 communicates with the electronic device 1202 through a first network 1298 (eg, a short-range wireless communication network) or a second network 1299 . It may communicate with the electronic device 1204 or the server 1208 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 1201 may communicate with the electronic device 1204 through the server 1208 . According to an embodiment, the electronic device 1201 includes a processor 1220 , a memory 1230 , an input module 1250 , a sound output module 1255 , a display module 1260 , an audio module 1270 , and a sensor module ( 1276), interface 1277, connection terminal 1278, haptic module 1279, camera module 1280, power management module 1288, battery 1289, communication module 1290, subscriber identification module 1296 , or an antenna module 1297 . In some embodiments, at least one of these components (eg, the connection terminal 1278 ) may be omitted or one or more other components may be added to the electronic device 1201 . In some embodiments, some of these components (eg, sensor module 1276 , camera module 1280 , or antenna module 1297 ) are integrated into one component (eg, display module 1260 ). can be

The processor 1220, for example, executes software (eg, a program 1240) to execute at least one other component (eg, a hardware or software component) of the electronic device 1201 connected to the processor 1220. It can control and perform various data processing or operations. According to an embodiment, as at least part of data processing or operation, the processor 1220 may store a command or data received from another component (eg, the sensor module 1276 or the communication module 1290 ) into the volatile memory 1232 . , process commands or data stored in the volatile memory 1232 , and store the result data in the non-volatile memory 1234 . According to an embodiment, the processor 1220 is a main processor 1221 (eg, a central processing unit or an application processor) or a secondary processor 1223 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 1201 includes a main processor 1221 and a sub-processor 1223 , the sub-processor 1223 uses less power than the main processor 1221 or is set to be specialized for a specified function. can The coprocessor 1223 may be implemented separately from or as part of the main processor 1221 .

The coprocessor 1223 may be, for example, on behalf of the main processor 1221 while the main processor 1221 is in an inactive (eg, sleep) state, or when the main processor 1221 is active (eg, executing an application). ), together with the main processor 1221, at least one of the components of the electronic device 1201 (eg, the display module 1260, the sensor module 1276, or the communication module 1290) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 1223 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 1280 or the communication module 1290). have. According to an embodiment, the auxiliary processor 1223 (eg, a neural network processing device) may include a hardware structure specialized in processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 1201 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 1208). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The AI model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 1230 may store various data used by at least one component (eg, the processor 1220 or the sensor module 1276) of the electronic device 1201 . The data may include, for example, input data or output data for software (eg, the program 1240 ) and instructions related thereto. The memory 1230 may include a volatile memory 1232 or a non-volatile memory 1234 .

The program 1240 may be stored as software in the memory 1230 , and may include, for example, an operating system 1242 , middleware 1244 , or an application 1246 .

The input module 1250 may receive a command or data to be used by a component (eg, the processor 1220 ) of the electronic device 1201 from the outside (eg, a user) of the electronic device 1201 . The input module 1250 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 1255 may output a sound signal to the outside of the electronic device 1201 . The sound output module 1255 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to an embodiment, the receiver may be implemented separately from or as a part of the speaker.

The display module 1260 may visually provide information to the outside (eg, a user) of the electronic device 1201 . The display module 1260 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 1260 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 1270 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 1270 acquires a sound through the input module 1250 or an external electronic device (eg, a sound output module 1255 ) directly or wirelessly connected to the electronic device 1201 . A sound may be output through the electronic device 1202 (eg, a speaker or headphones).

The sensor module 1276 detects an operating state (eg, power or temperature) of the electronic device 1201 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 1276 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 1277 may support one or more specified protocols that may be used for the electronic device 1201 to directly or wirelessly connect with an external electronic device (eg, the electronic device 1202 ). According to an embodiment, the interface 1277 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 1278 may include a connector through which the electronic device 1201 can be physically connected to an external electronic device (eg, the electronic device 1202 ). According to an embodiment, the connection terminal 1278 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 1279 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 1279 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 1280 may capture still images and moving images. According to an embodiment, the camera module 1280 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 1288 may manage power supplied to the electronic device 1201 . According to an embodiment, the power management module 1288 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 1289 may supply power to at least one component of the electronic device 1201 . According to an embodiment, the battery 1289 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 1290 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 1201 and an external electronic device (eg, the electronic device 1202, the electronic device 1204, or the server 1208). It can support establishment and communication through the established communication channel. The communication module 1290 may include one or more communication processors that operate independently of the processor 1220 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to an embodiment, the communication module 1290 may include a wireless communication module 1292 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 1294 (eg, : LAN (local area network) communication module, or a power line communication module) may be included. A corresponding communication module among these communication modules is a first network 1298 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 1299 (eg, legacy). It may communicate with the external electronic device 1204 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 1292 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 1296 within a communication network such as the first network 1298 or the second network 1299 . The electronic device 1201 may be identified or authenticated.

The wireless communication module 1292 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 1292 may support a high frequency band (eg, mmWave band) to achieve a high data rate. The wireless communication module 1292 uses various techniques for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 1292 may support various requirements specified in the electronic device 1201 , an external electronic device (eg, the electronic device 1204 ), or a network system (eg, the second network 1299 ). According to an embodiment, the wireless communication module 1292 includes a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 1297 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 1297 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 1297 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 1298 or the second network 1299 is connected from the plurality of antennas by, for example, the communication module 1290 . can be selected. A signal or power may be transmitted or received between the communication module 1290 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 1297 .

According to various embodiments, the antenna module 1297 may form a mmWave antenna module. According to an embodiment, the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving a signal of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 1201 and the external electronic device 1204 through the server 1208 connected to the second network 1299 . Each of the external electronic devices 1202 or 1204 may be the same or a different type of the electronic device 1201 . According to an embodiment, all or part of the operations performed by the electronic device 1201 may be executed by one or more of the external electronic devices 1202 , 1204 , or 1208 . For example, when the electronic device 1201 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 1201 may perform the function or service by itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 1201 . The electronic device 1201 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 1201 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 1204 may include an Internet of things (IoT) device. The server 1208 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 1204 or the server 1208 may be included in the second network 1299 . The electronic device 1201 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of this document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other such components, and refer to the component in another aspect (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of the present document may include a unit implemented in hardware, software, or firmware, for example, and interchangeably with terms such as logic, logic block, component, or circuit. can be used A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 1236 or external memory 1238) readable by a machine (eg, electronic device 1201) may be implemented as software (eg, the program 1240) including For example, a processor (eg, processor 1220 ) of a device (eg, electronic device 1201 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the at least one command called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. . According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, omitted, or , or one or more other operations may be added.

13 is a block diagram 1300 illustrating a camera module 1280, according to various embodiments. Referring to FIG. 13 , the camera module 1280 includes a lens assembly 1310 , a flash 1320 , an image sensor 1330 , an image stabilizer 1340 , a memory 1350 (eg, a buffer memory), or an image signal processor. (1360) may be included. The lens assembly 1310 may collect light emitted from a subject, which is an image capturing object. The lens assembly 1310 may include one or more lenses. According to an embodiment, the camera module 1280 may include a plurality of lens assemblies 1310 . In this case, the camera module 1280 may form, for example, a dual camera, a 360 degree camera, or a spherical camera. Some of the plurality of lens assemblies 1310 may have the same lens properties (eg, angle of view, focal length, auto focus, f number, or optical zoom), or at least one lens assembly may be a different lens assembly. It may have one or more lens properties different from the lens properties of . The lens assembly 1310 may include, for example, a wide-angle lens or a telephoto lens.

The flash 1320 may emit light used to enhance light emitted or reflected from the subject. According to an embodiment, the flash 1320 may include one or more light emitting diodes (eg, a red-green-blue (RGB) LED, a white LED, an infrared LED, or an ultraviolet LED), or a xenon lamp. The image sensor 1330 may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly 1310 into an electrical signal. According to an embodiment, the image sensor 1330 may include, for example, one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, the same It may include a plurality of image sensors having a property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 1330 may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

In response to the movement of the camera module 1280 or the electronic device 1201 including the same, the image stabilizer 1340 moves at least one lens or the image sensor 1330 included in the lens assembly 1310 in a specific direction or Operation characteristics of the image sensor 1330 may be controlled (eg, read-out timing may be adjusted, etc.). This makes it possible to compensate for at least some of the negative effects of the movement on the image being taken. According to an embodiment, the image stabilizer 1340 is, according to an embodiment, the image stabilizer 1340 is a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 1280 . Such a movement of the camera module 1280 or the electronic device 1201 may be detected using . According to an embodiment, the image stabilizer 1340 may be implemented as, for example, an optical image stabilizer. The memory 1350 may temporarily store at least a portion of the image acquired through the image sensor 1330 for a next image processing operation. For example, when image acquisition is delayed according to the shutter or a plurality of images are acquired at high speed, the acquired original image (eg, Bayer-patterned image or high-resolution image) is stored in the memory 1350 and , a copy image corresponding thereto (eg, a low-resolution image) may be previewed through the display device 1260 . Thereafter, when a specified condition is satisfied (eg, a user input or a system command), at least a portion of the original image stored in the memory 1350 may be obtained and processed by, for example, the image signal processor 1360 . According to an embodiment, the memory 1350 may be configured as at least a part of the memory 1230 or as a separate memory operated independently of the memory 1230 .

The image signal processor 1360 may perform one or more image processing on an image acquired through the image sensor 1330 or an image stored in the memory 1350 . The one or more image processes may include, for example, depth map generation, three-dimensional modeling, panorama generation, feature point extraction, image synthesis, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment, blurring ( blurring, sharpening, or softening. Additionally or alternatively, the image signal processor 1360 may include at least one of the components included in the camera module 1280 (eg, an image sensor). 1330), for example, exposure time control, readout timing control, etc. The image processed by the image signal processor 1360 is stored back in the memory 1350 for further processing. or may be provided as an external component of the camera module 1280 (eg, the memory 1230 , the display device 1260 , the electronic device 1202 , the electronic device 1204 , or the server 1208 ). According to an example, the image signal processor 1360 may be configured as at least a part of the processor 1220 or as a separate processor operated independently of the processor 1220. The image signal processor 1360 may include the processor 1220 and a separate processor, the at least one image processed by the image signal processor 1360 may be displayed through the display device 1260 either as it is by the processor 1220 or after additional image processing.

According to an embodiment, the electronic device 1201 may include a plurality of camera modules 1280 each having different properties or functions. In this case, for example, at least one of the plurality of camera modules 1280 may be a wide-angle camera, and at least the other may be a telephoto camera. Similarly, at least one of the plurality of camera modules 1280 may be a front camera, and at least the other may be a rear camera.

An electronic device according to an embodiment of this document may include a camera, a distance sensor, a display, and at least one processor electrically connected to the camera, the distance sensor, and the display. The at least one processor may be configured to obtain an image including a subject through the camera, obtain distance data on a distance between the subject and the electronic device through the distance sensor, and defocus based on the distance data determining that de-focus has occurred, detecting a region of interest including at least a portion of the subject in the image, and cropping a region including at least the region of interest in the image, and The cropped area may be enlarged and displayed together with a message instructing the defocus on the display.

In the electronic device according to an embodiment of the present document, the at least one processor displays a first visual affordance on the display in response to detecting the ROI, and provides information about the first visual affordance. When there is a user input, a region including at least the region of interest in the image may be cropped, and the cropped region may be enlarged and displayed together with a message instructing the defocus on the display.

In the electronic device according to an embodiment of the present document, the at least one processor automatically crops a region including the region of interest in the image in response to detecting the region of interest, and displays the image on the display. The cropped area may be enlarged and displayed together with the message indicating the defocus.

In the electronic device according to an embodiment of the present document, the at least one processor displays a second visual affordance together with a message instructing the defocus on the display and an enlarged image of the cropped area, and 2 If there is a user input for visual affordance, an image that is not cropped or enlarged may be displayed on the display.

In the electronic device according to an embodiment of the present document, the at least one processor displays a second visual affordance together with a message instructing the defocus on the display and an enlarged image of the cropped area, and 2 If there is a user input for visual affordance, an image that is not cropped or enlarged may be displayed on the display.

In the electronic device according to an embodiment of this document, the at least one processor may detect the ROI through object detection.

In the electronic device according to an embodiment of the present document, the at least one processor analyzes the texture of the subject through a texture check, and includes it in the image through extract high level features The characteristics of the blurred blur can be analyzed.

In the electronic device according to an embodiment of the present document, the at least one processor may display a line surrounding the ROI on the display in response to detecting the ROI.

In the electronic device according to an embodiment of the present document, the at least one processor may display a UI for guiding a user on the display so that the region of interest is located in the center of the image.

In the method of providing an image capturing guide of an electronic device according to an embodiment of the present document, the operation of acquiring an image including a subject through a camera included in the electronic device, and the subject through a distance sensor included in the electronic device an operation of acquiring distance data on a distance between the electronic device and the electronic device, an operation of determining that defocus has occurred based on the distance data, and an operation of detecting an ROI including at least a part of the subject in the image , an operation of cropping a region including at least the region of interest in the image, and an operation of enlarging and displaying the cropped region together with a message instructing the defocus on a display included in the electronic device. may include

A method of providing an image capturing guide of an electronic device according to an embodiment of the present document includes displaying a first visual affordance on the display in response to detecting the region of interest, and adding the first visual affordance to the first visual affordance. when there is a user input for , cropping a region including at least the region of interest in the image, and enlarging and displaying the cropped region together with a message instructing the defocus on the display. .

The method of providing an image capturing guide of an electronic device according to an embodiment of the present document includes automatically cropping a region including at least the region of interest in the image in response to detecting the region of interest, and displaying the image on the display. It may include an operation of enlarging and displaying the cropped area together with a message indicating defocus.

The method of providing an image capturing guide of an electronic device according to an exemplary embodiment of the present document includes displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display; When there is a user input for the second visual affordance, the operation may include displaying an image that is not cropped or enlarged on the display.

The method of providing an image capturing guide of an electronic device according to an exemplary embodiment of the present document includes displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display; When there is a user input for the second visual affordance, the operation may include displaying an image that is not cropped or enlarged on the display.

In the method of providing an image capturing guide for an electronic device according to an embodiment of this document, the detecting of the ROI may include detecting the ROI through object detection.

An electronic device according to an embodiment of this document may include a camera, a distance sensor, a display, and at least one processor electrically connected to the camera, the distance sensor, and the display. The at least one processor may be configured to acquire an image including a subject through the camera, obtain distance data on a distance between the subject and the electronic device through the distance sensor, and defocus based on the distance data is determined to have occurred, and a message for inducing a distance between the subject and the electronic device to be greater than or equal to a minimum focal length may be displayed on the display.

In the electronic device according to an embodiment of the present document, the at least one processor may remove the message from the display in response to a distance between the subject and the electronic device being equal to or greater than a minimum focal length.

The electronic device according to an embodiment of the present document includes at least one of a speaker and a light emitting device electrically connected to the at least one processor, wherein the at least one processor displays the message on the display, At least one of the speaker and the light emitting device may be output together.

In the electronic device according to an embodiment of the present document, the at least one processor may detect a region of interest in the image through object detection.

In the electronic device according to an embodiment of the present document, the at least one processor analyzes a texture of the subject through a texture check and analyzes a characteristic of a blur included in the image through high-level feature extraction. .

Claims (20)

In an electronic device,
camera;
distance sensor;
display; and
at least one processor electrically connected to the camera, the distance sensor, and the display;
The at least one processor comprises:
Obtaining an image including a subject through the camera,
obtaining distance data on a distance between the subject and the electronic device through the distance sensor;
It is determined that de-focus has occurred based on the distance data,
detecting a region of interest including at least a portion of the subject in the image,
crop a region including at least the region of interest in the image;
and enlarging and displaying the cropped area together with a message instructing the defocus on the display. The method according to claim 1,
The at least one processor comprises:
in response to detecting the region of interest, display a first visual affordance on the display;
When there is a user input for the first visual affordance, a region including at least the region of interest in the image is cropped, and the cropped region is enlarged and displayed together with a message instructing the defocus on the display. , electronic devices. The method according to claim 1,
the at least one processor,
In response to detecting the region of interest, automatically cropping a region including at least the region of interest in the image, and enlarging and displaying the cropped region together with a message instructing the defocus on the display, electronic device. 3. The method according to claim 2,
The at least one processor comprises:
displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display;
When there is a user input for the second visual affordance, an image that is not cropped or enlarged is displayed on the display. 4. The method according to claim 3,
The at least one processor comprises:
displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display;
When there is a user input for the second visual affordance, an image that is not cropped or enlarged is displayed on the display. The method according to claim 1,
the at least one processor,
An electronic device that detects the region of interest through object detection. 7. The method of claim 6,
The at least one processor comprises:
Analyze the texture of the subject through texture check,
An electronic device for analyzing a characteristic of a blur included in the image through extract high level feature. The method according to claim 1,
the at least one processor,
In response to detecting the ROI, displaying a line surrounding the ROI on the display. The method according to claim 1,
the at least one processor,
and displaying a UI for guiding a user on the display so that the region of interest is located in the center of the image. A method of providing an image capturing guide for an electronic device, the method comprising:
acquiring an image including a subject through a camera included in the electronic device;
acquiring distance data on a distance between the subject and the electronic device through a distance sensor included in the electronic device;
determining that defocus has occurred based on the distance data;
detecting an ROI including at least a portion of the subject in the image;
cropping a region including at least the region of interest in the image; and
and displaying an enlarged display of the cropped area together with a message instructing the defocus on a display included in the electronic device. 11. The method of claim 10,
displaying a first visual affordance on the display in response to detecting the region of interest; and
When there is a user input for the first visual affordance, a region including at least the region of interest in the image is cropped, and the cropped region is enlarged and displayed together with a message instructing the defocus on the display. A method of providing an image capturing guide of an electronic device, comprising an operation. 11. The method of claim 10,
In response to detecting the region of interest, automatically cropping a region including at least the region of interest in the image, and enlarging and displaying the cropped region together with a message instructing the defocus on the display A method of providing an image capturing guide for an electronic device, comprising: 12. The method of claim 11,
displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display; and
and displaying an image that is not cropped or enlarged on the display when there is a user input for the second visual affordance. 13. The method of claim 12,
displaying a second visual affordance together with a message instructing the defocus and an enlarged image of the cropped area on the display; and
and displaying an image that is not cropped or enlarged on the display when there is a user input for the second visual affordance. 11. The method of claim 10,
The detecting of the region of interest includes detecting the region of interest through object detection. In an electronic device,
camera;
distance sensor;
display; and
at least one processor electrically connected to the camera, the distance sensor, and the display;
The at least one processor comprises:
Obtaining an image including a subject through the camera,
obtaining distance data on a distance between the subject and the electronic device through the distance sensor;
It is determined that defocus has occurred based on the distance data,
and displaying a message inducing a distance between the subject and the electronic device to be greater than or equal to a minimum focal length on the display. 17. The method of claim 16,
the at least one processor,
and in response to a distance between the subject and the electronic device being greater than or equal to a minimum focal length, removing the message from the display. 17. The method of claim 16,
At least one of a speaker and a light emitting device electrically connected to the at least one processor,
the at least one processor,
When the message is displayed on the display, the electronic device outputs at least one of the speaker and the light emitting device together. 17. The method of claim 16,
the at least one processor,
An electronic device that detects a region of interest in the image through object detection. 20. The method of claim 19,
The at least one processor comprises:
Analyze the texture of the subject through texture check,
An electronic device for analyzing a characteristic of a blur included in the image through high-level feature extraction.

Images