WO2022191407A1 - Procédé, dispositif et support de stockage pour fournir une interface utilisateur - Google Patents

Procédé, dispositif et support de stockage pour fournir une interface utilisateur Download PDF

Info

Publication number
WO2022191407A1
WO2022191407A1 PCT/KR2022/000700 KR2022000700W WO2022191407A1 WO 2022191407 A1 WO2022191407 A1 WO 2022191407A1 KR 2022000700 W KR2022000700 W KR 2022000700W WO 2022191407 A1 WO2022191407 A1 WO 2022191407A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
image
subject
electronic device
display
Prior art date
Application number
PCT/KR2022/000700
Other languages
English (en)
Korean (ko)
Inventor
전진욱
최재성
Original Assignee
삼성전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 삼성전자 주식회사 filed Critical 삼성전자 주식회사
Publication of WO2022191407A1 publication Critical patent/WO2022191407A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
    • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet

Definitions

  • Various embodiments relate to a method, apparatus and storage medium for providing a user interface.
  • a camera may be disposed in front of an electronic device having a display in order to provide a function of taking a selfie or a video call.
  • the camera which was placed on the top of the front of the electronic device, is evolving to be placed under the display in order to increase screen size efficiency and a visual immersion experience.
  • an under display camera capable of maximizing an immersive experience by disposing a camera under a display so that the camera is not visible in an electronic device has been proposed.
  • the UDC may be visible on the display screen when running, and not visible on the display screen when not running, thereby enabling the realization of a full display.
  • the UDC may have a limit on the size of the camera sensor exposed on the display due to the nature of hiding the camera sensor on the display as much as possible. Accordingly, in UDC, it may be difficult to implement an auto focus function due to the limited size of the camera sensor.
  • the auto focus function may be difficult to implement even in a conventional hole-punching type camera. Accordingly, general cameras disposed on the front of the electronic device may typically implement a fixed focus function.
  • a UDC having a fixed focus function When a UDC having a fixed focus function is used, the distance between the UDC and the subject needs to be adjusted by the user until the distance corresponds to the focal length. However, since the user cannot clearly know the focal length, it may be inconvenient to adjust the distance between the UDC and the subject.
  • a method, an apparatus, and a storage medium for providing a user interface (UI) in an electronic device may be provided.
  • the electronic device may provide a UI that allows the user to recognize the location of the UDC.
  • the electronic device may provide a UI enabling the user to recognize the location of at least one UDC to be used among the plurality of UDCs in an environment in which a plurality of UDCs are used.
  • the electronic device may provide an ideal photographing result without using the auto focus function by providing a UI for inducing distance adjustment so that the distance between the UDC and the subject corresponds to the focal length of the UDC.
  • the electronic device may provide a UI for stably and clearly setting the composition of the subject based on the learned reference model.
  • an electronic device may include a memory; display; sound output unit; at least one camera disposed under the display; and at least one processor operatively coupled to the memory, the display, and the at least one camera, wherein the at least one processor detects a distance between the at least one camera and the at least one first object; , compares the preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, and at least one of the detected distance or a comparison result of the preview image and the reference image Based on , it is determined whether a shooting condition for the at least one camera is satisfied, and when the shooting condition is satisfied, a user interface (UI) for instructing shooting through the at least one camera is displayed. It may be set to provide through one of the display or the sound output unit.
  • UI user interface
  • a method of providing a user interface (UI) in an electronic device may include detecting a distance between at least one camera disposed under a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a pre-stored reference image; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or a comparison result of the preview image and the reference image; and performing at least one of displaying a first UI for instructing shooting through the at least one camera on the display or outputting the first UI as a voice message when the shooting condition is satisfied.
  • UI user interface
  • the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor, wherein the at least one The operation may include: detecting a distance between at least one camera disposed under the display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a pre-stored reference image; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or a comparison result of the preview image and the reference image; and displaying a first user interface (UI) for instructing shooting through the at least one camera on the display or outputting the first UI as a voice message when the shooting condition is satisfied It may include an operation of performing at least one.
  • UI user interface
  • the electronic device may clearly indicate the location of the UDC to the user through the UI.
  • the electronic device may indicate the location of at least one UDC to be used among the plurality of UDCs to the user through the UI.
  • the electronic device may provide an image having a clear focus to the user by allowing the distance between the UDC and the subject to correspond to the focal length of the UDC through the UI.
  • the electronic device may provide an image of a stable composition to the user through the UI.
  • FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure
  • FIG. 2 is a block diagram illustrating a camera module according to various embodiments.
  • FIG. 3 is a block diagram of an electronic device according to various embodiments of the present disclosure.
  • FIG. 4 is a flowchart illustrating a method of providing a UI in an electronic device according to various embodiments of the present disclosure
  • FIG. 5 is a flowchart illustrating an example of a method of providing a UI in an electronic device including a UDC according to various embodiments of the present disclosure.
  • FIG. 6 is a flowchart illustrating a method of generating, by an electronic device, a reference model for each category, according to various embodiments of the present disclosure
  • FIG. 7 is a flowchart illustrating another example of a method of providing a UI by an electronic device including a UDC according to various embodiments of the present disclosure.
  • FIG. 8 is a diagram illustrating a UI for instructing a UDC in an electronic device according to various embodiments of the present disclosure
  • 9A is a flowchart illustrating an example of a method of providing a UI in an electronic device including a plurality of UDCs according to various embodiments of the present disclosure
  • 9B is a flowchart illustrating an example of a method of providing a UI in an electronic device including a plurality of UDCs according to various embodiments of the present disclosure
  • FIG. 10 is a diagram illustrating a UI for indicating an activated UDC among a plurality of UDCs in an electronic device according to various embodiments of the present disclosure
  • FIG. 1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments of the present disclosure.
  • an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with at least one of the electronic device 104 and the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 .
  • a first network 198 eg, a short-range wireless communication network
  • a second network 199 e.g., a second network 199
  • the electronic device 101 may communicate with the electronic device 104 through the server 108 .
  • the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 .
  • at least one of these components eg, the connection terminal 178
  • some of these components are integrated into one component (eg, display module 160 ). can be
  • the processor 120 for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in , process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 .
  • software eg, a program 140
  • the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 .
  • the volatile memory 132 may be stored in , process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 .
  • the processor 120 is a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor).
  • a main processor 121 eg, a central processing unit or an application processor
  • a secondary processor 123 eg, a graphic processing unit, a neural network processing unit
  • NPU neural processing unit
  • an image signal processor e.g., a sensor hub processor, or a communication processor.
  • the secondary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states.
  • the coprocessor 123 eg, an image signal processor or a communication processor
  • may be implemented as part of another functionally related component eg, the camera module 180 or the communication module 190. have.
  • the auxiliary processor 123 may include a hardware structure specialized for processing an artificial intelligence model.
  • Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108).
  • the learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited
  • the artificial intelligence model may include a plurality of artificial neural network layers.
  • Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example.
  • the artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.
  • the memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 .
  • the data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto.
  • the memory 130 may include a volatile memory 132 or a non-volatile memory 134 .
  • the program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .
  • the input module 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 .
  • the input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).
  • the sound output module 155 may output a sound signal to the outside of the electronic device 101 .
  • the sound output module 155 may include, for example, a speaker or a receiver.
  • the speaker can be used for general purposes such as multimedia playback or recording playback.
  • the receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.
  • the display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 .
  • the display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device.
  • the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.
  • the audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 or an external electronic device (eg, a sound output module 155 ) directly or wirelessly connected to the electronic device 101 .
  • the electronic device 102) eg, a speaker or headphones
  • the sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do.
  • the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.
  • the interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ).
  • the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
  • the connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ).
  • the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).
  • the haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense.
  • the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
  • the camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.
  • the power management module 188 may manage power supplied to the electronic device 101 .
  • the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).
  • PMIC power management integrated circuit
  • the battery 189 may supply power to at least one component of the electronic device 101 .
  • battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.
  • the communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel.
  • the communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication.
  • the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a local area network (LAN) communication module, or a power line communication module).
  • a wireless communication module 192 eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module
  • GNSS global navigation satellite system
  • wired communication module 194 eg, : It may include a local area network (LAN) communication module, or a power line communication module.
  • a corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN).
  • a first network 198 eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)
  • a second network 199 eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN).
  • a telecommunication network
  • the wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 .
  • subscriber information eg, International Mobile Subscriber Identifier (IMSI)
  • IMSI International Mobile Subscriber Identifier
  • the electronic device 101 may be identified or authenticated.
  • the wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR).
  • NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)).
  • eMBB enhanced mobile broadband
  • mMTC massive machine type communications
  • URLLC ultra-reliable and low-latency
  • the wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example.
  • a high frequency band eg, mmWave band
  • the wireless communication module 192 uses various techniques for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna.
  • the wireless communication module 192 may support various requirements defined in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ).
  • the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: Downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) can be supported.
  • a peak data rate eg, 20 Gbps or more
  • loss coverage eg, 164 dB or less
  • U-plane latency for realizing URLLC
  • the antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device).
  • the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern.
  • the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna.
  • other components eg, a radio frequency integrated circuit (RFIC)
  • RFIC radio frequency integrated circuit
  • the antenna module 197 may form a mmWave antenna module.
  • the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, underside) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.
  • peripheral devices eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)
  • GPIO general purpose input and output
  • SPI serial peripheral interface
  • MIPI mobile industry processor interface
  • the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 .
  • Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 .
  • all or part of the operations performed by the electronic device 101 may be executed by one or more external electronic devices 102 , 104 , or 108 .
  • the electronic device 101 may perform the function or service itself instead of executing the function or service itself.
  • one or more external electronic devices may be requested to perform at least a part of the function or the service.
  • One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 .
  • the electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request.
  • cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used.
  • the electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing.
  • the external electronic device 104 may include an Internet of things (IoT) device.
  • the server 108 may be an intelligent server using machine learning and/or neural networks.
  • the external electronic device 104 or the server 108 may be included in the second network 199 .
  • the electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.
  • the camera module 201 includes a lens assembly 210 , a flash 220 , an image sensor 230 , an image stabilizer 240 , a memory 250 (eg, a buffer memory), or an image signal processor. (image signal processor: ISP, 260) may be included.
  • the lens assembly 210 may collect light emitted from a subject, which is an image to be captured.
  • the lens assembly 210 may include one or more lenses.
  • the camera module 201 may include a plurality of lens assemblies 210 .
  • the camera module 201 may form, for example, a dual camera, a 360 degree camera, or a spherical camera.
  • Some of the plurality of lens assemblies 210 may have the same lens properties (eg, angle of view, focal length, auto focus, f number, or optical zoom), or at least one lens assembly may be a different lens assembly. It may have one or more lens properties that are different from the lens properties of .
  • the lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.
  • the flash 220 may emit light used to enhance light emitted or reflected from the subject.
  • the flash 220 may include one or more light emitting diodes (eg, a red-green-blue (RGB) LED, a white LED, an infrared LED, or an ultraviolet LED), or a xenon lamp.
  • the image sensor 230 may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly 210 into an electrical signal.
  • the image sensor 230 may include, for example, one image sensor selected from image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor; It may include a plurality of image sensors having the same property or a plurality of image sensors having different properties.
  • Each image sensor included in the image sensor 230 may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.
  • CCD charged coupled device
  • CMOS complementary metal oxide semiconductor
  • the image stabilizer 240 moves at least one lens or the image sensor 230 included in the lens assembly 210 in a specific direction or Operation characteristics of the image sensor 230 may be controlled (eg, read-out timing may be adjusted, etc.). This makes it possible to compensate for at least some of the negative effects of the movement on the image being taken.
  • the image stabilizer 240 uses a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module 201 to the camera module 201 or electronic device. (201) can detect such a movement.
  • the image stabilizer 240 may be implemented as, for example, an optical image stabilizer.
  • the memory 250 may temporarily store at least a portion of the image acquired through the image sensor 230 for a next image processing operation. For example, when image acquisition is delayed according to the shutter or a plurality of images are acquired at high speed, the acquired original image (eg, a Bayer-patterned image or a high-resolution image) is stored in the memory 250 and , a copy image corresponding thereto (eg, a low-resolution image) may be previewed through the display module 160 . Thereafter, when a specified condition is satisfied (eg, a user input or a system command), at least a portion of the original image stored in the memory 250 may be obtained and processed by, for example, the image signal processor 260 . According to various embodiments, the memory 250 may be configured as at least a part of the memory 130 or as a separate memory operated independently of the memory 130 .
  • the image signal processor 260 may perform one or more image processing on an image acquired through the image sensor 230 or an image stored in the memory 250 .
  • the one or more image processes may include, for example, depth map generation, three-dimensional modeling, panorama generation, feature point extraction, image synthesis, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment, blurring ( blurring, sharpening, or softening.
  • the image signal processor 260 may include at least one of the components included in the camera module 201 (eg, an image sensor). 230), for example, exposure time control, readout timing control, etc.
  • the image processed by the image signal processor 260 is stored back in the memory 250 for further processing.
  • the image signal processor 260 may be configured as at least a part of the processor 120 or may be configured as a separate processor operated independently of the processor 120.
  • the image signal processor 260 may be configured as the processor 120 ) and a separate processor, at least one image processed by the image signal processor 260 may be displayed through the display device 260 as it is by the processor 120 or after additional image processing. .
  • the electronic device 101 may include a plurality of camera modules 201 each having different properties or functions.
  • at least one of the plurality of camera modules 201 may be a wide-angle camera, and at least the other may be a telephoto camera.
  • at least one of the plurality of camera modules 201 may be a front camera, and at least the other may be a rear camera.
  • FIG. 3 is a block diagram 300 of an electronic device according to various embodiments.
  • the electronic device 301 includes a display 310 , a first camera 312 , a second camera 314 , a memory 320 , a sound output unit 330 , and a processor 340 . can do.
  • the electronic device 301 is not limited to the configurations illustrated in FIG. 3 , and may be implemented to include more or fewer configurations.
  • the electronic device 301 may be implemented to further include the components described above with reference to FIG. 1 or FIG. 2 .
  • the electronic device 301 according to various embodiments may be the electronic device 101 of FIG. 1 or an electronic device to be described later.
  • display 310 displays at least one of information generated by processor 340 , images collected by at least one of first camera 312 or second camera 314 , or a message to provide to a user. one can be displayed.
  • the information displayed by the display 310 is not limited to the above-described example.
  • the display 310 may perform a display operation by controlling pixels on the display 310 based on at least one of a passive matrix driving method or an active matrix driving method.
  • the display 310 may include a first pixel area disposed at a location corresponding to the first camera 312 or a second pixel area disposed at a location corresponding to the second camera 314 .
  • At least one pixel included in the first pixel area may be deactivated when the first camera 312 is executed, and may be activated for display when the first camera 312 is not executed.
  • At least one pixel included in the second pixel area may be deactivated when the second camera 314 is executed, and may be activated for display when the second camera 312 is not executed.
  • the display 310 may be included in the display module 160 of FIG. 1 .
  • the first camera 312 and the second camera 314 may be UDCs disposed under the display 310 . In one embodiment, the first camera 312 and the second camera 314 may be disposed at different locations.
  • the first camera 312 may be disposed on the top of the display 310 when the display 310 is viewed from the front, and the second camera 314 may be disposed on the bottom of the display 310 .
  • the upper end of the display 310 refers to an end relatively close to the sound output unit 330 (or microphone), and the lower end of the display 310 refers to an end relatively far to the sound output unit 330 (or microphone). can do.
  • the first camera 312 may be disposed on the right side of the display 310 when the display 310 is viewed from the front, and the second camera 314 may be disposed on the left side of the display 310 .
  • the positions of the first camera 312 and the second camera 314 are not limited to the above-described examples and may be variously changed.
  • the electronic device 301 includes only one camera.
  • a camera or more than two cameras may be included.
  • At least one of the first camera 312 and the second camera 314 may be included in the camera module 180 of FIG. 1 or the camera module 201 of FIG. 2 .
  • the memory 320 may store information generated by the processor 340 , information for a face authentication operation, a command, and/or an indicator.
  • the memory 320 may store an image collected by at least one of the first camera 312 and the second camera 314 .
  • the memory 320 may store images collected by at least one of the first camera 312 and the second camera 314 for each category based on the image classification model.
  • a category may include a location where the image was taken (eg, indoor, outdoor, landmark, or local name), the time the image was taken (eg, morning, afternoon, or season), and the type of subject included in the image (eg, people). , food, landscape, animal, or object), or the number of subjects included in the image.
  • the memory 320 may store one of an image having a composition most frequently used for each category or a user preferred image selected by a user for each category as a reference image.
  • the memory 320 may include a volatile memory or a non-volatile memory, and may be included in the memory 130 of FIG. 1 .
  • the sound output unit 330 may output a message related to a user interface (UI) by voice.
  • the sound output unit 330 may include a voice message for instructing at least one of the first camera 312 or the second camera 314, or between the first camera 312 or the second camera 314 and the subject.
  • At least one of a voice message for adjusting the distance or a voice message for instructing shooting through at least one of the first camera 312 and the second camera 314 may be output to the outside of the electronic device 301 . .
  • the sound output unit 330 may be included in the sound output module 155 of FIG. 1 .
  • the processor 340 controls the display 310 , the first camera 312 , the second camera 314 , and the memory 320 to control the first camera 312 or the second camera 314 .
  • ) may be provided through at least one of the display 310 and the sound output unit 330 .
  • the processor 340 may perform operations performed in an electronic device according to various embodiments to be described later.
  • the processor 340 may include one or more processors and may be included in the processor 120 of FIG. 1 .
  • an electronic device may include a memory; display; sound output unit; at least one camera disposed under the display; and at least one processor operatively coupled to the memory, the display, and the at least one camera, wherein the at least one processor detects a distance between the at least one camera and the at least one first object; , compares the preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, and at least one of the detected distance or a comparison result of the preview image and the reference image Based on , it is determined whether a shooting condition for the at least one camera is satisfied, and when the shooting condition is satisfied, a user interface (UI) for instructing shooting through the at least one camera is displayed. It may be set to provide through one of the display or the sound output unit.
  • UI user interface
  • the at least one processor may include distance information for each size of a subject in the image stored in the memory, and a signal output from a sensor disposed under the display reflected by the at least one subject and input to the sensor.
  • the at least one camera and It may be set to detect a distance between the at least one first subject.
  • the at least one processor determines whether the detected distance corresponds to a focal length set for the at least one camera, and if the detected distance does not correspond to the focal length, the at least one It may be set to provide a UI for instructing to adjust the distance between the camera and the at least one first subject through at least one of the display and the sound output unit.
  • the at least one processor is configured to: As a result of comparing the preview image and the reference image, the composition of the at least one first object in the preview image is determined to be the composition of the at least one second object in the reference image. If not, a UI instructing to change the shooting angle of the at least one camera so that the composition of the at least one first object corresponds to the composition of the at least one second object is displayed on the display or the sound output unit It may be set to provide through at least one.
  • the at least one processor is configured to: determine that the detected distance corresponds to a focal length set for the at least one camera, and, as a result of comparing the preview image and the reference image, the at least one in the preview image When the composition of the first subject corresponds to the second composition of the at least one subject in the reference image, it may be set to determine that a photographing condition for the at least one camera is satisfied.
  • the at least one processor is configured to: When a shooting condition for the at least one camera is satisfied and the number of the at least one camera is plural, the location of the at least one first subject in the preview image; Detect at least one of the gaze directions of the first subject, and select at least one camera to collect an image of the at least one first subject from among a plurality of cameras based on at least one of the detected position and the gaze direction It may be set to provide a UI for making a selection and instructing the selected at least one camera through at least one of the display or the sound output unit.
  • the at least one processor stores, as the reference image, a user preferred image selected by a user from among at least one image collected by the at least one camera in the memory, or in the at least one camera It may be configured to analyze at least one image collected by the controller to identify an image having the most frequently used composition information, and to store the identified image as the reference image in the memory.
  • Operations of the electronic device according to various embodiments to be described with reference to FIGS. 4 to 10 include an electronic device (eg, the electronic device 101 of FIG. 1 or the electronic device 301 of FIG. 3 ) and at least one processor (eg: It may be performed by the processor 120 of FIG. 1 or the processor 340 of FIG. 3 ), or a combination thereof.
  • an electronic device eg, the electronic device 101 of FIG. 1 or the electronic device 301 of FIG. 3
  • at least one processor eg: It may be performed by the processor 120 of FIG. 1 or the processor 340 of FIG. 3
  • FIGS. 4 to 10 are performed by an electronic device.
  • At least one camera, a plurality of cameras, UDC, or a plurality of UDCs to be described in FIGS. 4 to 10 may be included in the camera module 180 of FIG. 1 and the camera module 201 of FIG. 2 , respectively, and FIG. 3 It may be the first camera 312 or the second camera 314 of Also, the display to be described with reference to FIGS. 4 to 10 may be included in the display module 160 of FIG. 1 or the display 310 of FIG. 3 .
  • FIG. 4 is a flowchart 400 illustrating a method of providing a UI in an electronic device according to various embodiments of the present disclosure.
  • the electronic device may detect a distance between at least one camera and at least one first subject.
  • the electronic device may compare a preview image of at least one first subject collected by at least one camera with a stored reference image.
  • the electronic device may determine whether a photographing condition for at least one camera is satisfied based on at least one of a detected distance or a comparison result between the preview image and the reference image.
  • the electronic device may display a UI for instructing at least one camera on the display or output a message related to the UI as a voice.
  • one camera or a plurality of cameras may be used in the electronic device.
  • a UI providing method may include a UI providing method when one camera is used in an electronic device and a UI providing method when a plurality of cameras are used in the electronic device.
  • FIGS. 5 to 8 a method of providing a UI when one camera is used in an electronic device according to various embodiments will be described with reference to FIGS. 5 to 8 , and a case in which a plurality of cameras are used in the electronic device according to various embodiments of the present invention will be described below.
  • a UI providing method will be described with reference to FIGS. 9A to 10 .
  • a case in which one camera or a plurality of cameras are UDCs will be described as an example.
  • FIG. 5 is a flowchart 500 illustrating an example of a method of providing a UI in an electronic device including a UDC according to various embodiments.
  • the electronic device may determine the occurrence of a UDC execution event.
  • the UDC execution event is an event for the execution of the UDC, which is a user's action with respect to the electronic device (eg, a user holding the electronic device in a hand and lifting or tilting it), a touch input on the screen of the electronic device, and the user may be generated based on at least one of a voice input of , or an input of a physical key.
  • the electronic device may execute the UDC based on the generated UDC execution event and display a UI for instructing the executed UDC on the display.
  • the UI for indicating the executed UDC may be provided using at least one pixel among surrounding pixels of the display pixel area disposed at a position corresponding to the executed UDC.
  • at least one pixel may provide a UI by being activated for a set time or until a specific event occurs (eg, until a user completes shooting through UDC).
  • at least one pixel may provide a UI by periodically repeating an activation operation and a deactivation operation.
  • the activation operation of the at least one pixel may indicate an operation in which an electrical signal is input to the at least one pixel and the at least one pixel emits light (a pixel-on operation), and the deactivation operation of the at least one pixel is applied to the at least one pixel. It may represent an operation in which at least one pixel does not emit light (a pixel off operation) because no electrical signal is input.
  • the electronic device may measure a distance to the subject. For example, the electronic device measures the distance to the subject based on the time it takes for a signal (eg, an infrared signal or a laser signal) output from a sensor disposed below the display to be reflected by the subject and input to the sensor. can do.
  • the sensor may be, for example, an infrared sensor disposed at a different location from the UDC or a time-of-flight (TOF) sensor.
  • TOF time-of-flight
  • the electronic device instead of measuring the distance to the subject, the electronic device checks the size of the subject in the preview image collected by the UCD, and determines the distance corresponding to the confirmed size of the subject, which is stored in advance by the distance by size of the subject. can be identified based on information.
  • the electronic device may identify a focal length of the UDC.
  • UDC may support the function of fixed focus.
  • the fixed focus function may indicate a function of focusing on a subject photographed by the UDC when the distance between the UDC and the subject corresponds to a set distance, that is, a focal length.
  • the electronic device may determine whether the measured distance corresponds to the focal length of the UDC. If the measured distance does not correspond to the focal length of the UDC, in operation 512 , the electronic device may provide a UI for instructing the distance adjustment. In an embodiment, when the distance between the UDC and the subject is greater than or equal to the focal length, the electronic device may provide at least one of a voice message saying “reduce the distance from the camera a little more” or a UI on the display for instructing the UDC .
  • the UI on the display for instructing the UDC may be, for example, a UI in which peripheral pixels of a display pixel area disposed at a position corresponding to the UDC are activated in the order from the outermost pixel to the innermost pixel.
  • the electronic device may provide at least one of a voice message “Increase the distance from the camera a little more” or a UI on the display for instructing the UDC.
  • the UI on the display for instructing the UDC may be, for example, a UI in which peripheral pixels of a display pixel area disposed at a position corresponding to the UDC are activated in the order from the innermost pixel to the outermost pixel.
  • the electronic device determines that the subject is in focus, and provides a UI for instructing photographing in operation 514 .
  • the UI for instructing shooting may provide at least one of a voice message “Please start shooting” or a UI on the display for instructing the UDC.
  • the UI on the display for instructing shooting may be the same as or different from the UI provided in operation 504 .
  • the user can easily recognize the location of the UDC and focus based on the UI in the UDC where it is difficult to support the auto-focus function, so that a better quality image can be obtained. can be obtained
  • FIG. 6 is a flowchart 600 illustrating a method for an electronic device to generate a reference model for each category according to various embodiments of the present disclosure.
  • the electronic device may collect and store images.
  • the image collection and storage may refer to an operation of collecting an image through the executed UDC and storing the collected image based on a user's selection when the UDC is executed based on the occurrence of the UDC execution event. have.
  • the electronic device may classify images by category based on the image classification model.
  • the image classification model may be generated by training based on a data set including images stored in the electronic device (eg, images collected by UDC) and data labeled for each image. have.
  • the data labeled for each image may include the location where the image was taken (eg, indoor, outdoor, landmark, or local name), the time the image was taken (eg, morning, afternoon, or season), and the number of subjects included in the image. It may relate to at least one of a type (eg, a person, a food, a landscape, an animal, or an object), or the number of subjects included in the image.
  • the data labeled on each image may be input by the user or based on at least one of location information obtained from GPS, time information set during UCD shooting, or image analysis operation based on big data obtained from the Internet. have.
  • a category of each image may be determined based on data labeled for each image.
  • the category of each image may be related to at least one of an image capturing location, an image capturing time, a type of subject included in the image, or the number of subjects included in the image.
  • the electronic device may generate a reference image by learning composition information for each category.
  • the electronic device learns composition information for each category related to at least one of an image capturing location, an image capturing time, a type of a subject included in an image, or the number of subjects included in an image, and uses the most frequently used composition information for each category.
  • One of the images having the selected composition or the user's preferred images selected by the user for each category may be determined as the reference image.
  • the determined reference image may be used to provide a UI as shown in FIG. 7 .
  • FIG. 7 is a flowchart 700 illustrating another example of a method of providing a UI by an electronic device including a UDC according to various embodiments of the present disclosure.
  • the electronic device may determine the occurrence of a UDC execution event.
  • the UDC execution event is a user's action on the electronic device (eg, a user's hand holding the electronic device, lifting or tilting), a touch input on the screen of the electronic device, a user's voice input, or a physical key input. It may be generated based on at least one of
  • the electronic device may execute the UDC based on the generated UDC execution event and display a preview image.
  • the electronic device may identify a reference image related to the preview image.
  • the electronic device may identify a category for the preview image based on the image classification model, and identify a reference image corresponding to the identified category.
  • the category for the preview image may be identified, for example, based on at least one of location information obtained from GPS for the preview image, time information at which the preview image was acquired, or information about a subject included in the preview image.
  • the information on the subject included in the preview image may be obtained, for example, based on at least one of an image analysis operation based on big data obtained from the Internet, a face recognition technology, and an eye tracking technology.
  • the electronic device may compare the preview image with the reference image to determine the difference in composition of the subject.
  • the difference in the composition of the subject may indicate a difference between the composition of the subject included in the preview image and the composition of the subject included in the reference image.
  • the face angle may be detected based on the face recognition technology, and the gaze direction of the person may be detected based on the eye tracking technology.
  • the electronic device may provide a UI for instructing to change the shooting angle.
  • the subject is a human face
  • the composition of the face included in the preview image is tilted to the right compared to the composition of the face included in the reference image, and the gaze direction is upward, “tilting the head to the left
  • a voice message such as “Please look straight ahead” may be output.
  • the surrounding pixels of the display pixel area disposed at the position corresponding to the UDC periodically repeat the activation operation and the deactivation operation to additionally provide a UI in the form of a blinking of the UDC periphery.
  • the electronic device may measure a distance to the subject and identify a focal length of the UDC. According to an embodiment, when the determined difference in composition is less than a threshold value, the electronic device may provide a UI for instructing the user to take a picture instead of performing operation 714 .
  • the electronic device may determine whether the measured distance corresponds to the focal length of the UDC. When the distance to the subject does not correspond to the focal length of the UDC, the electronic device may provide a UI for instructing to adjust the distance in operation 718 .
  • the electronic device may provide at least one of a voice message saying “reduce the distance from the camera a little more” or a UI on the display for instructing the UDC .
  • the UI on the display for instructing the UDC may be, for example, a UI in which peripheral pixels of a display pixel area disposed at a position corresponding to the UDC are activated in the order from the outermost pixel to the innermost pixel.
  • the electronic device may provide at least one of a voice message “Increase the distance from the camera a little more” or a UI on the display for instructing the UDC.
  • the UI on the display for instructing the UDC may be, for example, a UI in which peripheral pixels of a display pixel area disposed at a position corresponding to the UDC are activated in the order from the innermost pixel to the outermost pixel.
  • the electronic device may provide a UI for instructing the user to photograph.
  • the UI for instructing shooting may provide at least one of a voice message “Please start shooting” or a UI on the display for instructing the UDC.
  • the UI on the display for instructing shooting may be the same as or different from the UI provided in operation 704 .
  • FIG. 8 is a diagram 800 illustrating a UI for instructing a UDC in an electronic device according to various embodiments of the present disclosure.
  • a display 807 may be disposed on the front surface of the electronic device 801 , and a UDC 805 may be disposed under the display 807 .
  • the UDC peripheral UI 803 may be provided in a form in which peripheral pixels of a display pixel area disposed at a position corresponding to the UDC 805 are activated.
  • the peripheral pixels may be changed in real time as a pixel-on operation and a pixel-off operation are repeatedly performed in order.
  • peripheral pixels may be activated by being pixel-on for a set time period or until a specific event occurs (eg, until a user completes shooting through UDC).
  • the UDC peripheral UI 803 is not limited to the above-described examples and may be changed in various forms.
  • 9A and 9B are flowcharts 900 illustrating an example of a method of providing a UI in an electronic device including a plurality of UDCs according to various embodiments.
  • the electronic device may determine the occurrence of a UDC execution event.
  • the UDC execution event is a user's action on the electronic device (eg, a user's hand holding the electronic device, lifting or tilting), a touch input on the screen of the electronic device, a user's voice input, or a physical key input.
  • the electronic device may execute at least one of the plurality of UDCs based on the UDC execution event and display a first preview image.
  • the first preview image may be an image collected by at least one executed UDC.
  • the electronic device may identify a first reference image related to the first preview image.
  • the electronic device may identify a category for the first preview image based on the image classification model, and identify a first reference image corresponding to the identified category.
  • the electronic device may select at least one of the plurality of UDCs based on the first reference image.
  • the at least one UDC selected in operation 908 may be the same as or different from the at least one UDC executed in operation 904 .
  • the electronic device may select at least one of the plurality of UDCs based on at least one of a position or an angle of the subject in the first reference image. For example, when the first reference image is displayed on the display, the electronic device may select the UDC disposed under pixels located closest to the subject in the first reference image from among the plurality of UDCs.
  • the electronic device may select a UDC located in a direction toward which the person's gaze faces from among the plurality of UDCs based on the angle of the person's face.
  • a person's face angle detects facial landmarks (eg, eyes, nose, mouth, or ears) based on face recognition technology, and based on the relative size or position between the detected landmarks, the user turns the face
  • At least one of leaning, bowing, or tilting may be identified. For example, when the distance between two eyes detected from the face is shorter than a set value and the right eye is larger than the left eye, it may be identified that the face is turned to the left.
  • the electronic device may acquire a second preview image of at least one subject by using the selected at least one UDC.
  • the electronic device may identify a second reference image related to the second preview image. In operation 914 , the electronic device may determine whether to continue using the selected at least one UDC based on the identified second reference image.
  • the electronic device may determine whether to continue using the selected at least one UDC based on at least one of a position or an angle of the subject in the second reference image.
  • the electronic device may select at least one UDC in which at least one UDC disposed under pixels located closest to the subject in the second reference image from among the plurality of UDCs is selected. In response to , it may be determined to continue using the selected at least one UDC. When the at least one UDC disposed under pixels closest to the subject in the second reference image does not correspond to the selected at least one UDC, the electronic device may determine not to continue using the selected at least one UDC. .
  • the electronic device may determine to continue using the selected at least one UDC.
  • the electronic device may determine not to continue using the selected at least one UDC.
  • the electronic device re-performs operation 908 when it is determined not to continue using the selected at least one UDC, and performs operation 916 of FIG. 9B indicated by “A” when it is determined to continue using the selected at least one UDC. can be done
  • the electronic device compares the second preview image with the second reference image to determine the difference between the composition of the subject included in the second preview image and the composition of the subject included in the second reference image. can judge
  • the electronic device may provide a UI for instructing to change the shooting angle.
  • the electronic device may perform operation 920 and again perform operations 910 and lower in FIG. 9A indicated by “B”.
  • the electronic device may measure a distance to at least one subject and identify a focal length of the selected at least one UDC.
  • the electronic device may provide a UI for instructing the user to take a picture instead of performing operation 922 .
  • the electronic device may provide a UI for instructing to adjust the distance.
  • the electronic device may provide a UI for instructing the user to photograph.
  • an image of better quality may be obtained based on context conditions such as a distance or angle between each UDC and a subject. Users can be encouraged to use a specific UDC that allows
  • FIG. 10 is a diagram 1000 illustrating a UI for indicating an activated UDC among a plurality of UDCs in an electronic device according to various embodiments of the present disclosure.
  • a display 1007 may be disposed on the front surface of the electronic device 1001 , and a plurality of UDCs 1003 and 1007 may be disposed under the display 1007 . .
  • the preview image of the subject 1011 collected by the first UDC 1007 and the first UDC 1007 are performed.
  • a UDC peripheral UI 1005 for instructing the execution of may be displayed on the display 1009 .
  • the UDC peripheral UI 1005 may be provided in a form in which peripheral pixels of a display pixel area disposed at a position corresponding to the first UDC 1007 are activated.
  • the peripheral pixels may be changed in real time as a pixel-on operation and a pixel-off operation are repeatedly performed in order.
  • peripheral pixels may be activated by being pixel-on for a set time period or until a specific event occurs (eg, until a user completes shooting through UDC).
  • the UDC peripheral UI 1005 is not limited to the above-described examples and may be changed in various forms.
  • the remaining UDCs other than the first UDC 1007 among the plurality of UDCs 1003 and 1005, for example, the second UDC 1003 are not executed while the first UDC 1007 is being executed.
  • pixels in the display pixel area disposed at positions corresponding to the second UDC 1003 may be activated to display a preview image of the subject 1011 collected by the first UDC 1007 .
  • a method of providing a user interface (UI) in an electronic device may include detecting a distance between at least one camera disposed under a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a pre-stored reference image; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or a comparison result of the preview image and the reference image; and performing at least one of displaying a first UI for instructing shooting through the at least one camera on the display or outputting the first UI as a voice message when the shooting condition is satisfied.
  • UI user interface
  • the detecting of the distance includes: distance information for each size of the subject in the image stored in the memory, and a signal output from a sensor disposed under the display is reflected by the at least one subject to the sensor Based on at least one of the time taken until input or disparity information of the stereoscopic images of the at least one subject acquired by the plurality of cameras when the at least one camera is plural, the at least one camera and detecting a distance between the at least one first subject.
  • the method may further include performing at least one of outputting a UI as the voice message.
  • the at least Displaying a third UI on the display for instructing to change the shooting angle of the at least one camera so that the composition of one first subject corresponds to the composition of the at least one second object or the third UI may further include performing at least one of outputting the voice message.
  • the determining whether the shooting condition is satisfied may include determining whether the detected distance corresponds to a focal length set for the at least one camera, and a comparison result of the preview image and the reference image, the preview and determining that a shooting condition for the at least one camera is satisfied when the composition of the at least one first subject in the image corresponds to the second composition of the at least one subject in the reference image .
  • the shooting condition for the at least one camera is satisfied and the number of the at least one camera is plural, at least one of a position or a gaze direction of the at least one first subject in the preview image is detected selecting at least one camera for collecting an image of the at least one first object from among a plurality of cameras based on at least one of the detected position and the gaze direction, and the selected at least one
  • the method may further include performing at least one of displaying a fourth UI for instructing a camera on the display or outputting the fourth UI as the voice message.
  • the method may further include analyzing an image having the most frequently used composition information by analyzing and storing the identified image as the reference image.
  • the electronic device may have various types of devices.
  • the electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device.
  • a portable communication device eg, a smart phone
  • a computer device e.g., a smart phone
  • a portable multimedia device e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a wearable device e.g., a smart bracelet
  • a home appliance device e.g., a home appliance
  • first, second, or first or second may simply be used to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is “coupled” or “connected” to another (eg, second) component, with or without the terms “functionally” or “communicatively”. When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.
  • module used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit.
  • a module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions.
  • the module may be implemented in the form of an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101).
  • a storage medium eg, internal memory 136 or external memory 138
  • the processor eg, the processor 120
  • the device eg, the electronic device 101
  • the one or more instructions may include code generated by a compiler or code executable by an interpreter.
  • the device-readable storage medium may be provided in the form of a non-transitory storage medium.
  • 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is different from the case where data is semi-permanently stored in the storage medium. It does not distinguish between temporary storage cases.
  • a signal eg, electromagnetic wave
  • the method according to various embodiments disclosed in this document may be provided in a computer program product (computer program product).
  • Computer program products may be traded between sellers and buyers as commodities.
  • the computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play StoreTM) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly, online between smartphones (eg: smartphones).
  • a portion of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.
  • each component eg, a module or a program of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have.
  • one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added.
  • a plurality of components eg, a module or a program
  • the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. .
  • operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. , or one or more other operations may be added.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Input Circuits Of Receivers And Coupling Of Receivers And Audio Equipment (AREA)
  • Studio Devices (AREA)

Abstract

Un dispositif électronique, selon divers modes de réalisation, comprend : une mémoire ; un écran ; une unité de sortie de son ; au moins une caméra disposée sous l'écran ; et au moins un processeur couplé fonctionnellement à la mémoire, à l'écran et à la ou aux caméras, le ou les processeurs pouvant être conçus pour : détecter une distance entre la ou les caméras et le ou les premiers objets ; comparer une image de prévisualisation pour le ou les premiers sujets collectés par la ou les caméras avec une image de référence stockée dans la mémoire ; déterminer si une condition de photographie pour la ou les caméras est satisfaite, sur la base de la distance détectée et/ou du résultat de comparaison de l'image de prévisualisation et de l'image de référence ; et, lorsque la condition de photographie est satisfaite, fournir, à travers l'écran ou l'unité de sortie de son, une interface utilisateur (UI) pour donner l'instruction de photographier à l'aide de la ou des caméras. D'autres modes de réalisation peuvent être possibles.
PCT/KR2022/000700 2021-03-12 2022-01-14 Procédé, dispositif et support de stockage pour fournir une interface utilisateur WO2022191407A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2021-0032412 2021-03-12
KR1020210032412A KR20220127967A (ko) 2021-03-12 2021-03-12 사용자 인터페이스를 제공하기 위한 방법, 장치 및 저장 매체

Publications (1)

Publication Number Publication Date
WO2022191407A1 true WO2022191407A1 (fr) 2022-09-15

Family

ID=83226903

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/000700 WO2022191407A1 (fr) 2021-03-12 2022-01-14 Procédé, dispositif et support de stockage pour fournir une interface utilisateur

Country Status (2)

Country Link
KR (1) KR20220127967A (fr)
WO (1) WO2022191407A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080109519A (ko) * 2007-06-13 2008-12-17 삼성전자주식회사 영상 촬영 장치 및 촬영 방법
KR20120064545A (ko) * 2010-12-09 2012-06-19 한국전자통신연구원 시선일치 기능을 구비한 영상통화 장치 및 그 방법
KR20160043690A (ko) * 2014-10-14 2016-04-22 엘지전자 주식회사 이동 단말기 및 그 제어 방법
KR20200005060A (ko) * 2018-07-05 2020-01-15 십일번가 주식회사 가이드라인 기반 촬영 이미지를 이용한 이미지 검색 방법 및 이를 위한 장치
US10768356B1 (en) * 2019-05-10 2020-09-08 Wuhan China Star Optoelectronics Technology Co., Ltd. Panel device for under-display camera

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080109519A (ko) * 2007-06-13 2008-12-17 삼성전자주식회사 영상 촬영 장치 및 촬영 방법
KR20120064545A (ko) * 2010-12-09 2012-06-19 한국전자통신연구원 시선일치 기능을 구비한 영상통화 장치 및 그 방법
KR20160043690A (ko) * 2014-10-14 2016-04-22 엘지전자 주식회사 이동 단말기 및 그 제어 방법
KR20200005060A (ko) * 2018-07-05 2020-01-15 십일번가 주식회사 가이드라인 기반 촬영 이미지를 이용한 이미지 검색 방법 및 이를 위한 장치
US10768356B1 (en) * 2019-05-10 2020-09-08 Wuhan China Star Optoelectronics Technology Co., Ltd. Panel device for under-display camera

Also Published As

Publication number Publication date
KR20220127967A (ko) 2022-09-20

Similar Documents

Publication Publication Date Title
WO2022030855A1 (fr) Dispositif électronique et procédé permettant de générer une image par application d'un effet sur un sujet et un arrière-plan
WO2022108235A1 (fr) Procédé, appareil et support de stockage pour obtenir un obturateur lent
WO2022149654A1 (fr) Dispositif électronique pour réaliser une stabilisation d'image, et son procédé de fonctionnement
WO2022092706A1 (fr) Procédé de prise de photographie à l'aide d'une pluralité de caméras, et dispositif associé
WO2019066370A1 (fr) Dispositif électronique pour commander une caméra sur la base d'une lumière extérieure, et procédé de commande associé
WO2022235075A1 (fr) Dispositif électronique et son procédé de fonctionnement
WO2022244970A1 (fr) Procédé de capture d'image de dispositif électronique, et dispositif électronique associé
WO2022196993A1 (fr) Dispositif électronique et procédé de capture d'image au moyen d'un angle de vue d'un module d'appareil de prise de vues
WO2022191585A1 (fr) Procédé de capture d'image pour un dispositif électronique, et dispositif électronique associé
WO2022186646A1 (fr) Dispositif électronique pour une génération d'image et procédé de fonctionnement du dispositif électronique
WO2022245129A1 (fr) Procédé de suivi d'objet et appareil électronique associé
WO2021251631A1 (fr) Dispositif électronique comprenant une fonction de réglage de mise au point, et procédé associé
WO2022191407A1 (fr) Procédé, dispositif et support de stockage pour fournir une interface utilisateur
WO2022196916A1 (fr) Procédé, dispositif et support de stockage pour une authentification d'utilisateur
WO2022031050A1 (fr) Dispositif électronique et procédé de génération d'image
WO2022240186A1 (fr) Procédé de correction de distorsion d'image et dispositif électronique associé
WO2024117590A1 (fr) Dispositif électronique pour déterminer une zone de visualisation d'image, et son procédé de fonctionnement
WO2022250344A1 (fr) Dispositif électronique comprenant un capteur d'image et un capteur de vision dynamique, et son procédé de fonctionnement
WO2022025574A1 (fr) Dispositif électronique comprenant un capteur d'image et un processeur de signal d'image, et son procédé
WO2022245037A1 (fr) Dispositif électronique comprenant un capteur d'image et un capteur de vison dynamique, et son procédé de fonctionnement
WO2022092607A1 (fr) Dispositif électronique comportant un capteur d'image et procédé de fonctionnement de celui-ci
WO2022225248A1 (fr) Procédé et dispositif électronique de stabilisation d'image pendant la capture d'image
WO2022245148A1 (fr) Procédé de traitement d'image et dispositif électronique pour celui-ci
WO2022203355A1 (fr) Dispositif électronique comprenant une pluralité de caméras
WO2022231270A1 (fr) Dispositif électronique et son procédé de traitement d'image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22767304

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22767304

Country of ref document: EP

Kind code of ref document: A1