KR20240030378A - foldable electronic apparatus and control method thereof - Google Patents

Abstract

A foldable electronic device is disclosed. The electronic device generates a virtual projection surface of a curved shape based on the folding angle when the folding angle of the flexible display is identified as more than a critical angle based on data acquired through the sensor while an image is displayed on the flexible display and the flexible display. It may include one or more processors that identify a virtual projection surface, acquire a curved image by projecting an image on the virtual projection surface, and control the flexible display to display the obtained curved image.

Description

Foldable electronic device and control method thereof { foldable electronic apparatus and control method thereof }

The present disclosure relates to an electronic device and a method of controlling the same, and more specifically, to a foldable electronic device having a flexible display and a method of controlling the same.

Thanks to the development of electronic technology, various types of electronic devices are being developed and distributed. In particular, the development and distribution of smartphones or tablet PCs that users can carry are actively progressing.

Recently, the development of foldable devices that users can fold and unfold using flexible displays is actively underway.

A foldable electronic device according to an embodiment of the present disclosure includes a flexible display, a sensor, and a folding angle of the flexible display based on data acquired through the sensor while an image is displayed on the flexible display. If the folding angle is identified as being equal to or greater than the critical angle, it may include one or more processors that identify a virtual projection surface of a curved shape based on the folding angle. The one or more processors may acquire a curved image by projecting the image onto the virtual projection surface, and control the flexible display to display the obtained curved image.

According to one example, when the folding angle of the flexible display is identified as being less than the threshold angle, the one or more processors provide a guide UI that guides information about the folding angle, and the guide UI includes a visual UI and an audio UI. Alternatively, it may include at least one of a haptic UI.

According to one example, the one or more processors process the image based on the sizes of the first display area and the second display area divided based on a folding axis, and when the flexible display is folded, A plane connecting one corner of the first display area and one corner of the second display area is identified as a virtual plane, and the processed image is projected from the virtual plane to the virtual projection plane to obtain a curved image. You can.

According to one example, the one or more processors may determine the length between one corner of the first display area and one corner of the second display area when the flexible display is folded, the size of the first display area, and the second display area. 2 Identify a downscaling ratio based on the size of the display area, downscale the image based on the identified downscaling ratio, and project the downscaled image from the virtual plane to the virtual projection plane to produce a curved image. can be obtained.

According to one example, the one or more processors may create a virtual cylinder (virtual cylinder) based on the sizes of the first and second display areas divided based on a folding axis, the folding angle, and the height of the flexible display. cylinder), and an area of the virtual cylinder can be identified as the virtual projection surface.

According to one example, the one or more processors identify the center and radius of the virtual cylinder based on the first direction length of the first display area, the first direction length of the second display area, and the folding angle, The length of the flexible display in the second direction is identified as the height of the virtual cylinder, and the area between the points of the virtual cylinder corresponding to the upper point of the first display area and the upper point of the second display area is defined as the virtual cylinder. It can be identified by the projection surface.

According to one example, the sensor includes at least one of a motion sensor or a camera, and the one or more processors identify the user's gaze direction based on data acquired through at least one of the motion sensor or the camera, and , the virtual projection surface is rotated so that the angle between the user's gaze direction and the axis of the virtual projection surface becomes a preset angle, and the processed image is projected onto the rotated virtual projection surface to obtain the curved image. .

According to one example, the one or more processors rotate the virtual projection plane so that the angle formed by the user's gaze direction and the axis of the virtual projection plane is vertical, and project the processed image onto the rotated virtual projection plane to A curved image can be obtained.

According to one example, when the folding angle of the flexible display is identified as being greater than or equal to the threshold angle while a preset application is being executed, the one or more processors process the image displayed on the flexible display to obtain the curved image. And, the flexible display can be controlled to display the obtained curved image.

According to one example, when the folding angle of the flexible display is identified as being equal to or greater than the threshold angle while the preset application is executed while the electronic device is in the portrait mode, the one or more processors display the display on the flexible display. The image is processed to obtain the curved image, and when the electronic device switches from the portrait mode to the landscape mode, it is possible to identify whether the auto-rotation function is turned on. The one or more processors, when the auto-rotation function is turned on, process an image displayed on the flexible display to obtain a curved image, and when the auto-rotation function is turned off, turn on the auto-rotation function, The image displayed on the flexible display can be processed to obtain a curved image, and the flexible display can be controlled to display the obtained curved image.

A method of controlling a foldable electronic device according to an embodiment includes, while an image is displayed on a flexible display, a folding angle of the flexible display is greater than or equal to a threshold angle based on data acquired through a sensor. If identified, identifying a virtual projection surface of a curved shape based on the folding angle, acquiring a curved image by projecting the image onto the virtual projection surface, and It may include displaying a curved image on a flexible display in a folded state.

According to one embodiment, a non-transitory computer-readable medium storing computer instructions that, when executed by a processor of a foldable electronic device, cause the electronic device to perform an operation, wherein the operation is performed on data acquired through a sensor. When the folding angle of the flexible display is identified as being equal to or greater than a critical angle, identifying a curved virtual projection surface based on the folding angle; projecting the image onto the virtual projection surface; It may include acquiring a curved image and displaying the obtained curved image on a flexible display in a folded state.

1A and 1B are diagrams to explain an implementation example of an electronic device according to an embodiment.
FIG. 2A is a block diagram showing the configuration of an electronic device according to an embodiment.
FIG. 2B is a diagram illustrating a detailed configuration of an implementation example of an electronic device according to an embodiment of the present disclosure.
FIGS. 3A and 3B are diagrams for explaining a method of displaying an image in an electronic device according to an embodiment.
Figure 4 is a diagram schematically explaining an image conversion method according to an embodiment.
FIGS. 5A to 5C and FIG. 6 are diagrams for explaining a method of identifying a critical angle according to an embodiment.
Figures 7 and 8 are diagrams for explaining a method of providing a guide UI according to an embodiment.
9 and 10 are diagrams for explaining an image conversion method according to an embodiment.
11 and 12 are diagrams for explaining an image projection method according to an embodiment.
13 and 14 are diagrams for explaining an image conversion method according to an embodiment.
Figure 15 is a diagram for explaining a use case according to an embodiment.
Figure 16 is a diagram for explaining a use case according to an embodiment.
Figure 17 is a diagram for explaining a use case according to an embodiment.
Figure 18 is a diagram for explaining a use case according to an embodiment.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the art, the emergence of new technology, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

In this specification, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

The expression at least one of A or/and B should be understood as referring to either “A” or “B” or “A and B”.

As used herein, expressions such as “first,” “second,” “first,” or “second,” can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as “connected to,” it should be understood that a certain component can be connected directly to another component or connected through another component (e.g., a third component).

The expression “configured to” used in the present disclosure may mean, for example, “suitable for,” “having the capacity to,” depending on the situation. ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “consist of” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented by at least one processor (not shown), except for “modules” or “units” that need to be implemented with specific hardware. It can be.

Hereinafter, an embodiment of the present disclosure will be described in more detail with reference to the attached drawings.

1A and 1B are diagrams to explain an implementation example of an electronic device according to an embodiment.

The electronic device 100 (or user terminal) may be implemented as various portable terminals such as a smartphone, mobile phone, tablet, electronic whiteboard, navigation, PMP (Portable Media Player), MP3 player, game console, etc. However, it is not limited to this and other portable medical devices, wearable devices, etc. may also be included in the electronic device 100. However, for convenience of explanation, the following description will be made assuming that the electronic device 100 is implemented as a smart phone.

According to one embodiment, the electronic device 100 may be implemented to include a flexible display. A flexible display refers to a display that can bend, bend, fold, or roll like paper. Accordingly, the electronic device 100 can be implemented as a foldable device, a bendable device, etc. For example, the electronic device 100 may be manufactured on a flexible substrate with flexible characteristics, such as a plastic substrate, thin glass, or metal foil. According to one example, a flexible display provided in the electronic device 100 may be implemented to detect various types of touch input. For example, a flexible display can detect various types of touch input, such as touch input by the user's hand, touch input by an input device such as a stylus pen, and touch input by a specific electrostatic material. Here, the input device may be implemented as a pen-type input device that can be referred to by various terms such as an electronic pen, stylus pen, and S-pen, but does not necessarily have to be implemented in a pen shape. For example, it may be implemented to have a blocky or flat body.

According to one example, as shown in FIG. 1A, the electronic device 100 may be implemented as a device of various sizes that can be folded in various directions. For example, the electronic device 100 can be folded about the vertical axis and stand on a support such as a table to watch video content, photo content, etc. However, as shown in FIG. 1A, when an image is displayed while the electronic device 100 is in a folded state, image distortion occurs on the folding axis, which may reduce user experience.

Accordingly, according to one embodiment, when the electronic device 100 is folded as shown in FIG. 1B, the displayed image is converted into a curved image and displayed, thereby providing an immersive viewer experience.

Hereinafter, various embodiments of converting a displayed image into a curved image and displaying it according to the folding angle of the electronic device 100 will be described.

FIG. 2A is a block diagram showing the configuration of an electronic device according to an embodiment.

According to FIG. 2A, the electronic device 100 includes a flexible display 110, a sensor 120, and one or more processors 130.

The flexible display 110 may be implemented as a display including a self-emitting device or a display including a non-light-emitting device and a backlight. For example, Liquid Crystal Display (LCD), Organic Light Emitting Diodes (OLED) display, Light Emitting Diodes (LED), micro LED, Mini LED, Plasma Display Panel (PDP), and Quantum dot (QD) display. , QLED (Quantum dot light-emitting diodes), etc. can be implemented as various types of displays. The display 110 may also include a driving circuit and a backlight unit that may be implemented in the form of a-si TFT, low temperature poly silicon (LTPS) TFT, or organic TFT (OTFT).

The sensor 120 may include various types of sensors, such as a touch sensor, proximity sensor, acceleration sensor, geomagnetic sensor, gyro sensor, pressure sensor, position sensor, etc. For example, the touch sensor may be implemented as a capacitive or resistive type. In addition, infrared detection method, surface ultrasonic conduction method, integral tension measurement method, piezo effect method, etc. can be used to detect touch manipulation. An acceleration sensor is a sensor that can measure acceleration and the direction of acceleration when movement occurs. A gyro sensor is a sensor that detects angular velocity by measuring the Coriolis force acting in the direction of the rotational movement. According to the measured value of the gyro sensor, it is possible to detect which direction it was rotated, so the bending direction can be detected. A geomagnetic sensor is a sensor that detects azimuth using a 2-axis or 3-axis fluxgate. In addition, it may be bent itself and may include a bend sensor whose resistance value varies depending on the degree of bending. The bend sensor can be implemented in various forms, such as an optical fiber bending sensor, a pressure sensor, or a strain gauge.

One or more processors (hereinafter referred to as processors) 130 generally control the operation of the electronic device 100. Specifically, the processor 130 is connected to each component of the electronic device 100 and can generally control the operation of the electronic device 100. For example, the processor 130 may be electrically connected to the flexible display 110 and the sensor 120 to control the overall operation of the electronic device 100. The processor 130 may be comprised of one or multiple processors.

The processor 130 may perform operations of the electronic device 100 according to various embodiments of the present invention by executing at least one instruction stored in a memory (not shown).

According to one embodiment, the processor 130 includes a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, and a neural processor (NPU) that process digital image signals. Processing Unit), TCON (Time controller). However, it is not limited to this, and is not limited to a central processing unit (CPU), MCU (Micro Controller Unit), MPU (micro processing unit), and controller. It may include one or more of a (controller), an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by those terms. In addition, the processor 130 may be implemented as a System on Chip (SoC) with a built-in processing algorithm, large scale integration (LSI), or in the form of an application specific integrated circuit (ASIC) or a Field Programmable Gate Array (FPGA).

FIG. 2B is a diagram illustrating a detailed configuration of an implementation example of an electronic device according to an embodiment of the present disclosure.

According to FIG. 2B, the electronic device 100' includes a flexible display 110, a sensor 120, a processor 130, a memory 140, a communication interface 150, a user interface 160, and a speaker 170. Includes. Among the configurations shown in FIG. 2B, detailed descriptions of configurations that overlap with those shown in FIG. 2A will be omitted.

The memory 140 is electrically connected to the processor 130 and can store data necessary for various embodiments of the present disclosure. The memory 140 may be implemented as a memory embedded in the electronic device 100' or as a memory detachable from the electronic device 100' depending on the data storage purpose. For example, data for driving the electronic device 100' is stored in a memory embedded in the electronic device 100, and data for extended functions of the electronic device 100' is stored in the electronic device 100'. It can be stored in a removable memory. Meanwhile, in the case of memory embedded in the electronic device 100', volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (e.g. one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g. NAND flash or NOR flash) etc.), a hard drive, or a solid state drive (SSD). In addition, in the case of memory that is removable from the electronic device 100', a memory card (e.g., CF ( compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external connectable to USB port It may be implemented in the form of memory (for example, USB memory), etc.

According to one example, the memory 140 may store at least one instruction or a computer program including instructions for controlling the electronic device 100'.

According to one example, the memory 140 may store images received from an external device, an external storage medium (e.g., USB), an external server (e.g., a web hard drive), that is, an input image, various data, information, etc. there is.

According to one example, the memory 140 may store information such as threshold values and formulas used in various examples described later.

According to one embodiment, the memory 140 may be implemented as a single memory that stores data generated in various operations according to the present disclosure. However, according to another embodiment, the memory 140 may be implemented to include a plurality of memories each storing different types of data or data generated at different stages.

In the above-described embodiment, various data are described as being stored in the external memory 140 of the processor 130, but at least some of the above-described data may be stored in at least one of the electronic device 100' or the processor 130. Accordingly, it may be stored in the internal memory of the processor 130.

The communication interface 150 is AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired/wireless LAN (Local Area Network), WAN (Wide Area Network), and Ethernet. ), IEEE 1394, HDMI (High-Definition Multimedia Interface), DP (Display Port), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/European Broadcasting Union), optical Images can be streamed or downloaded from an external device 200, an external storage medium (e.g., USB memory), an external server (e.g., a web hard drive), etc. through communication methods such as (Optical) or Coaxial. Signals can be input. Here, the video signal may be a digital video signal of any one of Standard Definition (SD), High Definition (HD), Full HD, or Ultra HD video, but is not limited thereto.

The user interface 160 may be implemented as a device such as buttons, a touch pad, a mouse, and a keyboard, or as a touch screen that can also perform the display function and manipulation input function described above. According to one embodiment, the user interface 160 is implemented as a remote control transceiver and can receive a remote control signal. The remote control transceiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one communication method among infrared communication, Bluetooth communication, or Wi-Fi communication.

Speaker 170 outputs an audio signal. For example, the speaker 170 may convert the digital sound signal processed by the processor 130 into an analog sound signal, amplify it, and output it. For example, the speaker 170 may include at least one speaker unit capable of outputting at least one channel, a D/A converter, an audio amplifier, etc. According to one example, the speaker 170 may be implemented to output various multi-channel sound signals. In this case, the processor 130 may control the speaker 170 to enhance and output the input audio signal to correspond to the enhancement processing of the input image.

The electronic device 100' may additionally include a camera, a tuner, and a demodulator depending on the implementation. A camera (not shown) may be turned on according to a preset event to perform photography, convert the captured image into an electrical signal, and generate image data based on the converted signal. A tuner (not shown) may receive an RF broadcast signal by tuning a channel selected by the user or all previously stored channels among RF (Radio Frequency) broadcast signals received through an antenna. The demodulator (not shown) may receive the digital IF signal (DIF) converted from the tuner, demodulate it, and perform channel decoding.

According to one example, the electronic device 100' may include a housing (not shown) implemented to support the flexible display 110. For example, the housing (not shown) may include a plurality of housings supporting different areas of the flexible display 110. Here, the housing functions as a kind of case and can be implemented in various shapes depending on the folding method. According to one example, the housing (not shown) may be implemented to include a plurality of housings that support each of a plurality of display areas divided according to the folding of the flexible display 110. For example, the plurality of housings may be connected by hinges to enable folding. Other housings (not shown) may include components designed to cover, contain, or support elements such as bearings and mechanisms such as pumps or wheels. For example, it may include a bearing housing, motor housing, wheel housing, etc.

FIGS. 3A and 3B are diagrams for explaining a method of displaying an image in an electronic device according to an embodiment.

According to one embodiment shown in FIG. 3A, the processor 130 identifies the folding angle of the flexible display 110 (S320) while an image is displayed on the flexible display 110 (S310). According to one example, the processor 130 may identify the folding angle of the flexible display 110 while a preset application is executed.

For example, the processor 130 may identify the folding angle α based on data acquired through the sensor 120. The image displayed in step S310 may be an original image or an image on which general image processing (e.g., noise filtering, resolution processing, etc.) has been performed. However, below, for convenience of explanation, it will be explained assuming that it is the original image.

When the folding angle (α) is identified as being equal to or greater than the threshold angle (Ω) (S330:Y), the processor 130 may transform the original image (S340) and display the transformed image (S350). For example, the processor 130 may convert the original image into a curved image.

According to one example, the processor 130 may identify a virtual projection surface of a curved shape based on the folding angle α, and obtain a curved image by projecting the original image onto the virtual projection surface (hereinafter referred to as projection surface). there is.

According to an embodiment shown in FIG. 3B, when the folding angle α is identified as less than the threshold angle Ω (S330:N), the processor 130 may provide a guide UI (S360). For example, the processor 130 may provide a guide UI (or warning UI) that provides information about the folding angle. Here, the guide UI may include at least one of a visual UI, an audio UI, or a haptic UI. For example, the processor 130 may provide a graphic UI in the form of a pop-up UI. In this case, the processor 130 can display the original image until the folding angle is greater than the critical angle.

Since steps S310, S320, S340, and S350 are the same as those shown in FIG. 3A, detailed description will be omitted.

According to one example, the processor 130 may convert the original image into a curved image using a virtual cylinder as shown in FIG. 4 to overcome image distortion at the folding axis. For example, as shown in FIG. 4, a virtual cylinder 400 surrounding the electronic device 100 is built, and pixels of the original image are projected onto a partial area 410 of the virtual cylinder 400. ) to obtain a curved image 420.

According to one example, the processor 130 identifies a virtual cylinder based on the size of the first display area, the size of the second display area, the folding angle, and the height of the flexible display divided based on the folding axis, and An area of the cylinder can be identified as a virtual projection surface. The virtual transparent surface according to one example will be described in detail with reference to FIGS. 10 and 11.

According to one example, the processor 130 may process an image based on the sizes of the first display area and the second display area divided based on the folding axis. The processor 130 identifies a plane connecting one corner of the first display area and one corner of the second display area with the flexible display 110 in a folded state as a virtual plane, and converts the processed image into a virtual plane. A curved image can be obtained by projecting onto a projection surface. The virtual plane according to one example will be described in detail with reference to FIG. 12.

According to one example, the processor 130 may determine the length between one corner of the first display area and one corner of the second display area, the size of the first display area, and the size of the second display area when the flexible display 110 is folded. Based on the size, the downscaling ratio can be identified. The processor 130 may downscale the image based on the identified downscaling ratio and project the downscaled image from the virtual plane to the virtual projection plane to obtain a curved image.

Hereinafter, a specific method for identifying the critical angle used in step S330 and a specific method for obtaining a curved image in step S340 will be described.

FIGS. 5A to 5C and FIG. 6 are diagrams for explaining a method of identifying a critical angle according to an embodiment.

According to one embodiment, the critical angle used in step S330 may be determined based on the characteristics of a concave mirror. A concave mirror is a mirror whose reflecting surface shows the concave shape of a sphere. Because the reflecting surface is concave, there is a true focus where light gathers in front of the reflecting surface. Therefore, the appearance of the object reflected in the mirror changes depending on where the object is located.

For example, as shown in Figures 5a and 5b, when an object is located farther from the mirror than the focus f, an upside-down real image is created, and the size of the real image is determined by the center of the concave sphere. do. Specifically, as shown in Figure 5a, when the distance d to place the image on the mirror is greater than 2f (d > 2f) and when the distance d is greater than f and less than 2f as shown in Figure 5b (f< d < 2f) It is not suitable because it creates an upside-down real image. That is, as shown in Figure 5c, when an object is located between the focus f and the concave mirror (d < f), a large-sized virtual image is created. Meanwhile, the focal distance becomes shorter as the radius of curvature becomes smaller.

Accordingly, as shown in FIG. 5C, the critical angle that satisfies the case of d < f can be determined. Specifically, it is assumed that the electronic device 100 is folded about the central vertical axis. In this case, referring to Figure 6, A is the top point of the folding axis, B is the top-left point of the left display (or left wing), and C is the right display (or right wing). It may be the top-right point of (wing)). Additionally, e = AB = AC, and r may be the radius of the virtual cylinder 400. In this case, as shown in Figure 6, d = e*cos(α/2), and f = r/2 = e/(4*cos(α/2)), so to satisfy d < f, e* The formula cos(α/2) < e/(4*cos(α/2)) is established, and cos(α/2) < 0.5 can be obtained. Accordingly, α > 120, so the critical angle (Ω) can be set to 120. Accordingly, when the critical angle (Ω) is set to 120, the condition that the folding angle (α) is greater than the critical angle (Ω) can be interpreted as having the same meaning as the folding angle (α) exceeding the critical angle (Ω). In other words, it may not be included in the corresponding condition where the folding angle (α) is the critical angle (Ω). However, if the critical angle (Ω) is greater than 120 (for example, 121), the condition in which the folding angle (α) is greater than the critical angle (Ω) may include the case where the folding angle (α) is the critical angle. Of course.

Figures 7 and 8 are diagrams for explaining a method of providing a guide UI according to an embodiment.

According to an embodiment shown in FIG. 7, when the transform mode is turned on (S710), the processor 130 may provide a guide UI (S720). Here, the conversion mode may mean a mode in which a curved image is displayed under the condition that the folding angle (α) is greater than or equal to the critical angle (Ω). The guide UI may be a UI indicating that the conversion mode operates normally when the folding angle is greater than or equal to the critical angle.

For example, as shown in FIG. 8A, when a specific menu button 820 is selected while the image 810 is displayed, the conversion mode may be turned on, and the folding angle (α) is greater than or equal to the threshold angle (Ω). Under the condition, a guide UI 830 may be provided indicating that the conversion mode operates normally, for example, the corresponding application is activated. However, of course, the conversion mode may be set through the settings menu, rather than through the specific menu button 820, or may be set as default.

When the conversion mode is turned on, the processor 130 can identify whether the folding angle (α) of the electronic device 100 is greater than or equal to the threshold angle (Ω) (S730).

When the electronic device 100 is folded and the folding angle (α) is identified as greater than or equal to the threshold angle (Ω) (S730:Y), the processor 130 processes the image 810 to produce a converted image (e.g., a curved image). image) can be displayed (S740).

For example, in FIG. 8B , if the folding angle (α) is identified as greater than the threshold angle (Ω), the curved image 810' may be displayed and the guide UI 830 may disappear from the screen.

When the electronic device 100 is folded and the folding angle (α) is identified as less than the threshold angle (Ω) (S730:N), the processor 130 may provide a guide UI (S720).

For example, as shown in FIG. 8C, the conversion mode operates normally under the condition that the folding angle (α) is greater than the threshold angle (Ω). For example, a guide UI 830 indicating that the corresponding application is activated may be provided. there is. Afterwards, as shown in FIG. 8D, when the electronic device 100 is folded beyond the threshold angle (Ω), a curved image 810' is displayed and the guide UI 830 may disappear from the screen.

However, in the above-described embodiment, it was assumed that the guide UI is provided as a visual UI, but the guide UI may be provided in various forms such as audio UI, haptic UI, and different types of guide UI. may be provided simultaneously.

9 and 10 are diagrams for explaining an image conversion method according to an embodiment.

According to an embodiment shown in FIG. 9, the processor 130 may perform an image conversion process when the folding angle (α) is identified as greater than or equal to the threshold angle (Ω).

According to one example, the processor 130 may build a projection surface (S910) and project the original image onto the projection surface (S920). Here, the projection surface may be one area (or one segment) of the virtual cylinder. For example, the processor 130 converts (or processes) an image based on the sizes of the first display area and the second display area divided based on the folding axis, and converts the processed image in a virtual plane. A curved image can be obtained by projecting onto a virtual projection surface. For example, the processor 130 may identify the center and radius of the virtual cylinder based on the length of the first display area in the first direction, the length of the second display area in the first direction, and the folding angle. The processor 130 may identify the length of the flexible display 110 in the second direction as the height of the virtual cylinder. The processor 130 may identify the area between the points of the virtual cylinder corresponding to the upper point of the first display area and the upper point of the second display area as the virtual projection surface.

As an example, assuming that the electronic device 100 is folded in the vertical direction as shown in FIG. 10, the virtual cylinder 400 can be identified based on three points corresponding to the electronic device 100. . Here, A is the top point of the folding axis, B is the top-left point of the left display (or left wing), and C is the top of the right display (or right wing). It may be a top-right point.

The virtual cylinder 400 can be determined based on three factors: center (O), radius (r), and height (H).

The center O may be the intersection between the mid perpendiculars 431 of the left display (or left wing) AB and the mid perpendiculars 432 of the right display (or right wing) AC. The radius (r) can be determined by a function of the folding angle (α) as shown in Equation 1 below.

Here, referring to FIG. 10, e1=AB, e2=AC, e=BC, and α is the folding angle.

Meanwhile, the height H may be the same as the height of the electronic device 100.

In this case, the projection plane can be determined by the BAC segment of the virtual cylinder 400.

11 and 12 are diagrams for explaining an image projection method according to an embodiment.

According to an embodiment shown in FIG. 11, the processor 130 may downscale the original image at a preset ratio (S1110).

According to one example, when the flexible display 110 is folded, the processor 130 determines the length between one corner of the first display area and one corner of the second display area divided based on the folding axis, and the size of the first display area. And the downscaling ratio may be identified based on the size of the second display area.

For example, the processor 130 may identify the downscaling ratio ds based on Equation 2 below.

The processor 130 may place the downscaled image on a virtual plane (S1120). Here, the virtual plane may be a virtual plane including both edges of the electronic device 100. According to one example, the processor 130 may identify a plane connecting one corner of the first display area and one corner of the second display area as a virtual plane when the flexible display 110 is folded.

The processor 130 may project the downscaled image placed on the virtual plane onto the projection plane (S1130).

For example, referring to FIG. 12 , the processor 130 may downscale the original image 810, place the downscaled image on the virtual plane 440, and then project it onto the projection plane 410.

Afterwards, the processor 130 may display the projected image (S1140). Here, the projected image may be a curved image projected onto the curved projection surface 410 of the virtual cylinder 400.

13 and 14 are diagrams for explaining an image conversion method according to an embodiment.

According to an embodiment shown in FIG. 13, the processor 130 may identify the user's gaze direction (S1310) and rotate the projection surface based on the user's gaze direction (S1320).

According to one example, the processor 130 identifies the user's gaze direction based on data acquired through at least one of a motion sensor (gyroscopic sensor, acceleration sensor, etc.) or a camera, and creates a projection surface based on the user's gaze direction. can be rotated. According to one example, the processor 130 may rotate the projection surface so that the angle between the user's gaze direction and the axis of the projection surface becomes a preset angle. For example, the processor 130 may rotate the projection surface so that the angle between the user's gaze direction and the axis of the projection surface becomes vertical. Here, vertical may mean including a preset error range based on 90 degrees.

For example, as shown in FIG. 14, the processor 130 identifies the user's gaze direction 1401, and determines that the angle between the user's gaze direction 1401 and the axis 1403 of the projection plane 1402 is 90 degrees. The projection plane can be rotated whenever possible.

Hereinafter, various use cases according to an embodiment will be described.

According to one use case, when the folding angle of the flexible display 110 is identified as being greater than or equal to a threshold angle while a preset application is running, the processor 130 processes the image displayed on the flexible display 110 to create a curved image. The flexible display 110 can be controlled to acquire and display a curved image.

Figure 15 is a diagram for explaining a use case according to an embodiment.

According to an embodiment shown in FIG. 15, when the electronic device 100 is in a flat mode (S1510), a preset application, for example, a gallery application, is executed (S1520) and images and/or videos are displayed. There may be a use case in which a selection is made (S1530) and the electronic device 100 is folded (S1540).

In this case, the processor 130 may convert the image based on the folding angle (α) (S1550) and display the converted image (S1560). That is, the processor 130 may convert the displayed image into a curved image based on the folding angle α and display the converted image.

Figure 16 is a diagram for explaining a use case according to an embodiment.

According to an embodiment shown in FIG. 16, in a state in which the electronic device 100 is folded by the folding angle α (S1610), a preset application, for example, a gallery application, is executed (S1620) and images and/or There may be a use case in which a video is selected (S1630) and a converted image is displayed based on the folding angle (α) (S1640).

Afterwards, when the electronic device 100 changes to the flat mode (S1650), the original image can be acquired (S1660) and the acquired original image can be displayed (S1670).

Figure 17 is a diagram for explaining a use case according to an embodiment.

According to an embodiment shown in FIG. 17, according to an embodiment shown in FIG. 16, in a state in which the electronic device 100 is folded by the folding angle α (S1710), a preset application (e.g., There may be a use case in which a gallery application) is executed (S1720), an image and/or video is selected (S1730), and a converted image is displayed based on the folding angle (α) (S1740).

Afterwards, when the folding angle is changed from α to β (S1750), the processor 130 may identify whether the changed folding angle (β) is greater than the critical angle (Ω).

If the changed folding angle (β) is greater than (or greater than) the threshold angle (Ω) (S1760), the processor 130 converts the image and/or video based on the changed folding angle (β) (S1770), and converts Images and/or videos can be displayed (S1780).

If the changed folding angle (β) is smaller than the critical angle (Ω) (S1765), the processor 130 may provide a pop-up warning UI that provides information about the folding angle and display the original image.

Figure 18 is a diagram for explaining a use case according to an embodiment.

According to an embodiment shown in FIG. 18, the processor 130 runs a preset application (for example, a gallery application) while the electronic device 100 is in a flat mode state (S1810) in portrait mode (or portrait mode). ) may be executed (S1820) and an image and/or video may be selected (S1830).

In this case, when the electronic device 100 is folded (S1840), the processor 130 may convert the image in the vertical direction based on the folding angle (α) to obtain and display a curved image in the vertical direction (S1850). ). For example, if the processor 130 identifies that the folding angle (α) of the flexible display 110 is greater than or equal to the threshold angle (Ω), the processor 130 may process the image in the vertical direction to obtain a vertical curved image.

Afterwards, when the electronic device is rotated (or switched) from portrait mode to landscape mode (or landscape mode) (S1860), the processor 130 may identify whether the auto-rotation function is turned on (S1870). ).

When the auto-rotation function is turned on (S1870:Y), the processor 130 can obtain a curved image in the horizontal direction by converting the image in the horizontal direction (S1880).

However, if the auto-rotation function is turned off (S1870:N), the processor 130 turns on the auto-rotation function (S1885) and converts the image in the horizontal direction to obtain a horizontal curved image ( S1880).

Afterwards, the processor 130 may display the curved image obtained in step S1880.

According to the various embodiments described above, it is possible to obtain a curved image using a virtual cylinder and reduce image distortion in a folded screen. When the device is folded, curved images can be used to provide a more immersive viewer experience compared to flat images. In addition, the curved image allows the image shown to the user to be closer to ‘real-life’ and the size of the display can feel larger to the user.

Meanwhile, the methods according to the various embodiments described above may be implemented in the form of applications that can be installed on existing electronic devices. Alternatively, at least some of the methods according to various embodiments of the present disclosure described above may be performed using a deep learning-based artificial intelligence model, that is, a learning network model.

Additionally, the methods according to various embodiments of the present disclosure described above may be implemented only by upgrading software or hardware for an existing electronic device.

Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device or an external server of the electronic device.

Meanwhile, according to an example of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media (e.g., a computer). You can. The device is a device capable of calling instructions stored from a storage medium and operating according to the called instructions, and may include an electronic device (eg, electronic device A) according to the disclosed embodiments. When an instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or using other components under the control of the processor. Instructions may contain code generated or executed by a compiler or interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium does not contain signals and is tangible, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

Additionally, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed on a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or online through an application store (e.g. Play Store™). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or created temporarily in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

In addition, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be omitted. Additional components may be included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added. You can.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field pertaining to the disclosure without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: Electronic device 110: Flexible display
120: sensor 130: one or more processors

Claims (20)

In a foldable electronic device,
flexible display;
sensor; and
While an image is displayed on the flexible display, if the folding angle of the flexible display is identified as greater than a threshold angle based on data acquired through the sensor, a virtual projection surface of a curved shape is created based on the folding angle ( identify the virtual projecting surface,
Obtaining a curved image by projecting the image onto the virtual projection surface,
An electronic device comprising: one or more processors controlling the flexible display to display the obtained curved image. According to paragraph 1,
The one or more processors:
If the folding angle of the flexible display is identified as being less than the threshold angle, a guide UI providing information about the folding angle is provided,
The guide UI is,
An electronic device comprising at least one of a visual UI, an audio UI, or a haptic UI. According to paragraph 1,
The one or more processors:
Process the image based on the sizes of the first display area and the second display area divided based on the folding axis,
Identifying a plane connecting a corner of the first display area and a corner of the second display area as a virtual plane when the flexible display is folded,
An electronic device that obtains a curved image by projecting the processed image from the virtual plane onto the virtual projection plane. According to paragraph 3,
The one or more processors:
A downscaling ratio based on the length between one corner of the first display area and one corner of the second display area, the size of the first display area, and the size of the second display area when the flexible display is folded. identify,
Downscale the image based on the identified downscaling ratio,
An electronic device that obtains a curved image by projecting the downscaled image from the virtual plane to the virtual projection plane. According to paragraph 1,
The one or more processors:
Identifying a virtual cylinder based on the sizes of the first and second display areas divided based on a folding axis, the folding angle, and the height of the flexible display,
An electronic device that identifies an area of the virtual cylinder as the virtual projection plane. According to clause 5,
The one or more processors:
identify the center and radius of the virtual cylinder based on the first direction length of the first display area, the first direction length of the second display area, and the folding angle;
Identifying the length of the flexible display in the second direction as the height of the virtual cylinder,
The electronic device identifies an area between points of the virtual cylinder corresponding to each of an upper point of the first display area and an upper point of the second display area as the virtual projection surface. According to paragraph 1,
The sensor is,
Includes at least one of a motion sensor or a camera,
The one or more processors:
Identifying the user's gaze direction based on data acquired through at least one of the motion sensor or the camera,
Rotating the virtual projection surface so that the angle between the user's gaze direction and the axis of the virtual projection surface becomes a preset angle,
An electronic device that acquires the curved image by projecting the processed image onto the rotated virtual projection plane. In clause 7,
The one or more processors:
Rotating the virtual projection plane so that the angle formed by the user's gaze direction and the axis of the virtual projection plane is vertical,
An electronic device that obtains the curved image by projecting the processed image onto the rotated virtual projection plane. According to paragraph 1,
The one or more processors:
If the folding angle of the flexible display is identified as being greater than or equal to the threshold angle while a preset application is running, the image displayed on the flexible display is processed to obtain the curved image,
An electronic device that controls the flexible display to display the obtained curved image. According to paragraph 1,
The one or more processors:
If the folding angle of the flexible display is identified as being greater than or equal to the threshold angle while the preset application is being executed while the electronic device is in the portrait mode, the image displayed on the flexible display is processed to obtain the curved image. do,
Identify whether an auto-rotation function is turned on when the electronic device switches from the portrait mode to the landscape mode,
When the auto-rotation function is turned on, the image displayed on the flexible display is processed to obtain a curved image,
When the auto-rotation function is turned off, turn on the auto-rotation function and process the image displayed on the flexible display to obtain a curved image,
An electronic device that controls the flexible display to display the obtained curved image. In a method of controlling a foldable electronic device,
While an image is displayed on a flexible display, if the folding angle of the flexible display is identified as being greater than a critical angle based on data acquired through a sensor, a virtual curved shape is generated based on the folding angle. identifying a virtual projecting surface;
acquiring a curved image by projecting the image onto the virtual projection surface; and
A control method comprising: displaying the obtained curved image on a flexible display in a folded state. According to clause 11,
If the folding angle of the flexible display is identified as being less than the threshold angle, providing a guide UI providing information about the folding angle; further comprising:
The guide UI is,
A control method including at least one of a visual UI, an audio UI, and a haptic UI. According to clause 11,
The step of acquiring the curved image is,
Processing the image based on the sizes of a first display area and a second display area divided based on a folding axis;
identifying a plane connecting a corner of the first display area and a corner of the second display area as a virtual plane when the flexible display is folded; and
A control method comprising: acquiring a curved image by projecting the processed image from the virtual plane to the virtual projection plane. According to clause 13,
The step of acquiring the curved image is,
A downscaling ratio based on the length between one corner of the first display area and one corner of the second display area, the size of the first display area, and the size of the second display area when the flexible display is folded. identifying;
downscaling the image based on the identified downscaling ratio; and
A control method comprising: acquiring a curved image by projecting the downscaled image from the virtual plane to the virtual projection plane. According to clause 11,
The step of identifying the virtual projection surface is,
Identifying a virtual cylinder based on the sizes of a first display area and a second display area divided based on a folding axis, the folding angle, and the height of the flexible display; and
A control method comprising: identifying an area of the virtual cylinder as the virtual projection surface. According to clause 15,
The step of identifying the virtual projection surface is,
identifying the center and radius of the virtual cylinder based on the first direction length of the first display area, the first direction length of the second display area, and the folding angle;
identifying a length of the flexible display in a second direction as the height of the virtual cylinder; and
Identifying an area between points of the virtual cylinder corresponding to each of the upper point of the first display area and the upper point of the second display area as the virtual projection surface. According to clause 11,
The sensor is,
Includes at least one of a motion sensor or a camera,
The step of acquiring the curved image is,
Identifying a user's gaze direction based on data acquired through at least one of the motion sensor or the camera;
rotating the virtual projection surface so that the angle between the user's gaze direction and the axis of the virtual projection surface becomes a preset angle; and
A control method comprising: acquiring the curved image by projecting the processed image onto the rotated virtual projection plane. According to clause 17,
The step of rotating the virtual projection plane is,
A control method for rotating the virtual projection surface so that the angle formed by the user's gaze direction and the axis of the virtual projection surface is vertical. According to clause 11,
The step of acquiring the curved image is,
If the folding angle of the flexible display is identified as being greater than or equal to the threshold angle while a preset application is running, processing the image displayed on the flexible display to obtain the curved image; controlling, comprising a. method. According to clause 11,
If the folding angle of the flexible display is identified as being greater than or equal to the threshold angle while the preset application is being executed while the electronic device is in the portrait mode, the image displayed on the flexible display is processed to obtain the curved image. steps;
identifying whether an auto-rotation function is turned on when the electronic device is switched from the portrait mode to the landscape mode;
obtaining a curved image by processing an image displayed on the flexible display when the auto-rotation function is turned on;
When the auto-rotation function is turned off, turning on the auto-rotation function and processing an image displayed on the flexible display to obtain a curved image;
Controlling the flexible display to display the obtained curved image.

Images