KR20230164537A - Electronic device for receiving and displaying application screen from external device and method for controlling the same - Google Patents

Abstract

An electronic device is disclosed. The electronic device controls the display to display a display including a touch module, a communication interface, and an application screen and a pen pressure adjustment UI received from an external device through the communication interface, and displays a pen pressure value set through the pen pressure adjustment UI and the touch module. transmits information about the identified touch input to an external device, receives an application screen including a touch trace identified based on the pen pressure value and information about the touch input from the external device through a communication interface, and displays the received application screen It includes one or more processors that control the display to display.

Description

Electronic device for receiving and displaying application screen from external device and method for controlling the same }

This disclosure relates to an electronic device and a method of controlling the same, and more specifically, to an electronic device that receives a touch input and a method of controlling the same.

Touch screen refers to a screen that detects the location touched by the user by touching the screen. Recently, touch screens have been widely used not only in various mobile devices such as mobile phones, MP3 players, navigation, and PMPs, but also in large displays such as electronic whiteboards.

In order to detect the pen pressure value on the touch screen, you must use an active electronic pen or insert PIEZO into the panel.

The active type electronic pen inserts a piezoelectric sensor into the pen tip to convert the resistance value or change the capacitance and transfer it to a touch signal or transmit the pen pressure value through communication. When PIEZO is inserted into the panel, PIEZO detects the panel being pressed and the pressing force is converted into an electrical signal, through which pen pressure can be measured.

If the electronic pen itself has a pen pressure sensor, device complexity increases because a structure to measure pen pressure must be included in the thin pen, and the measured pen pressure must be included in the touch signal or transmitted to the touch screen using communication, which requires a battery. This requires frequent charging or replacement of the battery, which is inconvenient.

When inserting PIEZO into a panel, it is difficult to detect small forces as larger inches become larger, and it is difficult to accurately measure pen pressure values depending on the location. It is also difficult to measure pen pressure when additional forces other than the device for measuring pen pressure are transmitted. For example, when inputting a signal to the panel with a pen and the palm is touching, a different force is applied than when only the pen is used, so there is a problem in that accurate pen pressure cannot be measured.

According to one embodiment, the electronic device controls the display to display a display including a touch module, a communication interface, and an application screen and a pen pressure adjustment UI received from an external device through the communication interface, and the pen pressure adjustment UI. Transmits information about the pen pressure value set through and the touch input identified through the touch module to the external device, and identifies information about the pen pressure value and the touch input from the external device through the communication interface. and one or more processors to receive an application screen including a touch trace and control the display to display the received application screen.

The pen pressure adjustment UI may include a pen pressure adjustment bar that can adjust the size of the pen pressure value. The one or more processors transmit information including a pen pressure value set by the pen pressure adjustment bar, touch coordinates and touch size of the touch input identified through the touch module to the external device, and transmit the pen pressure value from the external device. An application screen containing a touch trace with a thickness corresponding to can be received.

The one or more processors may control the display to display a guide UI including thickness information corresponding to the pen pressure value set through the pen pressure adjustment UI.

It further includes a memory that stores thickness information according to a pen pressure value corresponding to each of a plurality of applications, wherein the one or more processors perform a touch operation corresponding to each of the plurality of applications based on the set pen pressure value and the information stored in the memory. The display can be controlled to display the guide UI including trace thickness information.

When a touch input is identified on an area separate from the application screen after the pen pressure value is set through the pen pressure adjustment UI, the one or more processors respond to the touch input based on thickness information identified based on the set pen pressure value. The display can be controlled to display a touch trace.

The touch module includes an infrared touch module, and the one or more processors identify a pen pressure value corresponding to the touch input based on the touch size of the touch input detected by the infrared touch module and adjust the pen pressure. At least one of a pen pressure value set through the UI or a pen pressure value identified based on the touch size may be transmitted to the external device.

The one or more processors may adjust the pen pressure value set through the pen pressure adjustment UI based on the touch size and transmit the adjusted pen pressure value to the external device.

The one or more processors apply a first weight to the first pen pressure value set through the pen pressure adjustment UI, and apply a second weight to the second pen pressure value identified based on the touch size, and the first weight is A third pen pressure value may be identified based on the applied first pen pressure value and the second pen pressure value to which the second weight is applied, and the identified third pen pressure value may be transmitted to the external device.

The one or more processors provide the pen pressure adjustment UI to the remaining area of the display when the application screen is provided to a partial area of the display, and provide the pen pressure adjustment UI to the remaining area of the display when the application screen is provided to the entire area of the display. may be provided as a pop-up screen in one area of the application screen.

A control method of an electronic device includes the steps of displaying an application screen and a pen pressure adjustment UI received from an external device, transmitting a pen pressure value set through the pen pressure adjustment UI and information about a touch input identified through a touch module to the external device. It may include receiving an application screen including a touch trace identified based on the pen pressure value and information about the touch input from the external device, and displaying the received application screen.

A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an operation, wherein the operation includes displaying an application screen and a pen pressure adjustment UI received from an external device. A step of transmitting a pen pressure value set through the pen pressure adjustment UI and information about a touch input identified through a touch module to the external device, identification based on the pen pressure value and information about the touch input from the external device. It may include receiving an application screen including a received touch trace and displaying the received application screen.

1 is a drawing for explaining the schematic operation of an electronic device according to an embodiment of the present invention.
Figure 2a is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.
FIG. 2B is a diagram illustrating a detailed configuration of an implementation example of an electronic device according to an embodiment of the present disclosure.
FIGS. 3A to 3D are diagrams for explaining a touch detection method according to an embodiment.
Figure 4 is a flowchart illustrating a method of providing a pen pressure adjustment UI according to an embodiment of the present invention.
FIGS. 5A and 5B are diagrams for explaining a method of providing a pen pressure adjustment UI according to an embodiment.
FIGS. 6A and 6B are diagrams for explaining the thickness of a touch trace provided on an application screen according to a pen pressure value according to an embodiment.
7 is a flowchart illustrating a method of providing a pen pressure adjustment UI according to an embodiment of the present invention.
FIGS. 8A to 8D are diagrams for explaining a method of providing a guide UI according to an embodiment.
Figure 9 is a flowchart for explaining a method of identifying a pen pressure value according to an embodiment.
Figure 10 is a diagram for explaining a method of identifying a pen pressure value according to a touch size according to an embodiment.
FIG. 11 is a diagram for explaining operations according to an implementation example of an electronic device according to an embodiment.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terminology used in the present disclosure selects general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used solely for the purpose of distinguishing one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “consist of” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

The expression at least one of A or B should be understood as indicating either “A” or “B” or “A and B”.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of “modules” or a plurality of “units” may be integrated into at least one module and implemented with one or more processors (not shown), except for “modules” or “units” that need to be implemented with specific hardware. You can.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present disclosure in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

1 is a drawing for explaining the schematic operation of an electronic device according to an embodiment of the present invention.

Referring to FIG. 1, the electronic device 100 according to an embodiment of the present invention may be implemented as an electronic whiteboard including an input panel such as a touch panel or a touch screen, but is not limited thereto, and may be implemented as a mobile phone or a smartphone. , electronic picture frame, DID (Digital Information Display), kiosk, PMP (Portable Media Mlayer), MP3 player, game console, LFD (Large Format Display), laptop, laptop), TV, monitor ( It can be implemented with various types of electronic devices capable of touch input, such as monitors, projector systems, etc. However, the present invention is not limited to this and other home appliances, medical devices, etc. may also be included in the electronic device 100.

The electronic device 100 may communicate with the external device 200 to receive a content screen from the external device 200 and display it. Here, the external device 200 may be implemented to execute various types of applications (or software or programs) and transmit application execution screens to the electronic device 100. For example, the external device 200 is a variety of source devices (or can be implemented as a host device). Here, the application may include various office applications such as PowerPoint, PDF, Word, and Excel, web browser applications, and various other utility applications.

According to one embodiment, the electronic device 100 may receive a touch input by an input device such as an electronic pen or an input object such as the user's hand. Here, the electronic pen 10 has a certain length and may be implemented in a thin and long pen shape so that the user can easily hold it. Accordingly, the electronic pen 10 may be referred to by various terms such as a pen-type input device, stylus pen, S-pen, etc., but does not necessarily have to be implemented in the shape of a pen. For example, it may be implemented to have a blocky or flat body. According to one example, the electronic pen 10 may be implemented in a passive manner. The passive method is a method in which the electronic device 100 detects the touch input of the electronic pen 10, and includes a capacitive method and an EMR (Electro Magnetic Resonance) method. When implemented in a capacitive manner, the electronic pen 10 includes a conductive tip with a constant area, and when implemented in an EMR method, the electronic pen 10 may include a coil that induces electricity by an external magnetic field signal. .

When the electronic pen 10 is implemented in a passive manner, the electronic device 100 cannot recognize the pen pressure value, so a fixed pen pressure value is used. In this case, when writing on the application screen received from the electronic device 100, the user will not be able to input text or drawings at the desired thickness. Although touch input can be supported with various thicknesses depending on the pen pressure value depending on the application, there is a problem that this function cannot be used when using the passive electronic pen 10.

Accordingly, the following will describe various embodiments that can provide handwriting input of various thicknesses supported by the application even when using a passive electronic pen.

Figure 2a is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.

According to FIG. 2A, the electronic device 100 includes a display 110, a communication interface 120, and one or more processors 130.

The display 110 may be implemented as a display including a self-emitting device or a display including a non-emitting device and a backlight. For example, Liquid Crystal Display (LCD), Organic Light Emitting Diodes (OLED) display, Light Emitting Diodes (LED), micro LED, Mini LED, Plasma Display Panel (PDP), and Quantum dot (QD) display. , QLED (Quantum dot light-emitting diodes), etc. can be implemented as various types of displays. The display 110 may also include a driving circuit and a backlight unit that may be implemented in the form of a-si TFT, low temperature poly silicon (LTPS) TFT, or organic TFT (OTFT). Meanwhile, the display 110 may be implemented as a flexible display, a rollable display, a 3D display, or a display in which a plurality of display modules are physically connected.

According to one embodiment, the display 110 may include a touch module. According to one example, the touch module may include a sensor such as a touch sensor or a proximity sensor, or may include a sensor and a computing device. For example, an arithmetic device may be implemented as an MCU (Micro Controller Unit) or MPU (Micro Processor Unit) that decodes instructions containing registers, arithmetic circuits, and control circuits to execute arithmetic and control operations, but is not limited to these. . According to one embodiment, the touch module may obtain information about a touch input detected through a touch sensor and transmit it to the processor 130. Here, information about touch input may include touch coordinates, touch size, pen pressure value, tilt information, etc. However, since the passive electronic pen 10 cannot detect the pen pressure value, the touch module transmits the preset pen pressure value to the processor 130.

According to one example, the touch sensor may be implemented as at least one of capacitive, resistive, infrared, and ultrasonic. Capacitive refers to a sensor that uses a dielectric coated on the display surface to calculate touch coordinates by detecting micro-electricity excited by the user's body when a part of the user's body touches the display surface. A pressure-sensitive touch sensor includes two built-in electrode plates and calculates touch coordinates by detecting current flow when the upper and lower plates of the touched point are touched. The infrared method is invisible to the human eye, but due to the nature of light, it has a tendency to go straight and is blocked when there is an obstacle, so it refers to a touch sensor that uses the property of not being able to proceed. In addition, surface ultrasonic conduction method, integral tension measurement method, piezo effect method, etc. can be used to detect touch manipulation.

In addition, the touch sensor may use a magnet, a magnetic field sensor, an optical sensor, or a proximity sensor to determine whether a touch object such as a finger or a stylus pen is in contact with or is close to it.

FIGS. 3A to 3D are diagrams for explaining a touch detection method according to an embodiment.

According to one embodiment, the capacitive touch sensor may have an electrode pattern structure as shown in FIG. 3A. The grid-like electrode pattern shown in FIG. 3A includes vertical lines 111 and horizontal lines 112-1, 112-2, and 112-3. This method is divided into a transmitter that transmits an electric field and a receiver that detects changes in the electric field, and can detect coordinates by detecting changes in the electric field due to contact with a conductor. Specifically, when the electronic pen 10 itself outputs an electric field, a change in the electric field is detected in the electrode pattern. If you want to detect the contact of a finger, you may use a method in which among the electrodes in two crossing directions, the electrode in the first direction functions as a transmitter and the electrode in the second direction functions as a receiver.

According to one embodiment, the infrared touch sensor has an infrared light-emitting diode (Infrared LED), a light-emitting device, and a photo-transistor (or photodiode), a light-receiving device, arranged to face each other to form an opto-matrix frame. ) can be made and mounted on the display 110. As shown in Figure 3b, when the screen is selected with an object that can block light, such as a finger, on the display 110 equipped with an opto-matrix frame, the light emitted from the infrared light-emitting diode is transmitted by the finger. Since it is blocked and cannot be detected by the photo transistor on the other side, it is possible to find out which cell was blocked. For example, as shown in FIG. 3C, infrared light emitting diodes 310 and 320 may be disposed on the left and lower sides of the display 110, and photo transistors 330 and 340 may be disposed on the right and upper sides.

For example, when a specific point 30 on the screen is touched with the electronic pen 10, an area where infrared light is blocked is created as shown in FIG. 3D, so that the touch point can be identified. In addition, the size and location of the area (i.e., shadow) obscured by infrared light varies depending on the size and location of the touch object, for example, the electronic pen 10, touched at a specific point 30, so the touch size can also be identified. You can do it. For example, in the case of a small object, the area blocking the light is small, so only a small number of photo transistors cannot receive light, and in the case of a large object, a large number of photo transistors cannot receive light, so the position/distance of the object and the shadow You can easily find the area of an object using its size.

The communication interface 120 may be a component that communicates with an external device (eg, 200 in Figure 1). For example, the communication interface 120 includes AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired/wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/ European Broadcasting Union), Optical Streaming or downloading from an external device (e.g., source device), external storage medium (e.g., USB memory), external server (e.g., web hard), etc. through communication methods such as coaxial, etc. You can receive video signals through . Here, the video signal may be a digital video signal of any one of Standard Definition (SD), High Definition (HD), Full HD, or Ultra HD video, but is not limited thereto.

According to one example, the communication interface 120 may include a first communication interface and a second communication interface. The first communication interface can communicate with the external device 200 to transmit/receive video data/audio data, etc. For example, the first communication interface may be implemented as an interface capable of transmitting/receiving video data/audio data, such as HDMI (High-Definition Multimedia Interface), DP (Display Port), etc., but is not limited thereto. The second communication interface may transmit the touch signal obtained through the touch module to the external device 200. Here, the touch signal may include at least one of touch coordinates, touch size, pen pressure, and tilt value of the electronic pen 10. For example, the second communication interface may be implemented as an interface such as USB (Universal Serial Bus), but is not limited to this.

One or more processors 130 (hereinafter referred to as processors) generally control the operation of the electronic device 100. Specifically, the processor 130 is connected to each component of the electronic device 100 and can generally control the operation of the electronic device 100. For example, the processor 130 may be electrically connected to the display 110 and the memory (FIG. 2B, 140) to control the overall operation of the electronic device 1100. The processor 130 may be comprised of one or multiple processors.

The processor 130 may perform operations of the electronic device 100 according to various embodiments by executing at least one instruction stored in the memory 140.

According to one embodiment, the processor 130 includes a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, and a neural processor (NPU) that process digital image signals. Processing Unit), TCON (Time controller). However, it is not limited to this, and is not limited to a central processing unit (CPU), MCU (Micro Controller Unit), MPU (micro processing unit), and controller. It may include one or more of a (controller), an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by those terms. In addition, the processor 130 may be implemented as a System on Chip (SoC) with a built-in processing algorithm, large scale integration (LSI), or in the form of an application specific integrated circuit (ASIC) or a Field Programmable Gate Array (FPGA).

FIG. 2B is a diagram illustrating a detailed configuration of an implementation example of an electronic device according to an embodiment of the present disclosure.

According to FIG. 2B, the electronic device 100' includes a display 110, a communication interface 120, a processor 130, a memory 140, a user interface 150, a camera 160, and a speaker 170. do. Among the configurations shown in FIG. 2B, detailed descriptions of configurations that overlap with those shown in FIG. 2A will be omitted.

The memory 140 is electrically connected to the processor 130 and can store data necessary for various embodiments of the present disclosure. The memory 140 may be implemented as a memory embedded in the electronic device 100' or as a memory detachable from the electronic device 100, depending on the data storage purpose. For example, data for driving the electronic device 100 is stored in a memory embedded in the electronic device 100, and data for extended functions of the electronic device 100' is stored in the electronic device 100'. It can be stored in removable memory. Meanwhile, in the case of memory embedded in the electronic device 100', volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (e.g. one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g. NAND flash or NOR flash) etc.), a hard drive, or a solid state drive (SSD). In addition, in the case of memory that is removable from the electronic device 100', a memory card (e.g., CF ( compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external connectable to USB port It may be implemented in the form of memory (for example, USB memory), etc.

According to one example, the memory 140 may store at least one instruction or a computer program including instructions for controlling the electronic device 100'.

According to one example, the memory 140 may include images received from an external device (e.g., a source device), an external storage medium (e.g., USB), an external server (e.g., a web hard drive), that is, an input image, Various data, information, etc. can be stored.

According to one example, the memory 140 may include various types of information described later, such as thickness information corresponding to pen pressure values for a plurality of applications, information about pen pressure values for each touch size, weight information for each touch size, and thickness information corresponding to pen pressure values for each pen tip. Information related to the embodiment can be stored. For example, the information may be stored in the form of a lookup table, or in the form of a formula or algorithm.

According to one embodiment, the memory 140 may be implemented as a single memory that stores data generated in various operations according to the present disclosure. However, according to another embodiment, the memory 140 may be implemented to include a plurality of memories each storing different types of data or data generated at different stages.

In the above-described embodiment, various data are described as being stored in the external memory 140 of the processor 130, but at least some of the above-described data may be stored in at least one of the electronic device 100' or the processor 130. Accordingly, it may be stored in the internal memory of the processor 130.

The user interface 150 may be implemented as a device such as buttons, a touch pad, a mouse, and a keyboard, or as a touch screen that can also perform the display function and manipulation input function described above. According to one embodiment, the user interface 150 may be implemented as a remote control transceiver and receive a remote control signal. The remote control transceiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one communication method among infrared communication, Bluetooth communication, or Wi-Fi communication.

The camera 160 may be turned on and perform photography according to a preset event. The camera 160 may convert the captured image into an electrical signal and generate image data based on the converted signal. For example, a subject is converted into an electrical image signal through a semiconductor optical device (CCD; Charge Coupled Device), and the converted image signal can be amplified and converted into a digital signal and then processed. However, the camera 160 may not be included depending on the implementation of the electronic device 100'.

The speaker 170 outputs an audio signal. For example, the speaker 170 may convert the digital sound signal processed by the processor 130 into an analog sound signal, amplify it, and output it. For example, the speaker 170 may include at least one speaker unit capable of outputting at least one channel, a D/A converter, an audio amplifier, etc. According to one example, the speaker 170 may be implemented to output various multi-channel sound signals. In this case, the processor 130 may control the speaker 170 to enhance and output the input audio signal to correspond to the enhancement processing of the input image.

The electronic device 100' may additionally include a tuner and a demodulator depending on implementation. A tuner (not shown) may receive an RF broadcast signal by tuning a channel selected by the user or all previously stored channels among RF (Radio Frequency) broadcast signals received through an antenna. The demodulator (not shown) may receive the digital IF signal (DIF) converted from the tuner, demodulate it, and perform channel decoding.

Figure 4 is a flowchart illustrating a method of providing a pen pressure adjustment UI according to an embodiment of the present invention.

According to an embodiment shown in FIG. 4, the processor 130 may control the display 110 to display the application screen and pen pressure adjustment UI received from the external device 200 (S410).

According to one example, when an application screen is provided to a partial area of the display 110, the processor 130 may provide a pen pressure adjustment UI to the remaining area of the display 110.

According to one example, when the application screen is provided as a full screen in the entire area of the display 110, the processor 130 may provide a pen pressure adjustment UI as a pop-up screen in one area of the application screen.

However, in some cases, when the application screen is provided in a certain area of the display 110, the processor 130 may provide the pen pressure adjustment UI as a pop-up screen in one area of the application screen.

The processor 130 may transmit the pen pressure value set through the pen pressure adjustment UI and information about the touch input identified through the touch module to the external device 200 (S420).

According to one example, the pen pressure adjustment UI may include a pen pressure adjustment bar that can adjust the size of the pen pressure value. Here, the pen pressure adjustment bar may be implemented in a form that allows one of continuous pen pressure values to be selected according to a user command. User commands may be entered through touch input or remote control signals. For example, the pen pressure value can be set by adjusting the position of the indicator included in the pen pressure adjustment bar. For example, the position of the indicator can be adjusted through touch input or a remote control signal.

However, it is not limited to this, and the pen pressure adjustment UI can also be implemented in a form that allows selection of one of a plurality of predefined pen pressure values (or pen pressure levels). Here, information about the touch input (hereinafter referred to as touch input information) may include the touch coordinates and touch size of the touch input. According to one example, touch input information may be received from a touch module. In this case, the processor 130 may transmit information including the pen pressure value set by the pen pressure adjustment bar, touch coordinates provided from the touch module, and touch size to the external device 200.

According to one embodiment, the touch input information received from the touch module may include at least one of touch coordinates, touch size, pen pressure, and tilt value of the touch input. Here, the pen pressure value may be a fixed value. This is because if the electronic pen 10 is a passive device, the touch module cannot detect the pen pressure value. In this case, the processor 130 may replace the fixed pen pressure value received from the touch module with the pen pressure value set through the pen pressure adjustment UI and transmit it to the external device 200. According to one example, the touch input information is acquired by a processor such as an MCU provided in the touch module and transmitted in the form of a packet containing other touch input information such as touch coordinates and tilt, but the pen pressure value is stored in a SOC separate from the touch module. This is because it is obtained by the provided processor 130.

Afterwards, the processor 130 may receive an application screen including a touch trace identified based on the pen pressure value and other touch input information from the external device 200 (S430), and display the received application screen (S430). S440). According to one example, the application screen received from the external device 200 may be an application screen including a touch trace with a thickness corresponding to the pen pressure value. That is, since the processor 130 transmitted the pen pressure value set through the pen pressure adjustment UI as information about the touch input in step S420, an application screen including a touch trace corresponding to the pen pressure value is received in step S430. Accordingly, even when using the passive electronic pen 10, it is possible to perform a touch input with a thickness desired by the user.

FIGS. 5A and 5B are diagrams for explaining a method of providing a pen pressure adjustment UI according to an embodiment.

According to one embodiment, as shown in FIG. 5A, when the application screen 510 received from the external device 200 is provided in a partial area of the display 110, the processor 130 operates the pen pressure adjustment UI 520. can be provided to the remaining area of the display 110. However, in some cases, the processor 130 may provide the pen pressure adjustment UI 520 as a pop-up screen in one area of the application screen 510. For example, the form of provision of the application screen 510 and/or the pen pressure adjustment UI 520 may be set and/or changed by the user.

According to one embodiment, as shown in FIG. 5B, when the application screen 530 received from the external device 200 is provided as a full screen in the entire area of the display 110, the pen pressure adjustment UI 520 is displayed on the application screen. It can be provided as a pop-up screen in one area of (530).

FIGS. 6A and 6B are diagrams for explaining the thickness of a touch trace provided on an application screen according to a pen pressure value according to an embodiment.

According to one embodiment, as shown in FIG. 6A, when the application screen 510 is provided, the pen pressure value is adjusted to a first value, for example, the smallest value through the pen pressure adjustment UI 520, and the electronic pen ( When a touch input is received through 10), the thickness of the touch trace may be displayed as thin. This means that the pen pressure value set through the pen pressure adjustment UI 520, that is, the first value, is transmitted to the external device 200 along with information about other touch inputs, for example, touch coordinates, and the first value is transmitted from the external device 200 to the external device 200. This is because an application screen reflecting a touch trace with a thickness corresponding to the value was received.

According to one embodiment, as shown in FIG. 6B, when the application screen 510 is provided, the pen pressure value is adjusted to a second value, for example, the largest value, through the pen pressure adjustment UI 520, and the electronic pen ( When a touch input is received through 10), the thickness of the touch trace may be displayed as relatively thick. This means that the pen pressure value set through the pen pressure adjustment UI 520, that is, the second value, is transmitted to the external device 200 along with information about other touch inputs, for example, touch coordinates, and the second value is transmitted from the external device 200. This is because an application screen reflecting a touch trace with a thickness corresponding to the value was received.

7 is a flowchart illustrating a method of providing a pen pressure adjustment UI according to an embodiment of the present invention.

According to one embodiment, the processor 130 may provide a guide UI including thickness information corresponding to the pen pressure value set through the pen pressure adjustment UI as well as the pen pressure adjustment UI.

According to an embodiment shown in FIG. 7, the processor 130 may control the display 110 to display the application screen and pen pressure adjustment UI received from the external device 200 through the communication interface 120 ( S710).

Additionally, the processor 130 may display a guide UI including thickness information corresponding to the pen pressure value set through the pen pressure adjustment UI (S720). This will be explained in detail with reference to FIGS. 8A to 8D.

The processor 130 may transmit the pen pressure value set through the pen pressure adjustment UI and information about the touch input identified through the touch module to the external device 200 (S730).

Afterwards, the processor 130 may receive an application screen including a touch trace identified based on information about the pen pressure value and touch input from the external device 200 (S740), and display the received application screen. (S750).

FIGS. 8A to 8D are diagrams for explaining a method of providing a guide UI according to an embodiment.

According to one embodiment, the processor 130 may provide a guide UI so that the user can recognize the thickness corresponding to the pen pressure value set through the pen pressure adjustment UI.

According to one example, when the pen pressure value is set through the pen pressure adjustment UI 520 as shown in FIG. 8A, a guide UI 810 including a thickness corresponding to the set pen pressure value may be provided. In this case, the processor 130 cannot identify the application corresponding to the application screen 510 and therefore cannot actually identify the exact thickness provided by the application. Accordingly, a preset thickness corresponding to the corresponding pen pressure value can be provided through the guide UI 810. In the embodiment, the guide UI 810 is shown as being provided as a pop-up screen on the application screen 510, but the guide UI 810 is not necessarily limited to this. For example, it is possible for the guide UI 810 to be provided in an area other than the area where the application screen 510 is provided, for example, in an area to the right of the area where the pen pressure adjustment UI 520 is provided.

According to one example, after the pen pressure value is set through the pen pressure adjustment UI 520 as shown in FIG. 8B, when a touch input is identified on an area 111 separate from the application screen 510, the processor 130 A touch trace 820 corresponding to a touch input may be displayed based on thickness information identified based on the set pen pressure value. Accordingly, the user can receive rough guidance on thickness information for the set pen pressure value.

According to one example, when the pen pressure value is set through the pen pressure adjustment UI 520 as shown in FIG. 8C, a guide UI 830 including thickness information corresponding to the pen pressure value set for each of the plurality of applications may be provided. . Thickness information corresponding to the pen pressure value for each of the plurality of applications may be received from the external device 200 or may be received from an external server and previously stored in the memory 140.

The processor 130 may display a guide UI 830 including thickness information of a touch trace corresponding to each of a plurality of applications based on the set pen pressure value and information stored in the memory 140. This is because the processor 130 cannot identify the application corresponding to the application screen 510 and therefore cannot actually identify the exact thickness provided by the application.

For example, as shown in FIG. 8C, the guide UI 830 includes thickness information corresponding to the pen pressure value set for each of application A (APP A), application B (APP B), and application C (APP C). can be provided. Accordingly, the user can identify the writing thickness corresponding to the currently provided application screen 510.

According to one example, when the pen pressure value is set through the pen pressure adjustment UI 520 as shown in FIG. 8D, a guide UI 840 including the thickness corresponding to the pen pressure value set for each pen tip is provided. It can be. Thickness information corresponding to the pen pressure value for each of the plurality of pen tips may be previously stored in the memory 140 as a preset value. Here, the pen tip may include various pen tip types such as a triangular shape, a round shape, a blade shape, and a spatula shape.

Figure 9 is a flowchart for explaining a method of identifying a pen pressure value according to an embodiment.

According to an embodiment shown in FIG. 9, the processor 130 may display an application screen received from the external device 200 (S910).

Next, the pen pressure value corresponding to the touch input can be identified based on the touch size of the touch input detected by the infrared touch module (S920). For example, as described in FIG. 3D, the size of the touch input can be identified based on the location and/or size of the area where the infrared light is obscured. As an example, information about the pen pressure value corresponding to the touch size may be previously stored in the memory 140 in the form of a look-up table.

The processor 130 may transmit the pen pressure value identified based on the touch size and information on the identified touch input through the infrared touch module to the external device 200 (S930). Here, information about the touch input may include at least one of touch coordinates, touch size, and tilt information. According to one example, the processor 130 may replace the pen pressure value included in the information about the touch input received from the infrared touch module with the pen pressure value identified based on the touch size and transmit it to the external device 200. This is because in the case of the passive electronic pen 10, the infrared touch module does not detect the pen pressure value, so the pen pressure value included in the information about the touch input received from the infrared touch module may be a fixed value.

Afterwards, the processor 130 may receive an application screen including a touch trace identified based on information about the pen pressure value and touch input from the external device 200 (S940) and display the received screen (S950). ).

According to one example, when the first pen pressure value is identified through the pen pressure adjustment UI and the second pen pressure value is identified based on the touch size, the processor 130 selects one of the first pen pressure value and the second pen pressure value It can be transmitted to an external device 200. For example, the processor 130 may select one of the first pen pressure value and the second pen pressure value according to a preset priority. Alternatively, the processor 130 may select a larger value (or smaller value) among the first pen pressure value and the second pen pressure value. Alternatively, the processor 130 may transmit the average value of the first pen pressure value and the second pen pressure value to the external device 200.

According to one example, the processor 130 may adjust the pen pressure value set through the pen pressure adjustment UI based on the touch size and transmit the adjusted pen pressure value to the external device 200. For example, the processor 130 may adjust the pen pressure value by applying the weight identified based on the touch size to the pen pressure value set through the pen pressure adjustment UI. As an example, weights for each touch size may be previously stored in the memory 140 in the form of a lookup table.

According to one example, the processor 130 may apply a first weight to the first pen pressure value set through the pen pressure adjustment UI and apply a second weight to the second pen pressure value identified based on the touch size. The processor 130 may identify a third pen pressure value based on the first pen pressure value to which the first weight is applied and the second pen pressure value to which the second weight is applied and transmit the value to the external device 200 . For example, the first weight and the second weight may be preset or may be varied based on the size, difference, etc. of the first and second pen pressure values. As an example, the processor 130 may identify the average value of the first pen pressure value to which the first weight is applied and the second pen pressure value to which the second weight is applied as the third pen pressure value. As an example, the processor 130 may identify the larger or smaller value of the first pen pressure value to which the first weight is applied and the second pen pressure value to which the second weight is applied as the third pen pressure value.

Figure 10 is a diagram for explaining a method of identifying a pen pressure value according to a touch size according to an embodiment.

According to one example, the pen tip on one side of the passive electronic pen 10 may be implemented as a thin pen tip, and the pen tip on the opposite side may be implemented as a relatively thick pen tip.

According to the embodiment shown in FIG. 10, when a touch input is made with a thin pen tip on one side of the electronic pen 10, a thin touch trace is displayed and a touch trace on the other side of the electronic pen 10, that is, a thick pen, is displayed. When a touch input is made with a tip, a thick touch trace may be displayed. That is, if the electronic pen 10 is a passive type, the pen pressure value can be identified based on the touch size.

However, the embodiment shown in FIG. 10 is only an example, and of course, different pen pressure values can be applied to other input objects such as fingers depending on the touch area.

FIG. 11 is a diagram for explaining operations according to an implementation example of an electronic device according to an embodiment.

According to FIG. 11 , electronic devices 100 and 100' may include a touch module 1110 and SoC 1120. The touch module 1110 is provided on the display 110 and can obtain various information related to touch input. For example, the touch module 1110 may be implemented to detect touch input using an infrared method, but is not limited to this. SoC (System on Chip) is a chip that is responsible for the overall control of the electronic device (100, 100') and may include various electronic components such as a microprocessor, DSP, RAM, and ROM.

The SoC (1120) identifies the pen pressure value set through the pen pressure adjustment UI (520) (S1110). For example, when the position of the indicator included in the pen pressure adjustment bar is set, the pen pressure value corresponding to the position of the indicator can be identified.

The touch module 1110 may identify touch input information including touch coordinates and touch area (or touch size) corresponding to the touch input (S1120) and transmit the touch input information to the SoC 1120 (S1130). Here, the touch input information may further include tilt information and pen pressure information in addition to touch coordinates and touch area. However, the pen pressure information may be a fixed value preset for the passive electronic pen 10.

When touch input information is received from the touch module 1110, the SoC 1120 may transmit the touch input information from the touch module 1110 and the pen pressure value set through the pen pressure adjustment UI 520 to the external device 200 ( S1140). For example, if the touch input information from the touch module 1110 includes a fixed pen pressure value, the corresponding pen pressure value can be replaced with the pen pressure value set through the pen pressure adjustment UI 520 and transmitted to the external device 200. .

Afterwards, the SoC 1120 may display the application screen received from the external device 200, that is, the application screen including the touch trace. In this case, the touch trace included in the application screen may be a touch trace with a thickness corresponding to the pen pressure value set through the pen pressure adjustment UI 520.

Meanwhile, according to one embodiment, the electronic devices 100 and 100' may set the pen pressure value based on a remote control signal, user voice, user motion, etc. For example, when a remote control signal for decreasing or increasing the pen pressure value is received, the electronic device 100 or 100' may provide a guide UI corresponding to the adjusted pen pressure value. In this case, even if the pen pressure adjustment UI is not provided, the user can recognize the set pen pressure value.

At least some of the configurations included in the various embodiments and drawings described above may be combined with at least some of the configurations included in other embodiments and other drawings.

According to the various embodiments described above, it is possible to provide a writing input (or drawing input) with a thickness desired by the user even when using an electronic pen that cannot detect the pen pressure value. For example, in the case of an electronic whiteboard, it is often used by connecting an external device such as a PC in addition to the writing application provided internally. Since it is mainly used for meetings and educational purposes, the frequency of using document work applications by connecting to external devices is very high. These applications often support pen pressure, but since the touch module does not directly measure pen pressure, when using a passive electronic pen, the pen pressure value is fixed to the set value and the user cannot adjust the thickness. In this case, the thickness can be adjusted by adjusting the pen pressure value to the user's desired thickness through the pen pressure adjustment UI. In addition, in the case of the IR touch module, it is possible to distinguish the thickness of an object, so by matching the touch size information with the pen pressure value, touch input is possible with a thin touch when used with a thin pen tip, and with a thick object such as a marker, touch input is possible with a thick touch. Touch input becomes possible.

Meanwhile, the various embodiments described above may also be applied to a projector system. For example, when the pen pressure value is adjusted through the pen pressure adjustment UI, the thickness of the laser point may be adjusted based on the value set through the pen pressure adjustment UI.

Meanwhile, the methods according to the various embodiments described above may be implemented in the form of applications that can be installed on existing electronic devices. Alternatively, at least some of the methods according to various embodiments of the present disclosure described above may be performed using a deep learning-based artificial intelligence model, that is, a learning network model.

Additionally, the methods according to various embodiments of the present disclosure described above may be implemented only by upgrading software or hardware for an existing electronic device.

Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device or an external server of the electronic device.

Meanwhile, according to an example of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media (e.g., a computer). You can. The device is a device capable of calling instructions stored from a storage medium and operating according to the called instructions, and may include an electronic device (eg, electronic device A) according to the disclosed embodiments. When an instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or using other components under the control of the processor. Instructions may contain code generated or executed by a compiler or interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium does not contain signals and is tangible, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

Additionally, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed on a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or online through an application store (e.g. Play Store™). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or created temporarily in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

In addition, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be omitted. Additional components may be included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added. You can.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field pertaining to the disclosure without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: electronic device 110: display
120: communication interface 130: one or more processors

Claims (20)

In an electronic device that receives and displays an application screen from an external device,
A display including a touch module;
communication interface; and
Control the display to display an application screen and a pen pressure adjustment UI received from an external device through the communication interface,
Transmitting information about the pen pressure value set through the pen pressure adjustment UI and the touch input identified through the touch module to the external device,
Receiving an application screen including a touch trace identified based on the pen pressure value and information about the touch input from the external device through the communication interface,
An electronic device comprising: one or more processors controlling the display to display the received application screen. According to paragraph 1,
The pen pressure adjustment UI is,
Includes a pen pressure adjustment bar that can adjust the size of the pen pressure value,
The one or more processors:
Transmitting information including the pen pressure value set by the pen pressure adjustment bar, the touch coordinates and touch size of the touch input identified through the touch module to the external device,
An electronic device that receives an application screen including a touch trace with a thickness corresponding to the pen pressure value from the external device. According to paragraph 1,
The one or more processors:
An electronic device that controls the display to display a guide UI including thickness information corresponding to the pen pressure value set through the pen pressure adjustment UI. According to paragraph 3,
It further includes a memory that stores thickness information according to the pen pressure value corresponding to each of the plurality of applications,
The one or more processors:
An electronic device that controls the display to display the guide UI including thickness information of a touch trace corresponding to each of the plurality of applications based on the set pen pressure value and information stored in the memory. According to paragraph 1,
The one or more processors:
After the pen pressure value is set through the pen pressure adjustment UI, when a touch input is identified on an area separate from the application screen, a touch trace corresponding to the touch input is displayed based on thickness information identified based on the set pen pressure value. An electronic device that controls the display. According to paragraph 1,
The touch module is,
Contains an infrared touch module,
The one or more processors:
Identifying a pen pressure value corresponding to the touch input based on the touch size of the touch input detected by the infrared touch module,
An electronic device that transmits at least one of a pen pressure value set through the pen pressure adjustment UI or a pen pressure value identified based on the touch size to the external device. According to clause 6,
The one or more processors:
An electronic device that adjusts a pen pressure value set through the pen pressure adjustment UI based on the touch size and transmits the adjusted pen pressure value to the external device. According to clause 6,
The one or more processors:
Applying a first weight to the first pen pressure value set through the pen pressure adjustment UI, and applying a second weight to the second pen pressure value identified based on the touch size,
Identifying a third pen pressure value based on the first pen pressure value to which the first weight is applied and the second pen pressure value to which the second weight is applied,
An electronic device that transmits the identified third pen pressure value to the external device. According to paragraph 1,
The one or more processors:
When the application screen is provided to a partial area of the display, the pen pressure adjustment UI is provided to the remaining area of the display,
When the application screen is provided in the entire area of the display, the electronic device provides the pen pressure adjustment UI as a pop-up screen in one area of the application screen. According to paragraph 1,
The touch input is,
An electronic device, input by a passive input device. In a control method of an electronic device that receives and displays an application screen from an external device,
Displaying an application screen and pen pressure adjustment UI received from an external device;
Transmitting information about the pen pressure value set through the pen pressure adjustment UI and the touch input identified through the touch module to the external device;
Receiving an application screen including a touch trace identified based on the pen pressure value and information about the touch input from the external device; and
A control method comprising: displaying the received application screen. According to clause 11,
The pen pressure adjustment UI is,
Includes a pen pressure adjustment bar that can adjust the size of the pen pressure value,
The step of transmitting to the external device is,
Transmitting information including the pen pressure value set by the pen pressure adjustment bar, the touch coordinates and touch size of the touch input identified through the touch module to the external device,
The step of receiving the application screen is,
A control method for receiving an application screen including a touch trace with a thickness corresponding to the pen pressure value from the external device. According to clause 11,
A control method further comprising displaying a guide UI including thickness information corresponding to a pen pressure value set through the pen pressure adjustment UI. According to clause 13,
The step of displaying the guide UI is,
A control method comprising: displaying the guide UI including thickness information of a touch trace corresponding to each of the plurality of applications based on the set pen pressure value and thickness information according to the pen pressure value corresponding to each of the plurality of applications. . According to clause 11,
After the pen pressure value is set through the pen pressure adjustment UI, when a touch input is identified on an area separate from the application screen, a touch trace corresponding to the touch input is displayed based on the thickness information identified based on the set pen pressure value. A control method further comprising: According to clause 11,
The touch module is,
Contains an infrared touch module,
Further comprising: identifying a pen pressure value corresponding to the touch input based on the touch size of the touch input detected by the infrared touch module,
The step of transmitting to the external device is,
A control method for transmitting at least one of a pen pressure value set through the pen pressure adjustment UI or a pen pressure value identified based on the touch size to the external device. According to clause 16,
The step of transmitting to the external device is,
A control method for adjusting the pen pressure value set through the pen pressure adjustment UI based on the touch size and transmitting the adjusted pen pressure value to the external device. According to clause 16,
The step of transmitting to the external device is,
Applying a first weight to a first pen pressure value set through the pen pressure adjustment UI and applying a second weight to a second pen pressure value identified based on the touch size;
Identifying a third pen pressure value based on the first pen pressure value to which the first weight is applied and the second pen pressure value to which the second weight is applied; and
A control method comprising: transmitting the identified third pen pressure value to the external device. According to clause 11,
The step of displaying the pen pressure adjustment UI is,
When the application screen is provided to a partial area of the display, the pen pressure adjustment UI is provided to the remaining area of the display,
When the application screen is provided in the entire area of the display, the control method provides the pen pressure adjustment UI as a pop-up screen in one area of the application screen. According to clause 11,
The touch input is,
A control method entered by a passive input device.

Images