KR20230105601A - Electronic apparatus and control method thereof - Google Patents

Abstract

An electronic device is disclosed. The electronic device includes a display, a sensor provided in the display, a memory for storing at least one command, and a processor connected to the display, the sensor, and the memory to control the electronic device. The processor executes at least one command, When the position of the electronic pen is identified on the display by the sensor, a slide bar displayed on the display is identified based on the identified position, and information about a rolling gesture based on the vertical axis of the electronic pen is obtained. Obtain, identify the variation coefficient of the slide bar and the enforcement coefficient of the rolling gesture, and control the slide bar based on the information about the variation coefficient of the slide bar, the application coefficient of the rolling gesture, and the rolling gesture. can

Description

Electronic apparatus and its control method {Electronic apparatus and control method thereof}

The present invention relates to an electronic device and a control method thereof, and more particularly, to an electronic device receiving an electronic pen input and a control method thereof.

Thanks to the development of electronic technology, various types of electronic devices are being developed and supplied.

In particular, in recent years, the spread of smart phones or tablet PCs that can be carried by users is actively progressing. A smart phone or tablet PC mainly has a touch screen, and a user can control functions of an electronic device using the touch screen.

A user may touch the touch screen not only with his or her body (eg, a finger) but also with a pen-shaped input device. The electronic device may perform different control operations according to touch coordinates of a point where the body or an input device touches the touch screen and a menu (or icon) displayed on the touch coordinates. However, there was a problem that only the touch function of the electronic pen was utilized.

An object of the present disclosure is to provide an electronic device for adjusting a slide bar displayed on a screen through a rolling gesture of an electronic pen and a method for controlling the same.

An electronic device according to an embodiment of the present invention for achieving the above object includes a display, a sensor provided in the display, a memory for storing at least one command, and a display, the sensor, and the memory connected to the electronic device. and a processor for controlling the device, wherein the processor executes the at least one instruction and, when the position of the electronic pen is identified on the display by the sensor, the slide bar displayed on the display based on the identified position. A slide bar is identified, information on a rolling gesture based on a vertical axis of the electronic pen is obtained, and a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture are obtained. ) is identified, and the slide bar may be controlled based on a variation coefficient of the slide bar, an application coefficient of the rolling gesture, and information on the rolling gesture.

Meanwhile, a control method of an electronic device according to an embodiment includes identifying a slide bar displayed on the display based on the identified position when the position of the electronic pen is identified on the display; Acquiring information about a rolling gesture based on a vertical axis, identifying a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture, and and controlling the slide bar based on a variation coefficient, an application coefficient of the rolling gesture, and information on the rolling gesture.

According to various embodiments as described above, the user UX of the control menu can be improved without touching the screen by controlling the screen through the rolling gesture of the electronic pen. In addition, it is possible to reduce the user interaction time required to search and adjust the menu. In addition, the slide bar can be easily and flexibly adjusted through a rolling gesture of the electronic pen.

1A and 1B are diagrams for explaining schematic operations of an electronic device and an electronic pen according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.
3A to 3C are diagrams for explaining a method of detecting an electronic pen according to an exemplary embodiment.
4 is a diagram for explaining a method for determining a rolling gesture according to an exemplary embodiment.
5 is a diagram for explaining a method of adjusting a slide bar according to a rolling gesture according to an exemplary embodiment.
6A and 6B are diagrams for explaining a method of adjusting a slide bar according to a rolling gesture according to an exemplary embodiment.
7 is a diagram for explaining a method of adjusting a slide bar for each area according to an exemplary embodiment.
8A and 8B are diagrams for explaining a method for adjusting a pen style according to an exemplary embodiment.
9 is a diagram for explaining a method of determining a variation coefficient of a slide bar according to an exemplary embodiment.
10 is a diagram for explaining a method of enlarging and reducing an image using a rolling gesture according to an exemplary embodiment.
11A and 11B are views for explaining a method of adjusting an object using a rolling gesture according to an exemplary embodiment.
12 is a block diagram illustrating an implementation example of an electronic device according to an exemplary embodiment.
13 is a block diagram illustrating an implementation example of an electronic pen according to an exemplary embodiment.
14 is a flowchart illustrating a method of controlling an electronic device according to an exemplary embodiment.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. . In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the disclosure. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

In this specification, expressions such as “has,” “can have,” “includes,” or “can include” indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

The expression at least one of A and/or B should be understood to denote either "A" or "B" or "A and B".

Expressions such as "first," "second," "first," or "second," as used herein, may modify various components regardless of order and/or importance, and may refer to one component It is used only to distinguish it from other components and does not limit the corresponding components.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that an element may be directly connected to another element, or may be connected through another element (eg, a third element).

The expression “configured to (or configured to)” as used in this disclosure means, depending on the situation, for example, “suitable for,” “having the capacity to.” ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "consist of" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented in hardware or software or a combination of hardware and software. In addition, a plurality of "modules" or a plurality of "units" are integrated into at least one module and implemented by at least one processor (not shown), except for "modules" or "units" that need to be implemented with specific hardware. It can be.

Hereinafter, an embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings.

1A and 1B are diagrams for explaining schematic operations of an electronic device and an electronic pen according to an embodiment of the present invention.

Referring to FIG. 1A , the electronic pen 200 according to an embodiment of the present invention has a constant length and may be implemented in a thin and long pen shape for a user to hold comfortably. Accordingly, the electronic pen 200 may be referred to by various terms such as a pen type input device, a stylus pen, an S-pen, etc., but does not necessarily have to be implemented in a pen shape. For example, it may be implemented to have a blunt or flat body.

The electronic pen 200 may be implemented in an active manner. In the active method, a battery is built into the electronic pen 200 to generate a unique signal, a sensor for detecting the electronic pen 200 is added to the electronic device 100, and the electronic device 100 operates the electronic device 100. This is a method for recognizing the pen 200 . In this case, by distinguishing between the electronic pen 200 and a finger touch or distinguishing and recognizing each input device, various operations can be performed by going back and forth between the finger and the pen.

The electronic device 100 is implemented as an input panel such as a touch panel or a touch screen, or includes a touch panel or a touch screen, a laptop computer, a laptop computer, a mobile phone, a smart phone, an electronic blackboard, an electronic photo frame, a digital information display (DID), or a kiosk. (kiosk), PMP (Portable Media Player), MP3 player, game console, LFD (Large Format Display), laptop (notebook), laptop (laptop), TV, monitor (monitor), projector system, etc. can However, it is not limited thereto, and other home appliances, medical devices, and wearable devices may also be included in the electronic device 100 .

A capacitive touch panel according to an embodiment may have an electrode pattern structure as shown in FIG. 1B. The grid-like electrode pattern shown in FIG. 1B includes vertical lines 211 and horizontal lines 212-1, 212-2, and 212-3. This method is divided into a transmitter that transmits an electric field and a receiver that detects a change in the electric field, and can detect a coordinate by detecting a change in the electric field caused by contact with a conductor. Specifically, when an electric field is output from the electronic pen 200 itself, a change in the electric field is sensed in the electrode pattern. If a finger touch is to be sensed, among electrodes in two directions crossing each other, a method in which an electrode in a first direction is a transmitting unit and an electrode in a second direction is a receiving unit may be used.

The electronic device 100 may perform different control operations according to signals output from the electronic pen 200 . For example, when the user touches the electronic pen 200 to the touch screen of the electronic device 100 and moves the touch point, the electronic device 100 displays a line, that is, a writing trajectory according to the moving trajectory. do. In this state, it is possible to adjust the thickness, brightness, size, color, saturation, font, shape, etc. of the writing trajectory according to the user's manipulation.

Meanwhile, according to an embodiment, the electronic device 100 may recognize a rolling gesture of the electronic pen 200 and perform a control operation corresponding thereto.

Hereinafter, various embodiments in which the electronic device 100 recognizes the rolling gesture of the electronic pen 200 and performs a control operation based on the recognition will be described.

2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.

According to FIG. 2 , the electronic device 100 includes a display 110 , a sensor 120 , a memory 130 and a processor 140 .

The display 110 may be implemented as a display including a self-light emitting element or a display including a non-light emitting element and a backlight. For example, LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, LED (Light Emitting Diodes), micro LED (micro LED), Mini LED, PDP (Plasma Display Panel), QD (Quantum dot) display , QLED (Quantum dot light-emitting diodes), etc. can be implemented in various types of displays. The display 110 may also include a driving circuit, a backlight unit, and the like that may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT). Meanwhile, the display 110 may be implemented as a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, and the like.

The sensor 120 may be provided in the display 110 . According to an example, the sensor 120 may be implemented as a touch sensor or a proximity sensor. For example, the touch sensor may be implemented as a capacitive type or a resistive type. A capacitive touch sensor refers to a sensor that calculates touch coordinates by detecting microelectricity excited by a user's body when a part of the user's body touches the display surface using a dielectric coated on the display surface. The resistive touch sensor refers to a type of touch sensor that includes two built-in electrode plates and calculates touch coordinates by detecting current flowing as the top and bottom plates are in contact with each other at a touched point. In addition, an infrared sensing method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, and the like may be used to detect a touch manipulation. In addition, the sensor 120 may determine whether or not a touch object such as a finger or a stylus pen has come into contact with or approached by using a magnetic and magnetic field sensor, an optical sensor, or a proximity sensor.

According to another example, the sensor 120 may be implemented to receive an optical input such as a laser. For example, the sensor 120 may be implemented to include an optical sensor, a laser detection sensor, and the like.

The memory 130 may store data necessary for various embodiments. The memory 130 may be implemented in the form of a memory embedded in the electronic device 100 or in the form of a removable memory in the electronic device 100 according to a data storage purpose. For example, data for driving the electronic device 100 is stored in a memory embedded in the electronic device 100, and data for an extended function of the robot 100 is detachable from the electronic device 100. can be stored in memory. On the other hand, in the case of memory embedded in the electronic device 100, volatile memory (eg, DRAM (dynamic RAM), SRAM (static RAM), SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (non-volatile memory) ( Examples: OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash, etc.) ), a hard drive, or a solid state drive (SSD). In addition, in the case of a memory attachable to and detachable from the robot 100, a memory card (eg, compact flash (CF)) , SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to the USB port (eg For example, a USB memory) may be implemented in the form of the like.

At least one processor 140 controls the overall operation of the electronic device 100 . Specifically, the processor 140 may be connected to each component of the electronic device 100 to control the overall operation of the electronic device 100 . For example, the processor 140 may be electrically connected to the display 110 and the memory 130 to control overall operations of the electronic device 1100. Processor 140 may be composed of one or a plurality of processors.

The processor 140 may perform the operation of the electronic device 100 according to various embodiments by executing at least one instruction stored in the memory 130 .

According to an embodiment, the processor 140 may include a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, and a neural processing unit (NPU) for processing digital image signals. Processing Unit), time controller (TCON), but is not limited to, central processing unit (CPU), micro controller unit (MCU), micro processing unit (MPU), controller (controller), an application processor (AP), a communication processor (CP), or one or more of an ARM processor, or may be defined by the term. In addition, the processor 140 may be implemented in the form of a system on chip (SoC) with a built-in processing algorithm, large scale integration (LSI), application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

The processor 140 may recognize a rolling gesture of the electronic pen 200 . Here, the rolling gesture may be a gesture in which rolling is performed at a threshold angle or more. In this case, the critical angle may be a predetermined angle based on the characteristics of at least one of the electronic device 100 and the electronic pen 200, but is not limited thereto. For example, it may be changed based on characteristics of the slide bar, rolling history, or the like, or may be set by the user.

Meanwhile, the rolling gesture may be received in various ways, such as a proximity input, a laser pointer input, and the like, as well as a touch input as described above.

According to an embodiment, when the position of the electronic pen 200 is identified on the display 110 by the sensor 120, the processor 140 displays a slide bar on the display 110 based on the identified position. can identify. According to an example, a slide bar within a critical distance from the position of the electronic pen 200 may be identified. According to another example, when there is only one slide bar displayed on the display 100, the corresponding slide bar may be identified regardless of a threshold distance. According to another example, when there are a plurality of slide bars displayed on the display 110, a slide bar with the closest distance may be identified.

Next, when the rolling gesture of the electronic pen 200 is recognized, the processor 140 may obtain information about the rolling gesture based on the vertical axis of the electronic pen 200 . According to an example, the processor 140 may identify a rolling gesture of the electronic pen 200 based on sensing information received from the electronic pen 200 . According to another example, the processor 140 may identify a rolling gesture of the electronic pen 200 based on a change in touch coordinates (or hovering coordinates) recognized by the sensor 120 . According to another example, it is also possible to identify a rolling gesture of the electronic pen 200 based on an image captured by a camera built into the display 110.

Here, the information on the rolling gesture may include at least one of a rolling angle, a rolling direction, and a rolling speed based on the vertical axis of the electronic pen 200 . However, it is not limited thereto, and other information such as the inclination of the electronic pen 200 may also be included in the rolling gesture information.

Subsequently, the processor 140 may identify a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture.

Here, the coefficient of variation of the slide bar may be information about a basic movement unit for adjusting the movement of the slide bar. Also, the application coefficient of the rolling gesture may be information about a basic rolling angle of the rolling gesture corresponding to the basic movement unit of the slide bar.

According to an embodiment, the processor 140 may determine the variation coefficient c of the slide bar based on the movement range of the slide bar. Here, the range of movement of the slide bar may be different depending on the characteristics of the slide bar (eg, a slide bar for adjusting a web page, a slide bar for adjusting a playback time of a video, etc.).

According to an example, the processor 140 may determine a value corresponding to a quotient obtained by dividing the movement range of the slide bar by equal values as a coefficient of variation. For example, when the movement range of the slide bar is between 0 and 10, the coefficient of variation may be determined in units of 1. For example, when the slide bar is a video processing bar, the coefficient of variation according to the length of the video may be 10 seconds for a 10-minute video, but is not limited thereto.

For example, when a slide bar is provided on the execution screen of an application, the processor 140 determines the type of application (eg, a video playback application, a still image playback application, a social application, etc.) or a slide bar provided on the execution screen. The movement range of the slide bar is identified based on at least one of the types (video, still image, chat window, shopping window, word file, PDF file, etc.), and a coefficient of variation of the slide bar is identified based on the movement range of the slide bar. can

In this case, the processor 140 may increase or decrease the slide bar by a variation coefficient (eg, 1 unit) whenever the application coefficient of the rolling gesture is achieved.

According to an example, the processor 140 may identify an application coefficient θ of the rolling gesture based on an angle of the rolling gesture. Here, the application coefficient of the rolling gesture based on the rolling angle may be an angular unit. For example, the application coefficient of the rolling gesture may be 10 degrees, but is not limited thereto.

According to another embodiment, the processor 140 may obtain an application coefficient when the rolling angle of the rolling gesture is greater than or equal to a predetermined angle for a predetermined time. For example, an application coefficient may be obtained when the rolling angle is 10 degrees or more for 1 second. For example, the application coefficient may be obtained as a value of 1 when the rolling angle is equal to or greater than a predetermined angle for each period, and the application coefficient may be obtained as a value of 0 when the rolling angle is less than the predetermined angle.

Then, the processor 140 may control the slide bar based on the variation coefficient of the slide bar, the application coefficient of the rolling gesture, and information about the rolling gesture.

According to an embodiment, the processor 140 may control the movement of the slide bar by the basic movement unit corresponding to the variation coefficient of the slide bar based on the number of times the rolling gesture of the basic rolling angle is identified.

According to an embodiment, the processor 140 may differently control the movement value of the slide bar based on the rolling speed of the rolling gesture. For example, if the rolling speed of the rolling gesture is within a threshold range, the processor 140 may control the slide bar to correspond to the basic variation coefficient of the slide bar.

Also, when the rolling speed of the rolling gesture exceeds a threshold range, the processor 140 may control the slide bar based on a value greater than a variation coefficient of the slide bar. That is, the processor 140 may increase the coefficient of variation of the slide bar from the basic coefficient of variation.

Also, if the rolling speed of the rolling gesture is less than the critical range, the processor 140 may control the slide bar based on a value smaller than the variation coefficient of the slide bar. That is, the processor 140 may reduce the coefficient of variation of the slide bar from the basic coefficient of variation.

According to an embodiment, when a rolling gesture for an image displayed on the display 110 is input, the processor 140 may enlarge or reduce the image based on the rolling direction of the input rolling gesture and display the image. For example, the processor 140 may enlarge the image when the rolling direction of the rolling gesture is clockwise and reduce the image when the rolling direction of the rolling gesture is counterclockwise. In this case, the processor 140 can, of course, adjust the amount of enlargement/reduction and the amount of enlargement/reduction based on at least one of the rolling angle and the rolling speed in the same principle as the control of the slide bar.

According to an embodiment, when a rolling gesture for a specific object displayed on the display 110 is input, the processor 140 may adjust attributes of the specific object in ascending or descending order based on the rolling direction of the input rolling gesture, where , a specific object may be an object having properties that can be adjusted in ascending or descending order, such as setting values and font sizes. For example, if the rolling direction of the rolling gesture is clockwise, the processor 140 may adjust the properties of the object in an ascending direction, and if the rolling direction of the rolling gesture is in a counterclockwise direction, the properties of the object may be adjusted in a descending direction.

According to an embodiment, when a rolling gesture is input on the drawing screen, the processor 140 may adjust pen style information of the electronic pen 200 based on the rolling gesture. For example, the processor 140 may change various properties related to the pen style, such as pen thickness, pen type, and pen color, according to the rolling direction.

According to an embodiment, the processor 140 may perform different functions based on tilt information or a tilt direction of the electronic pen 200 while performing a rolling gesture.

In addition to the above-described embodiment, various adjustments may be possible according to the rolling gesture. For example, upon inputting a rolling gesture for a line, it may be possible to adjust a line type (solid line, dotted line, etc.), line thickness, or the like, or to change a keyboard language or symbol type.

3A to 3C are diagrams for explaining a method of detecting an electronic pen according to an exemplary embodiment.

In FIGS. 3A to 3C , it is assumed that three slide bars 310 , 320 , and 330 are displayed on the display 110 .

As shown in FIG. 3A, the processor 140 detects a touch of the electronic pen 200 on a specific slide bar 320 among the number of slide bars 310, 320, and 330 displayed on the screen, and the user selects the slide bar ( 320) can be judged to be preparing to adjust. Alternatively, as shown in FIG. 3B , the processor 140 may determine that the user is preparing to adjust the slide bar 320 when the distance d1 between the electronic pen 200 and the screen is equal to or less than a predetermined distance. However, as shown in FIG. 3C , the processor 140 may determine that the adjustment of the slide bar 320 is irrelevant when the distance d2 between the electronic pen 200 and the screen exceeds a predetermined distance.

As described above, the processor 140 may determine whether the user is preparing to adjust the slide bar 320 based on the position of the electronic pen 200 and the distance from the screen, and may perform a necessary operation. For example, the processor 140 may activate a communication module with the electronic pen 200 (when the communication module is inactive) or turn on a camera to capture a rolling gesture of the electronic pen 200 .

4 is a diagram for explaining a method for determining a rolling gesture according to an exemplary embodiment.

As shown in FIG. 4A, when the electronic pen 200 is rolled by the user, the sensor inside the electronic pen 200 measures the angle corresponding to the three axes (x-axis, y-axis, z-axis) of the electronic pen 200. changes can be detected. Here, a change in angle corresponding to the z-axis may be a rolling gesture applied to control the slide bar.

According to an example, all changes in angles corresponding to three axes (x-axis, y-axis, and z-axis) of the electronic pen 200 may be considered in order to be distinguished from a user's arbitrary action with respect to the electronic pen 200 .

For example, as shown in FIG. 4B , angles corresponding to the x-axis and y-axis of the electronic pen 200 are detected as less than a threshold value, that is, angles corresponding to the x-axis and y-axis are not significantly changed and at the same time An action that causes a large change in the z-axis may be determined as a rolling gesture. That is, as shown in FIG. 4C , an action in which an angle corresponding to the x-axis and the y-axis is significantly changed beyond a threshold value may be determined as not being a rolling gesture.

However, even in the case shown in FIG. 4B , the rolling gesture may be determined as a rolling gesture only when the rolling angle is greater than or equal to a preset angle, but is not necessarily limited thereto.

5 is a diagram for explaining a method of adjusting a slide bar according to a rolling gesture according to an exemplary embodiment.

As shown in FIG. 5 , the slide bar 510 displayed on the electronic device 100 may be adjusted based on the rolling angle of the electronic pen 200 .

According to an example, the processor 140 may adjust the slide bar 510 based on the rolling angle when the electronic pen 200 rolls at a critical angle α or more with respect to the z-axis. For example, if the roll is greater than the threshold angle α, it is determined that there is a rolling gesture according to an embodiment, and the slide bar 510 may be adjusted to correspond to the angle α′ exceeding the threshold angle α. there is. For example, a movement value d of the slide bar 510 corresponding to the angle α′ may be identified, and the slide bar may be adjusted based on the identified movement value d. Here, the movement value d of the slide bar 510 corresponding to the angle α' is the coefficient of variation of the slide bar as described above, the motion coefficient of the rolling gesture, and information about the rolling gesture (for example, rolling speed). can be determined based on

6A and 6B are diagrams for explaining a method of adjusting a slide bar according to a rolling gesture according to an exemplary embodiment.

As shown in FIG. 6A, the processor 140 determines whether the electronic pen 200 is additionally rolled to correspond to the motion coefficient α1 of the rolling gesture when the electronic pen 200 is rolled by the threshold angle α at a specific time point t1. can identify. When the electronic pen 200 is additionally rolled by the motion coefficient α1 of the rolling gesture and the rolling speed is v1, which is within a threshold range, the processor 140 may adjust the slide bar 510 by the variation coefficient d1 of the surface slide bar. .

Meanwhile, according to an embodiment, the processor 140 may differently adjust the coefficient of variation of the slide bar 510 when the rolling speed is different even though the rolling angle of the rolling gesture is the same. According to an example, the processor 140 may control the slide bar 510 to increase or decrease a value larger than the basic variation coefficient when the rolling speed is greater than the threshold speed. Also, when the rolling speed is smaller than the critical speed, the slide bar 510 may be controlled to increase or decrease a value smaller than the basic coefficient of variation.

For example, as shown in FIG. 6B , at time t2 when the slide bar 510 is adjusted by the variation coefficient d1, the electronic pen 200 is rolled by the critical angle α, and α2 equal to the motion coefficient α1 of the rolling gesture. When additionally rolled as much as possible, the coefficient of variation may be adjusted based on the rolling speed. That is, as shown, when the rolling speed is v2 greater than the critical range to which v1 belongs, the slide bar 510 may be adjusted by increasing the coefficient of variation of the slide bar from d1 to d2. Here, d2 may be determined based on the ratio of v2 to the rolling speed v1. For example, d2 may be determined by multiplying d1 by a ratio equal to the ratio of v2 to the rolling speed v1, but is not limited thereto.

According to an example, the processor 140 may adjust the movement value D of the slide bar based on Equation 1 below, but the equation is not necessarily limited thereto.

Here, C is a variation coefficient of the slide bar, α is a rolling angle that is greater than or equal to a critical rolling angle, θ is a motion coefficient of a rolling gesture, v _pre is a preset speed, and v _average is an average rolling speed.

7 is a diagram for explaining a method of adjusting a slide bar for each area according to an exemplary embodiment.

According to an embodiment, it is assumed that the user performs a rolling operation while placing the electronic pen 200 at an arbitrary position in a corresponding area without accurately positioning the electronic pen 200 at the position of the slide bar. Even in this case, the slide bar corresponding to the corresponding area can be controlled through the rolling gesture of the electronic pen 200 .

For example, as shown in FIG. 7 , when a rolling gesture of the electronic pen 200 is recognized at an arbitrary position within a plurality of web view areas 710, 720, and 730 in a web browser, the corresponding web view area 710 , 720, 730) can be scrolled up or down. In this case, an area corresponding to a specific slide bar is an area affected when the corresponding slide bar is adjusted, and may be implemented so as not to be influenced by other slide bars.

8A and 8B are diagrams for explaining a method for adjusting a pen style according to an exemplary embodiment.

8A illustrates a case in which a corresponding UI 820 is opened by selecting a setting button 810 to adjust a pen style on a drawing screen (or drawing view screen) according to a related art. However, according to an embodiment, the pen style of the electronic pen 200 may be adjusted through a rolling gesture of the electronic pen 200 on the drawing screen. In this case, as shown in FIG. 8B , the processor 140 may display the slide bar 830 and display pen style information according to the adjustment of the slide bar together. Meanwhile, the processor 140 may change various properties related to the pen style, such as pen thickness, pen type, and pen color, according to the rolling direction.

9 is a diagram for explaining a method of determining a variation coefficient of a slide bar according to an exemplary embodiment.

According to an embodiment, when the range of the slide bar cannot be determined based on predefined information, the processor 140 may determine the coefficient of variation of the slide bar based on the starting point, the ending point, and the location of the pointer.

For example, as shown in FIG. 9 , the processor 140 displays the coordinates of the starting point (x1, y), the coordinates of the ending point (x2, y), and the coordinates of the pointer (xn) of the slide bar 910 displayed on the display 110. , y), the coefficient of variation of the slide bar 910 may be determined. For example, the variation coefficient (c) of the slide bar 910 may be determined as (x2-x1)/a. Here, the value “a” may be determined based on the type of application, the type of adjustment target provided on the screen, and the like. For example, a video application may have a relatively smaller value of “a” than a setting application.

On the other hand, when the application count of the rolling gesture is achieved, the processor 140 moves from the current position (xp, y) of the pointer to the next position (sp+(x2-x1)/a, y) of the pointer as shown in the last figure. You can adjust the slide bar by sending swipe events to your application.

10 is a diagram for explaining a method of enlarging and reducing an image using a rolling gesture according to an exemplary embodiment.

According to an embodiment, an image/video, a live view image, or the like may be enlarged or reduced through a rolling gesture of the electronic pen 200 .

For example, as shown in FIG. 10 , when a rolling gesture of the electronic pen 200 is input at an arbitrary position on an image displayed on the screen, the image is enlarged or scaled based on the rolling direction, rolling speed, rolling angle, etc. of the rolling gesture. can be scaled down According to this embodiment, when taking a picture or a video using a camera, the user can enlarge or reduce the live view by rolling the pen without touching the screen.

11A and 11B are diagrams for explaining a method of adjusting an object using a rolling gesture according to an exemplary embodiment.

According to an embodiment, properties of various objects may be adjusted through a rolling gesture of the electronic pen 200 . For example, when a rolling gesture of the electronic pen 200 for a specific object is input, the processor 130 may adjust properties of the object in an ascending or descending order.

For example, as shown in FIG. 11A , if a rolling gesture is input after text is selected, the processor 130 may increase or decrease the font size of the selected text based on the rolling gesture.

As another example, as shown in FIG. 11B , if a rolling gesture is input after an object such as a timer or a date included in a box is selected, the processor 130 may increase or decrease the number of the selected object based on the rolling gesture. there is.

12 is a block diagram illustrating an implementation example of an electronic device according to an exemplary embodiment.

12, an electronic device 100' includes a display 110, a sensor 120, a memory 130, a processor 140, a communication interface 150, a user interface 160, and a speaker 170. can do. Among the components shown in FIG. 12 , detailed descriptions of components overlapping those shown in FIG. 2 will be omitted.

Of course, the communication interface 150 may be implemented as various interfaces according to the implementation example of the electronic device 100'. For example, the communication interface 150 includes Bluetooth, AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Zigbee, wired/wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/ European Broadcasting Union), Optical Communication may be performed with an external device, an external storage medium (eg, a USB memory), an external server (eg, a web hard drive), etc. through a communication method such as , coaxial, or the like. According to an example, the communication interface 150 may communicate with the electronic pen 200 through a wireless communication method.

The user interface 160 may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or may be implemented as a touch screen capable of performing the above-described display function and manipulation input function together, or a remote control transceiver. The remote control transmitting/receiving unit may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one of infrared communication, Bluetooth communication, and Wi-Fi communication.

The speaker 170 may be configured to output not only various kinds of audio data, but also various notification sounds or voice messages. The processor 140 may control the speaker 170 to output information corresponding to the UI screen or various notifications in the form of audio according to various embodiments of the present disclosure.

A microphone (not shown) is a component for receiving a user's voice or other sounds and converting them into audio data. However, according to another embodiment, the electronic device 100' may receive the user's voice input through the external device through the communication interface 150.

A camera (not shown) may be turned on according to a predetermined event to perform photographing. A camera (not shown) may convert a captured image into an electrical signal and generate image data based on the converted signal. For example, an object may be converted into an electrical image signal through a charge coupled device (CCD), and the image signal thus converted may be amplified and converted into a digital signal and then signal processed.

Meanwhile, various embodiments described above may be provided through a specific application communicating with the electronic device 100, but are not necessarily limited thereto. According to an example, the specific application may be implemented as an application communicating with the electronic device 100 through a server (not shown) or an application directly communicating with the electronic device 100 . The application may be a type of software directly used by the user on the OS. The application may be pre-installed in the electronic device 100' or may be downloaded from a server (not shown), and may be provided on the screen in the form of an application UI (hereinafter App UI), for example, an icon interface.

13 is a block diagram illustrating an implementation example of an electronic pen according to an exemplary embodiment.

According to FIG. 13 , the electronic pen 200 includes a pen tip 210, a communication interface 220, an electrode unit 230, a sensor 240, and a processor 250.

According to an embodiment, the electronic pen 200 may include a straight body suitable for a user to hold and an input unit coupled to one end thereof. For example, the input unit may be implemented in a conical shape as shown, and the pen tip 210 may be exposed at one end. However, the body part and the input part are separated for convenience of description, and it goes without saying that the input part can be implemented as a part of the body part. That is, the input unit and the body unit may be implemented as one main body without distinction.

The pen tip 210 may be formed at one end of the electronic pen 200 . The pen tip 210 may be formed of, for example, a conductive metal tip. The pen tip 210 may exist inside a non-conductive material or may be implemented in a form in which a part of the pen tip 210 is exposed to the outside. In addition, an insulating material that prevents the pen tip 210 from directly contacting the touch screen of the electronic device 100 may be further included to soften the writing feeling during use. The insulating material may be, for example, rubber, plastic, or glass.

The communication interface 220 communicates with the electronic device 100 . The communication interface 220 may include at least one of a short-distance communication module, a wireless LAN communication module, and a wired Ethernet module according to the performance and structure of the electronic pen 200 . For example, it may be implemented as a communication module such as an infrared communication module or a Bluetooth communication module. For example, the communication module may be provided in the pen tip 210.

According to an embodiment, a separate communication interface 220 may not be provided and the pen tip 210 may be implemented to perform the corresponding function. For example, the pen tip 210 may serve as a transmitting electrode. When the pen tip 210 is in contact with or hovering on the touch panel of the electronic device 100, the conductive pen tip 210 is coupled to the touch panel at the point of contact (or proximity) and touches A signal can be transmitted to the point of contact (or proximity) of the panel.

The electrode unit 230 is formed in an input unit separated from the pen tip 210 by a predetermined distance. The electrode unit 230 may be implemented in various shapes within a range in which it can operate as the electronic device 100 and a capacitor. According to an embodiment, the electrode unit 230 may be formed in the form of an electrode ring or an electrode plate. In addition, various electrode materials may be used as the electrode unit 230, and may be implemented with, for example, indium tin oxide (ITO). According to an example, the processor 240 may apply a preset frequency signal to the electrode unit 230 and output a signal whose frequency is changed according to a capacitance formed between the electrode unit 230 and the electronic device 100 .

The sensor 240 may obtain rolling information of the electronic pen 200 . According to an example, the sensor 240 may be implemented with various types of sensors capable of detecting the rolling of the electronic pen 200, such as an acceleration sensor and a geomagnetic sensor.

The processor 250 is electrically connected to the pen tip 210, the communication interface 220, the electrode unit 230, and the sensor 240 to control the operation of the electronic pen 200. According to an example, the processor 250 may be implemented as a digital signal processor (DSP). However, depending on the implementation form of the electronic pen 200, it may be implemented in the form of a field programmable gate array (FPGA) as well as a DSP. Alternatively, the processor 140 may include a central processing unit (CPU), a micro controller unit (MCU) controller, an application processor (AP), or a communication processor (CP), It may include, or be defined in terms of, one or more of the ARM processors.

The processor 250 may control the communication interface 220 to transmit the rolling information acquired through the sensor 240 to the electronic device 100 .

According to an example, the processor 250 transmits the sensing value (eg, acceleration value) acquired through the sensor 240 to the electronic device 100 as it is, or information obtained from the sensing value (eg, rolling information). ) may be transmitted to the electronic device 100.

In addition, the processor 250 transmits the sensed value (eg, capacitance variation) sensed through the electrode unit 230 to the electronic device 100, or transmits information (eg, slope information) obtained from the sensed value to the electronic device 100. It can be transmitted to the device 100.

14 is a flowchart illustrating a method of controlling an electronic device according to an exemplary embodiment.

According to the control method of the electronic devices 110 and 100' shown in FIG. 14, when the position of the electronic pen 200 is identified on the display, a slide bar displayed on the display is identified based on the identified position. Do (S1410).

Next, information on a rolling gesture based on the vertical axis of the electronic pen 200 is acquired (S1420). Here, the information on the rolling gesture may include at least one of a rolling angle, a rolling direction, and a rolling speed based on the vertical axis of the electronic pen 200 . For example, information about the rolling gesture may be received from the electronic pen 200 .

Subsequently, a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture are identified (S1430). However, steps S1420 and S1430 do not necessarily have to be in the corresponding order, and in some cases, at least some information (eg, coefficient of variation) obtained in step S1430 may be obtained before information on the rolling gesture obtained in step S1420. Of course there is.

Thereafter, the slide bar is controlled based on the coefficient of variation of the slide bar, the application coefficient of the rolling gesture, and information about the rolling gesture (S1440).

In step S1430, a coefficient of variation of the slide bar may be identified based on the movement range of the slide bar. Here, the coefficient of variation of the slide bar may be information on a basic movement unit for adjusting the movement of the slide bar.

Also, in step S1430, an application coefficient of the rolling gesture may be identified based on the angle of the rolling gesture. Here, the application coefficient of the rolling gesture may be information about the basic rolling angle of the rolling gesture corresponding to the basic movement unit of the slide bar.

Also, in step S1430, when the slide bar is provided on the execution screen of the application, a movement range of the slide bar may be identified based on the type of application, and a coefficient of variation of the slide bar may be identified based on the movement range of the slide bar. .

Also, in step S1440, the movement of the slide bar may be controlled by the basic movement unit corresponding to the variation coefficient of the slide bar based on the number of times the rolling gesture of the basic rolling angle is identified.

In step S1440, if the rolling speed of the rolling gesture is within the threshold range, the slide bar is controlled to correspond to the coefficient of variation of the slide bar, and if the rolling speed of the rolling gesture exceeds the threshold range, the coefficient of variation of the slide bar is greater than the coefficient of variation of the slide bar. The slide bar may be controlled based on the value, and if the rolling speed of the rolling gesture is less than a threshold range, the slide bar may be controlled based on a value smaller than a variation coefficient of the slide bar.

The control method may further include, when a rolling gesture for an image displayed on a display is input, displaying an enlarged or reduced image based on a rolling direction of the input rolling gesture.

In addition, the control method may further include, when a rolling gesture for a specific object displayed on the display is input, adjusting properties of the specific object in an ascending or descending order based on a rolling direction of the input rolling gesture.

Also, the control method may further include adjusting pen style information of the electronic pen 200 based on the rolling gesture when the rolling gesture is input on the drawing screen.

According to various embodiments as described above, the user UX of the control menu can be improved without touching the screen by controlling the screen through the rolling gesture of the electronic pen. In addition, it is possible to reduce the user interaction time required to search and adjust the menu. In addition, the slide bar can be easily and flexibly adjusted through a rolling gesture of the electronic pen.

In the above-described embodiments, a case in which the electronic pen 200 is implemented in an active manner has been described, but the above-described embodiment may be applied even in a case in which the electronic pen 200 is implemented in a passive manner. However, in an embodiment in which the electronic pen 200 transmits rolling information, it goes without saying that the active electronic pen 200 may be advantageous.

Meanwhile, the methods according to various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed in an existing electronic device. Alternatively, the above-described methods according to various embodiments of the present disclosure, for example, calculating the coefficient of variation of the slide bar may be performed using a deep learning-based artificial neural network (or deep artificial neural network), that is, a learning network model.

In addition, the methods according to various embodiments of the present disclosure described above may be implemented only by upgrading software or hardware of an existing electronic device.

In addition, various embodiments of the present disclosure described above may be performed through an embedded server included in the electronic device or an external server of the electronic device.

Meanwhile, according to an exemplary embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media (eg, a computer). can A device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device A) according to the disclosed embodiments. When a command is executed by a processor, the processor may perform a function corresponding to the command directly or by using other components under the control of the processor. An instruction may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium does not contain a signal and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

Also, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be included in a computer program product and provided. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)) or online through an application store (eg Play Store™). In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

In addition, each of the components (eg, modules or programs) according to various embodiments described above may be composed of a single object or a plurality of entities, and some sub-components among the aforementioned sub-components may be omitted, or other sub-components may be used. Components may be further included in various embodiments. Alternatively or additionally, some components (eg, modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by modules, programs, or other components may be executed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be executed in a different order, may be omitted, or other operations may be added. can

Although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and is common in the technical field belonging to the present disclosure without departing from the gist of the present disclosure claimed in the claims. Of course, various modifications and implementations are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present disclosure.

100: electronic device 110: display
DESCRIPTION OF SYMBOLS 120: sensor 130: memory 140: processor 200: electronic pen

Claims (20)

In electronic devices,
display;
a sensor provided in the display;
a memory storing at least one instruction; and
A processor connected to the display, the sensor, and the memory to control the electronic device;
the processor,
By executing the at least one command,
When the position of the electronic pen on the display is identified by the sensor, a slide bar displayed on the display is identified based on the identified position;
Obtaining information about a rolling gesture based on a vertical axis of the electronic pen;
identify a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture;
Controlling the slide bar based on a variation coefficient of the slide bar, an application coefficient of the rolling gesture, and information about the rolling gesture. According to claim 1,
Information about the rolling gesture,
An electronic device comprising at least one of a rolling angle, a rolling direction, and a rolling speed based on a vertical axis of the electronic pen. According to claim 1,
the processor,
Identifying a coefficient of variation of the slide bar based on the range of movement of the slide bar;
The coefficient of variation of the slide bar is,
Information about a basic movement unit for adjusting the movement of the slide bar, the electronic device. According to claim 3,
the processor,
Identifying an application coefficient of the rolling gesture based on an angle of the rolling gesture;
The application coefficient of the rolling gesture is,
Information about a basic rolling angle of the rolling gesture corresponding to a basic movement unit of the slide bar, the electronic device. According to claim 4,
the processor,
Controlling the movement of the slide bar by a basic movement unit corresponding to a variation coefficient of the slide bar based on the number of times the rolling gesture of the basic rolling angle is identified. According to claim 1,
the processor,
When the slide bar is provided on the execution screen of the application, identifying a movement range of the slide bar based on the type of the application;
An electronic device that identifies a coefficient of variation of the slide bar based on the movement range of the slide bar. According to claim 1,
the processor,
When the rolling speed of the rolling gesture is within a threshold range, controlling the slide bar to correspond to the variation coefficient of the slide bar;
When the rolling speed of the rolling gesture exceeds the threshold range, controlling the slide bar based on a value greater than a variation coefficient of the slide bar;
Controlling the slide bar based on a value smaller than a variation coefficient of the slide bar when the rolling speed of the rolling gesture is less than the threshold range. According to claim 1,
the processor,
When a rolling gesture for the image displayed on the display is input, the electronic device enlarges or reduces the image based on a rolling direction of the input rolling gesture and displays the image. According to claim 1,
the processor,
When a rolling gesture for a specific object displayed on the display is input, properties of the specific object are adjusted in an ascending or descending order based on a rolling direction of the inputted rolling gesture. According to claim 1,
the processor,
If a rolling gesture is input on the drawing screen, the electronic device adjusts pen style information of the electronic pen based on the rolling gesture. According to claim 1,
It further includes a communication interface;
the processor,
An electronic device that receives information on the rolling gesture from the electronic pen through the communication interface. In the control method of an electronic device,
identifying a slide bar displayed on the display based on the identified position when the position of the electronic pen is identified on the display;
obtaining information about a rolling gesture based on a vertical axis of the electronic pen;
identifying a variation coefficient of the slide bar and an enforcement coefficient of the rolling gesture; and
and controlling the slide bar based on a variation coefficient of the slide bar, an application coefficient of the rolling gesture, and information about the rolling gesture. According to claim 12,
Information about the rolling gesture,
A control method comprising at least one of a rolling angle, a rolling direction, and a rolling speed based on a vertical axis of the electronic pen. According to claim 12,
Identifying the coefficient of variation and the applied coefficient comprises:
Identifying a coefficient of variation of the slide bar based on the range of movement of the slide bar;
The coefficient of variation of the slide bar is,
Information on a basic movement unit for adjusting the movement of the slide bar, the control method. According to claim 14,
Identifying the coefficient of variation and the applied coefficient comprises:
Identifying an application coefficient of the rolling gesture based on an angle of the rolling gesture;
The application coefficient of the rolling gesture is,
Information on a basic rolling angle of the rolling gesture corresponding to a basic movement unit of the slide bar, the control method. According to claim 15,
In the step of controlling the slide bar,
Controlling the movement of the slide bar by a basic movement unit corresponding to a variation coefficient of the slide bar based on the number of times the rolling gesture of the basic rolling angle is identified. According to claim 12,
Identifying the coefficient of variation and the applied coefficient comprises:
When the slide bar is provided on the execution screen of the application, identifying a movement range of the slide bar based on the type of the application;
The control method of identifying a coefficient of variation of the slide bar based on the movement range of the slide bar. According to claim 12,
In the step of controlling the slide bar,
When the rolling speed of the rolling gesture is within a threshold range, controlling the slide bar to correspond to the variation coefficient of the slide bar;
When the rolling speed of the rolling gesture exceeds the threshold range, controlling the slide bar based on a value greater than a variation coefficient of the slide bar;
Controlling the slide bar based on a value smaller than a variation coefficient of the slide bar when the rolling speed of the rolling gesture is less than the threshold range. According to claim 12,
When a rolling gesture for the image displayed on the display is input, enlarging or reducing the image based on a rolling direction of the inputted rolling gesture and displaying the image; further comprising, the control method. According to claim 12,
When a rolling gesture for a specific object displayed on the display is input, adjusting properties of the specific object in an ascending or descending order based on a rolling direction of the inputted rolling gesture; further comprising a control method.

Images