TWI442274B - Electronic device and graphical user interface control method thereof - Google Patents

Description

Electronic device and graphical user interface control method thereof

The invention relates to computer technology, in particular to an electronic device and a graphical user interface control method thereof.

Digitally formatted books are commonly referred to as e-books. The ever-popular e-book reader is characterized by a reflective display and free access to the wireless communication capabilities of the Internet bookstore. Since it is specifically designed for reading, the e-reader has a primary surface area reserved for the display, while a physical control element, such as a button or keyboard, is placed around the display. Among these physical displays, the function keys, when actuated, cause the e-reader to perform certain functions such as returning to the main menu, launching an e-mail application, connecting to a web bookstore, or other functions. While function keys are designed to facilitate the execution of specific general or commonly used functions or transactions, the functions or transactions that the user actually uses are different from user to user. The function key is printed with an icon or symbol for its corresponding function, but it may be misleading if you want to change the correspondence between the function key and its corresponding function.

The present invention provides an embodiment of an electronic device including a housing, a physical control component, a display, a detector, and a processor. The physical control component is disposed on a surface of the outer casing, and is first according to the first function of the physical control component and the electronic device An association relationship for actuating the first function when the physical control element is actuated. The display is placed on the surface of the housing to display the virtual control elements required to drive the second function. The detector is configured to detect a drag operation, and the drag operation moves the virtual control component to a specific position on a frame of the display, adjacent to the physical control component. The processor associates the second function and the physical control element to replace the first association relationship in response to the drag operation, and when the physical control element is actuated, the processor is based on the second function and the entity The association of the control elements to perform the second function described above in response to actuation of the physical control elements described above.

The present invention provides an embodiment of a user interface control method, which is implemented in an electronic device, the electronic device includes a display and a physical control component disposed on a surface of the electronic device, wherein the physical control component is based on the physical control component and the electronic device A first association of a first function of the device for actuating the first function when the physical control element is actuated, the display displaying a virtual control element for driving the second function. Detecting a drag operation, the drag operation moves the virtual control element to a specific position on the frame of the display, in close proximity to the physical control element. The second function and the physical control element are associated with each other to replace the first association relationship in response to the drag operation. When the physical control element is actuated, the processor is responsive to the second function and the association of the physical control element to perform the second function in response to actuation of the physical control element.

1-9‧‧‧Transparent window

11-19‧‧‧LED

201-202‧‧‧ Drag operation

41-45‧‧‧ icon

431-433‧‧‧ icon

51-54‧‧‧Area

60‧‧‧ entity control module

61‧‧‧Touchpad

62‧‧‧ button

600‧‧‧Infrared LED

70‧‧‧ lines

81-83‧‧‧Detection board

81a, 81b, 82a, 82b, 83a, 83b‧‧‧ detection board

811, 821 and 831‧‧ ‧ busbars

91, 92 and 93‧‧‧through holes

100‧‧‧Electronic reading device

152‧‧‧ main memory

153‧‧‧ Non-volatile memory

154‧‧‧Many storage devices

155‧‧‧ display

156‧‧‧Wireless communication unit

158‧‧‧Power supply

159‧‧‧Crystal Oscillator

160‧‧‧Input and output devices

161‧‧‧Optical output unit

164‧‧‧埠

165‧‧‧ Controller

166‧‧‧ housing

167‧‧‧Touch detector

500‧‧‧PD Receiver

510‧‧‧PD Receiver

600‧‧‧Infrared LED

610‧‧‧Infrared LED

700‧‧‧Upholstered

701‧‧‧shims

702, 703‧‧‧ wire terminal

704‧‧‧PCB

705‧‧‧ busbar

L1-L5‧‧‧

P1-P2‧‧‧ position

1 is a block diagram showing an embodiment of an electronic reader; FIG. 2 is a top view of the electronic reader; FIG. 3A is a cross-sectional view showing the physical control module; 3B-3F are schematic diagrams showing the structure of each layer in the physical control module; FIG. 3G shows another embodiment of the second layer structure of the physical control module; FIG. 4 is a schematic diagram showing the icon drag operation; FIG. 6 is a flow chart of actuating the operating point of the physical control element; FIG. 7-9 is a schematic diagram of the above electronic reader, wherein the physical control element displays different icons; FIG. 10 shows the electronic The reader has a structural block diagram of another embodiment of a touch sensor utilizing light; FIG. 11 shows a block diagram of another embodiment of an electronic reader having a physical control element for making a button; and FIG. 12 shows a button of a physical control element. Sectional view.

The implementation of the electronic device and its graphical user interface control method is as follows:

1. System Overview

The electronic device disclosed in the present invention can be implemented as a single device or integrated into various electronic devices, such as a mobile phone, a tablet personal computer, a laptop computer, and a multimedia player. , digital camera, set-top box, personal digital assistant (PDA), navigation device or mobile internet device (MID).

1.1 Examples of electronic devices

Referring to FIG. 1, the processor 151 is a central processing unit of the electronic device 100, and may be composed of an integrated circuit (IC) for processing data and executing a computer program. The wireless communication unit 156 establishes a wireless communication channel for the electronic device 100 to connect to the online bookstore through the wireless communication channel and download the electronic book from the online bookstore. The communication unit 156 may include an antenna, a base band, and a radio frequency (RF) chip set for performing wireless local area network communication and/or cellular communication system communication, such as broadband frequency division. Wideband Code Division Multiple Access (W-CDMA) and High Speed Downlink Packet Access (HSDPA).

The processor 10 may be composed of a single packaged IC or a plurality of packaged ICs having the same function or different functions. For example, the processor 151 may only include a central processing unit (CPU), or a CPU, a digital signal processor (DSP), and a communication controller (for example, the communication unit 156). Control the combination of wafers. The above-described communication controller may comprise cellular communication system communication, infrared, Bluetooth (Bluetooth ^TM) or a wireless LAN device. The CPU can be a single computing core or a CPU with multiple computing cores. The communication controller is used to control communication of components in the electronic device 100, or communication between the electronic device 100 and an external device, and can support direct memory access.

The power supply 158 supplies power to the various components in the electronic device 100. The quartz oscillator 159 provides a clock signal to the processor 151 and other components in the electronic device 100. FIG. 1A shows a connection relationship of components in the electronic device 100, and the connection is permeable to the serial port. Stream or parallel bus. The input/output device 160 includes a control button, an alphanumeric keyboard, a touch panel, a touch screen, and a plurality of light emitting diodes (LEDs). The controller 165 controls the operation of the input and output device 160. The processor 151 can control the input/output device 160 through the controller 165. The 埠164 can be connected to a variety of computerized interfaces, such as an external computer device or peripheral device. The 埠164 may be an entity that conforms to a standard such as a universal serial bus (USB) or an Institute of Electrical and Electronics Engineers (IEEE).

The non-volatile memory 153 stores operating systems and applications executed by the processor 151. The processor 151 loads the running program and data data from the non-volatile memory 153 into the main memory 152 and stores the digital content in the mass storage device 154. The main memory 152 may be a random access memory (RAM), such as a static random access memory (SRAM) or a dynamic random access memory (Dynamic RAM, DRAM for short). ). The non-volatile memory 153 may be an Electronically Erasable Programmable Read-Only Memory (EEPROM), such as a reverse (NOR) flash memory or a reverse (NAND) flash. Memory.

The audio output unit 161 includes a digital analog converter for converting the audio signal output by the processor 151 from a digital format to an analog format.

Display 155 displays text and images and may include a reflective display, such as an electrophoretic display, or a display that utilizes an interferometric modulation of light. Or, on The display 155 can include an electronic paper, an OLED organic light emitting diode (OLED), a field emission display (FED), or a liquid crystal display (LCD). The display 155 can display various graphical user interfaces (GUIs) as virtual control elements, including but not limited to windows, scroll axes, icons, and scrapbooks. The display 155 described above may include a single display or a plurality of displays of different sizes. The following paragraphs will describe that the processor 151 described above can display various GUI elements on the display 155 described above.

The electronic reading device 100 described above includes a housing for housing components of the electronic reading device 100.

The I/O unit 160 includes a touch detector 167 for detecting a touch operation on the display 155. The touch detector 167 can include a transparent touch panel over the display 155, or an optical touch transmitter and receiver array disposed on the frame of the display 155, such as disclosed in US Patent Publication No. 20090189878.

Referring to FIG. 2, the electronic reading device 100 includes a housing 166 for integrating components of the electronic reading device 100 described above. The physical control unit of the I/O component 160 includes a control module 60, a touchpad 61, and a button 62 that form part of the frame of the display 155. The touch panel 61 is used to flip the page of the e-book when it is operated. The button 62 is used to display a menu on the display 155 as described above. The control module 60 includes an actuation point for driving an operation action combined by the actuation point when the constant movement point is pressed and operated. Each consistent moving point can initially be combined with a preset operational action or without any operational action. The display 155, the control module 60, the touch panel 61 and the button 62 are disposed on the housing 166 of the electronic reading device 100 and formed Part of it. In the following description, combining the two objects is to associate the two objects.

1.2 Structure of the physical control unit

FIG. 3A shows a cross-sectional view of the physical control unit 60 along line 70 of FIG. 2. The control module 60 includes levels L1-L5. Referring to Figures 3A and 3B, the hierarchy L1 forms the uppermost layer of the control module 60 and has a transparent window array, wherein the transparent windows 1-9 are shown in Figure 3A. Transparent windows other than transparent windows 1-9 are shown in similar unnumbered squares in Figures 3B-3D. The transparent window of level L2 aligns with the transparent window of level L1. The transparent window of the hierarchy L3 aligns the transparent windows of the classes L2 and L4. Fig. 3F shows the illumination device of the hierarchy L4 in a similar circle, wherein the illumination devices 11-19 are shown in Fig. 3A. The windows of the levels L1-L3 may be filled with a transparent dielectric material such that the illumination means of the above-described level L4 can emit light through the transparent windows.

Fig. 3C shows the upper surface of the above-described hierarchy L2. The above-mentioned hierarchy L2 includes a printed circuit board (PCB), three metal planar regions 81-83 (shown in FIG. 3C) are formed thereon in a printed manner, and each metal planar region serves as a detection panel. One electrode. There are multiple cutouts in each metal plane area to form a plurality of windows. The through holes 91, 92, and 93 in the layer L2 indicated by dots in FIG. 3C are formed on the detecting plates 81-83, respectively, and filled with a conductive material. The bus bars 811, 821, and 831 of the connection detecting plates 81-83 are on the lower surface of the level L2, and thus are indicated by broken lines in Fig. 3C. Referring to FIG. 3D, the bus bars 811, 821, and 831 are connected to the detecting plates 81-83 through the through holes 91, 92, and 93, respectively. One of the terminals of each of the bus bars 811, 821, and 831 is connected to one detection board, and the other end is connected to the controller 165. The controller 165 detects a touch change in the vicinity of the operating point of the control module 60 by determining a change in capacitance to the first detecting board. Alternatively, the controller 165 passes the determination to the first detection. The thermal parameters or bio-related parameters of the panel are changed to detect touch operations near the operating point of the control module 60. Referring to FIG. 3E, the above-described level L3 includes a dielectric layer, such as another printed circuit board on which a window array is formed to align the windows of the above-described level L2. In Fig. 3F, the illuminator of the level L4 is formed on the upper surface of the level L5 and aligned with the window of L3. The illuminator at the above-described level L5 may include an LED or an OLED. A bus bar electrically connected to the above illuminator is also formed at the above-described level L5. The distance between any two detection boards is greater than the thickness of the above-mentioned layer L1. The above levels L2 and L5 may include a PCB.

It should be understood that more detection boards can be formed on the above-mentioned layer L2. The operating point of the control module 60 can be combined with a set of detection boards. For example, FIG. 3G shows another embodiment of the above-described hierarchy L2, in which the vias and busbars are shown in the manner of FIG. 3C. The operation point of the control module 60 is combined with three detection boards, a middle detection board (such as the detection board 81), a left detection board (such as the detection board 81a), and a right detection board (such as the detection board 81b). The controller 165 is connected via corresponding through holes and bus bars. When the operation point of the control module 60 is combined with a plurality of group detection boards, the controller 165 can detect the actuation sequence of the detection boards in the detection board group to detect the operation position. The direction in which the touch action moves.

2. GUI control method

The electronic reading device 100 described above can provide a plurality of functions and display icons corresponding to the above plural functions.

2.1 Icon Status

The status of the icon shows the events received or generated by the program that the icon is combined with. When the display 155 is turned off or another program in the foreground is displayed, the processor 151 may update the image displayed by the illuminator of the control module 60. The status of the target to synchronize with the relevant program event or status of the above icon.

The state of the icon can include at least three states, such as ON, OFF, and animation. The open state of the icon may include a plurality of sub-states for respectively representing the program function status of the icon. For example, the secondary status may indicate performing or pausing a function, logging in or logging out of a service, connecting or disconnecting a communication channel, and other events. The closed state of the icon indicates that the electronic reading device 100 described above is in the pause mode. The animated status of the icon displays a message related to the icon's related function or program.

Referring to FIG. 2, icons 41-45 respectively correspond to the following functions: returning to the main page, displaying the favorite folder (commonly referred to as "my favorite"), starting the music player, and finally simple syndication (RSS). ) Reading programs, as well as instant Internet messaging applications. The icons 431-435 correspond to the option functions of the music player combined with the icon 43 described above, respectively. The icon 431 above corresponds to the pause and resume function of music playback. The icons 432 and 433 described above respectively correspond to the backward and forward operation of the music playback. The icon 434 above corresponds to the function of displaying the information query GUI. The icon 435 described above corresponds to the function of displaying the player setting GUI. When it is detected that the touch operation activates one of the icons shown in FIG. 2, the processor 151 performs the function corresponding to the activated icon.

Referring to FIG. 4, the areas 51-53 on the control module 60 are in the vicinity of the detection boards 81-83, respectively. Each of the above-described areas 51-53 serves as an operating point of the control module 60, and when activated, causes the electronic reading device 100 to perform the function combined by the operating point. When detecting the touch operation at the operation point, the controller 165 can interpret the detected touch operation as an actuation operation of the operation point. Each of the above regions 51-53 may initially be combined with no function, in the above regions 51-53 The illuminator can be turned off at first, without displaying any iconified images.

2.2 Example Operation

Referring to FIG. 5, the processor 151 detects a drag operation of moving an icon to the specific position of the edge of the display 155, and the specific location is adjacent to an operation position of the control module 60 (step S2), and The actuation event of the above operational position immediately follows the first drag operation described above (step S4). The processor 151 responds to the above steps S2 and S4, adds the icon to a reserved area of the main memory 152 (step S6), and combines the operation position with the function of the icon (step S8), and A part of the illuminators in the operation position area is turned on to display an iconized image for indicating the icon at the operation position (step S10). Before the above step S2, if the above-mentioned operation position has been associated with another original icon and its original function, the processor 151 replaces the association generated in step S8 with the association of the above-mentioned operation position with the original icon and its original function. The processor 151 can represent the association as data and store it in the main memory 152 or the non-volatile memory 153. If the drag operation is detected at the end of the display 155, and the time until the actuation event of the operation position occurs does not exceed the predetermined time length D, the processor 151 may determine the actuation of the operation position. The event is immediately following the drag operation described above. If the end point of the above drag operation on the edge of the display 155 is within the vicinity of an operation position, the processor 151 may determine that the drag operation of the icon is a special drag operation. After the drag operation described above, the processor 151 displays the iconized image as a copy of the icon, and may not need to delete the icon in the display 155. Referring to FIG. 6, when receiving an actuation event of the above-described operation position (step S12), the processor 151 interprets the actuation event as the operation position The actuating event of the combined icon (step S14), and the function combined by the icon is executed (step S16). It should be understood that, in step S12, the actuation event of the above-mentioned operating position does not follow the special drag operation. The processor 151 updates the iconized image on the activated operation position to synchronize the appearance of the image with the icon.

Referring to FIG. 4, for example, the processor 151 detects a first drag operation 201 for moving the icon 431 to the specific position P1 of the edge of the display 155. The specific position P1 is adjacent to an operation position 51 of the control module 60. (Step S2), and the above-described operation position 51 is actuated immediately after the above-described drag operation (step S4). The processor 151, in response to the above steps S2 and S4, adds the icon 431 to a reserved area of the main memory 152 (step S6), and combines the operation position 51 with the music play and pause function associated with the upper icon 431. a first association relationship between the operation position 51 and the function of the icon 431 is established (step S8), and a part of the illuminators in the operation position area 51 is turned on to display the iconized image 301 for representing at the operation position. The above icon 431 (step S10) is as shown in FIG. When receiving the actuation event of the operation position 51 (step S12), the processor 151 interprets the actuation event as an actuation event of the icon 431 combined with the operation position according to the first association relationship (step S14), And the music playing function combined with the icon is executed (step S16). The icon 431 described above can change the appearance in accordance with the functions performed as described above. The processor 151 controls the illuminator at the operated position at the actuation to synchronize the display of the image 301 with the icon 431. As shown in FIG. 8, for example, after the above step S14, the icon 431 and the image in the area 51 are synchronously changed to display the music play pause function.

In addition, the processor 151 described above can smoothly display the above icon in the above step S2. 431 moves along the trajectory of the drag operation 201, and when the above-described drag operation reaches the position P1, the icon 431 is continuously moved within a period of time D according to the moving speed and direction of the icon 431 at the position P1.

Furthermore, the processor 151 detects a first drag operation 202 for moving the icon 432 to the specific position P2 of the edge of the display 155, and the specific position P2 is adjacent to an operation position 51 of the control module 60 (step S2). And the actuation event of the above-described operating position 51 follows the second drag operation 202 described above (step S4). The processor 151, in response to the above steps S2 and S4, adds the icon 432 to a reserved area of the main memory 152 (step S6), and skips the operation of the music playback associated with the upper icon 432. The functions are combined to establish a second association relationship between the operation position 51 and the function of the icon 432 (step S8), and a part of the illuminators in the operation position area 51 is turned on to display the iconized image 302 (as shown in FIG. 9). ), for indicating the above icon 432 at the above operation position (step S10), instead of executing the music play pause function. The processor 151 replaces the association between the operation position 51 and the icon 432 and its function (ie, the second association relationship) with the association between the operation position 51 and the original icon 431 and its original function (ie, the first association relationship). When receiving the actuation event of the operation position 51 (step S12), the processor 151 interprets the actuation event as an actuation event of the icon 432 combined with the operation position according to the second association relationship (step S14), And the backward operation operation function of the music playback in which the icon is combined is executed (step S16).

3. Other implementations

3.1 Other Structures of Touch Detector

A light guide component, such as an optical fiber, can be used to connect each illuminator to a corresponding window on the detection board to interfere with the illuminator. And detecting interference between boards. The size of the transparent window on the control module 60 can be as small as the cross section of the optical fiber. In addition, one end of the optical fiber is connected to one transparent window of the control module 60, and the other end is connected to one or more pixels of the display 155.

Referring to Fig. 10, the above-described electronic reading device 100 can utilize an infrared LED and a photodiode (PD) receiver disclosed in U.S. Patent Publication No. 20090189878. The infrared LEDs are represented by circles with horizontal stripes, and the PD receivers are represented by circles without horizontal stripes. The infrared LEDs 600 are arranged in a vertical LED array on the left edge of the display 155, and the PD receivers 500 are arranged in a vertical receiver array on the left edge of the display 155 for receiving the light emitted by the LEDs 600. The infrared LED 610 is arranged in a horizontal LED array on the lower edge of the display 155 and the control module 60. The PD receiver 510 is arranged in a horizontal receiver array on the upper edge of the display 155 and the control module 60. The light emitted by the LED 610 is received. The processor 151 can use the infrared LED 600, the infrared LED 610, the PD receiver 500, and the PD receiver 510 to detect touch operations in the area surrounded by the infrared LED and the PD receiver. The infrared LED and the PD receiver described above may be fixed to the inner surface of the casing 166.

Referring to Fig. 11, the operation position of the control module 60 described above may be designed as a button in the area 51-54. Each operating position must be pressed to actuate. Figure 12 shows a cross-sectional view of the button in region 53 along line 70. The same or similar elements in FIGS. 3A and 12 are denoted by the same reference numerals. The upper layer L1 of the button includes a plurality of transparent windows, including but not limited to transparent windows 1-9, through which the illuminators 11-19 can emit light. An illuminator including 11-19 is formed on the PCB L5 and connected to the above-described controller 165 via the elastic bus bar 705. Bus 705 can be a soft printed battery Flexible printed circuit board (FPCB). The pad 700 of the above-mentioned button is made of an elastic material, and the pad 701 on the surface of the pad 700 is made of a conductive material. Reference numerals 702 and 703 denote two wire terminals printed on the PCB 704. When the button is pressed, the pad 701 can be connected to the terminals 702 and 703. The processor 151 can determine whether the button is accepted or not by determining whether the terminals 702 and 703 are connected to be turned on.

3.2 Other touch operations

The iconized image displayed on the control module 60 described above may also be referred to as an icon. The processor 151 can respond to the touch operation at the operation position according to the state of the icon combined with the operation position.

When the processor 151 moves from the operation position of the control module 60 to the drag operation of the display 155 in response to the icon of the open state, the processor 151 can delete the combination relationship between the operation position and the program of the icon, and is not in the above control. The above icon is displayed on the module 60. After the combination of the above operation position and the program of the icon is deleted, the combination between the operation position and the function operation of the icon and the program of the icon is also deleted. The processor 151 performs the function represented by the icon when the pressing operation is performed on the icon in response to the on state. The processor 151 brings the program combined with the icons to the foreground of the display 155 when the long-time pressing operation on the icon in response to the on state is performed. The long-time pressing operation described above is a contact or pressing operation for more than a predetermined period of time in the above-described operating position.

When the electronic reading device 100 is in the pause state, the processor 151 can enter a power saving state, and the control module 60 and the at least one control IC connected thereto (for example, the controller 165 described above) are still in a normal working state. When the control IC responds to the pressing operation of the icon in the closed state, the processor 151 may be interrupted, and the above The processor 151 changes the operation mode of the electronic reading device 100 to restore the icon on the control module 60 to the on or animation state.

The processor 151 can change the state of the icon displayed on the control module 60 from the on state to the animation state in response to the program event or message. When the processor 151 moves in response to the pressing operation on the icon of the animation state of the control module 60, when the operation position is moved to the drag operation of the display 155, the GUI component can be presented to display the program combined with the icon. Message.

The processor 151, when responding to the dragging operation of the icon of the animation state from the operation position of the control module 60 to the drag operation of the display 155, brings the program combined with the icon to the foreground of the display 155.

The processor 151 cancels the message notification of the program combined with the icon when responding to the drag operation of moving the icon of the animation state from the operation position of the control module 60 to the edge of the control module 60, and the icon is The status changes from the animation state to the on state.

4 Conclusion

As described above, the electronic reading device 100 allows an icon to move between the display 155 and a physical control module that forms part of the bezel of the display 155. Each of the above-mentioned physical control modules can be used as a function key, and the settings can be changed to drive different functions. The above entity control module has no printed icons, so it can provide more features that can be set by the user.

Claims (10)

An electronic device comprising: a housing; a physical control component disposed on a surface of the housing, in accordance with a first association relationship between the physical control component and a first function of the electronic device, when the physical control component is actuated Activating a first function; a display disposed on a surface of the outer casing for displaying a virtual control component required to drive the second function; and a detector for detecting a drag operation, the dragging operation The control element moves to a specific position on the frame of the display, adjacent to the physical control element; and a processor associates the second function with the physical control element to replace the first association in response to the drag operation And when the physical control element is actuated, the processor is responsive to the second function and the association of the physical control element to perform the second function in response to actuation of the physical control element. The electronic device of claim 1, wherein the physical control element comprises a button having an array of illumination devices. The electronic device of claim 2, wherein the processor controls the array of illumination devices to display a first icon corresponding to the first function before detecting the drag operation, and detects the drag After the operation, the array of the light-emitting devices is controlled to display a second icon corresponding to the second function. The electronic device of claim 3, further comprising: a second detector for detecting an end point of the drag operation, wherein an end point of the drag operation is in a vicinity of the physical control component, The processor will perform the above second function and The entity control elements are associated. The electronic device of claim 4, wherein the processor cancels an association between the second function and the physical control element in response to a cancel gesture operation, the cancel gesture operation, the second icon from the The physical control element is dragged to the above display. The electronic device of claim 4, wherein the processor changes the state of the second icon from an original state to an animated state in response to a message or event notification of a program associated with the second icon. The electronic device of claim 6, wherein the processor displays a message of the program associated with the second icon in response to a pressing operation on the second icon. The electronic device of claim 6, wherein the processor brings the program associated with the second icon to a foreground of the display in response to an inward drag operation, and the inward drag operation is to perform the second The icon is dragged from the above-described physical control element to the above display. The electronic device of claim 6, wherein the processor cancels the message notification from the program associated with the second icon, and returns the second icon to the original state to respond to the outward drag operation. The outward drag operation drags the second icon from the physical control element to the edge of the physical control element. A user interface control method is implemented in an electronic device, the electronic device includes a display and a physical control component disposed on a surface of the electronic device, wherein the physical control component is based on the first function of the physical control component and the electronic device a first association relationship, when the physical control element is actuated to actuate the first function, the display displays a virtual control element for driving the second function, the method comprising: detecting a drag operation, the drag operation The virtual control element is moved to a specific position on the frame of the display, adjacent to the physical control component; the second function and the physical control component are associated with the first association relationship in response to the drag operation; When the physical control element is actuated, the electronic device performs the second function in response to the actuation of the physical control element in response to the second function and the association of the physical control element.