WO2021121223A1 - 交互系统的显示方法、交互系统及电子设备 - Google Patents

交互系统的显示方法、交互系统及电子设备 Download PDF

Info

Publication number
WO2021121223A1
WO2021121223A1 PCT/CN2020/136498 CN2020136498W WO2021121223A1 WO 2021121223 A1 WO2021121223 A1 WO 2021121223A1 CN 2020136498 W CN2020136498 W CN 2020136498W WO 2021121223 A1 WO2021121223 A1 WO 2021121223A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
screen
area
display area
extended
Prior art date
Application number
PCT/CN2020/136498
Other languages
English (en)
French (fr)
Inventor
陈浩
陈晓晓
胡凯
杨孝云
顾兵
王剑锋
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20901033.9A priority Critical patent/EP4060472A4/en
Publication of WO2021121223A1 publication Critical patent/WO2021121223A1/zh
Priority to US17/843,596 priority patent/US20220317964A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/03—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes specially adapted for displays having non-planar surfaces, e.g. curved displays
    • G09G3/035—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes specially adapted for displays having non-planar surfaces, e.g. curved displays for flexible display surfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1641—Details related to the display arrangement, including those related to the mounting of the display in the housing the display being formed by a plurality of foldable display components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1647—Details related to the display arrangement, including those related to the mounting of the display in the housing including at least an additional display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1652—Details related to the display arrangement, including those related to the mounting of the display in the housing the display being flexible, e.g. mimicking a sheet of paper, or rollable
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04803—Split screen, i.e. subdividing the display area or the window area into separate subareas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2310/00—Command of the display device
    • G09G2310/02—Addressing, scanning or driving the display screen or processing steps related thereto
    • G09G2310/0232—Special driving of display border areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G09G2340/0414—Vertical resolution change
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G09G2340/0421—Horizontal resolution change
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0442—Handling or displaying different aspect ratios, or changing the aspect ratio
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2380/00—Specific applications
    • G09G2380/02—Flexible displays

Definitions

  • This application relates to the field of human-computer interaction technology, and in particular to a display method of an interactive system, an interactive system, an electronic device, and a computer-readable storage medium.
  • the ultra-curved screens of electronic devices such as flexible screens, folding screens and other product forms
  • the size of the screen is getting larger and larger.
  • the present application provides a display method of an interactive system, an interactive system, an electronic device, and a computer-readable storage medium, which can bring a brand new interactive method to the user and improve the user's experience.
  • Some embodiments of the present application provide a display method of an interactive system.
  • the following describes the application from multiple aspects, and the implementations and beneficial effects of the following multiple aspects can be referred to each other.
  • the present application provides a display method of an interactive system, which is applied to an interactive system.
  • the interactive system includes a display screen.
  • the method includes: determining a display area and an extended area divided in the display screen;
  • the operation of the extended area generates event information, where the operation of the extended area can be a user’s touch behavior, or a behavior gesture, etc.
  • the event information can include a tap touch event, a long press touch event, and a sliding touch screen One or more of events and user-predetermined touch screen events.
  • the user can influence the operation of the display area through the operation in the extended area, bring a new interactive mode to the user, and enhance the user's interactive experience.
  • the display area and the extended area are two different areas belonging to the same physical display screen, for example, they may be the first screen and the first screen of the folding screen in the flexible screen.
  • the second screen, the flat part and the curved part on the curved screen, or the flat screen can be divided into two parts, etc.
  • the division of the display area and the extended area is determined by an underlying display system.
  • dividing the display screen into a display area and an extended area includes: The display system determines the original screen size of the display screen, the screen size of the display area, and the screen size of the extended area, and the underlying display system notifies the screen size of the display area as the screen size for system display, and The display area is used for display of the interactive system.
  • the division of the display area and the extended area is determined by a window layer.
  • the window layer divides the display screen into a display area and an extended area, including: The original screen size, the screen size of the display area, and the screen size of the extended area of the display screen are determined, and the display area and the extended area size can be delineated by the user's split-screen operation instructions, thereby determining the original screen size and display size of the display screen.
  • the screen size of the area and the screen size of the extended area and notify the screen size of the display area as the screen size for system display, and use the display area for display of the interactive system.
  • the display area and the extended area belong to two different physical display screens, for example, a physically divided folding screen, and one of the screens is used as the display area, The other screen is used as an extended area, and the size of the two screens is fixed.
  • responding to the operation corresponding to the request message according to the event information includes: determining the screen size of a display area and an extended area; and based on the display area and the extended area The relationship between the screen sizes of, maps the event information to the display area, and responds to operations corresponding to the event information in the display area.
  • mapping the event information to the display area according to the relationship between the size of the display area and the extended area includes: determining the display area, the The position of the extended area in a preset coordinate system; based on the preset coordinate system, determine the coordinates of the original report point of the event information in the preset coordinate system, and determine the original report point and the The relative position of the display area and the extended area; according to the coordinates, the size relationship between the display area and the extended area, the coordinates of the original report point of the event information are mapped to the corresponding position of the display area, that is to say , The coordinates of the original report point of the event information are mapped to the position corresponding to the display area through the mapping relationship, and then the operation of affecting the display area through the operation of the extended area can be realized, and a new way of interaction is brought to the user.
  • responding to the operation corresponding to the event information in the display area includes: combining a preset operation of the extended area with the display area
  • the operation establishes the mapping relationship, so that the user inputs the preset operation in the extended area and maps the user’s operation in the display area. That is to say, the mapping relationship can be set in the system in advance, and the user is in the display area according to the mapping relationship.
  • the preset operation input in the extended area is mapped to an operation in the display area, where the preset operation can be preset by the system or can be set according to the user's own needs to further enhance the user's interactive experience.
  • the display screen is a curved screen, a flat screen, or a folding screen
  • the expansion area is set at an edge area or a part of the curved screen or the flat screen
  • the expansion The area is set on one of the folding screens, or a part of the one screen, or the folding area of the folding screen after being folded.
  • the present application provides an interactive system, including a display screen, a processing module, and a generation module, which is used to determine the display area and the extended area divided in the display screen, and is used to determine the display area and the extended area divided in the display Event information is generated, where the event information includes one or more of a touch screen tap event, a long-press touch screen event, a sliding touch screen event, a user-predetermined touch screen event, and screen recording, and the processing module is used to Event information, responding to the operation corresponding to the event information in the display area, wherein the operation corresponding to the event information includes a tap touch event, a long touch screen event, a sliding touch screen event, and a user predetermined touch screen event One or more of.
  • the user can influence the operation of the display area through the operation in the extended area, bring a new interactive mode to the user, and enhance the user's interactive experience. .
  • the display area and the extended area are two different areas belonging to the same physical display screen.
  • they may be the first screen and the first screen of the folding screen in the flexible screen.
  • the second screen, the flat part and the curved part on the curved screen, or the flat screen can be divided into two parts, etc.
  • the division of the display area and the extended area is determined by the underlying display system, the display screen is divided into the display area and the extended area, and the processing module is also used for : Determine the original screen size of the display screen, the screen size of the display area, and the size of the extended area; and notify the size of the display area as the screen size for system display, and use the display area for all The display of the interactive system.
  • the division of the display area and the extended area is determined by a window layer
  • the display screen is divided into a display area and an extended area
  • the processing module is further configured to: Determine the original size of the screen, the size of the display area, and the screen size of the extended area of the display screen; and notify the screen size of the display area as the screen size for system display, and use the display area for all The display of the interactive system.
  • the display area and the extended area belong to two different physical display screens, for example, a physically divided folding screen, and one of the screens is used as the display area, The other screen is used as an extended area, and the screen size of the two screens is fixed.
  • the processing module is specifically configured to: determine the screen size of the display area and the extended area; and calculate the size of the display area and the screen size of the extended area based on the relationship between the display area and the screen size of the extended area.
  • Event information is mapped to the display area, and an operation corresponding to the event information is responded to in the display area.
  • the processing module is specifically further configured to: determine the positions of the display area and the extended area in a preset coordinate system; and determine based on the preset coordinate system The coordinates of the original report point of the event information in the preset coordinate system, and determine the relative position of the original report point to the display area and the extended area; according to the coordinates, the display area and the extended area The relationship between the size of the screen maps the coordinates of the original report point of the event information to the position corresponding to the display area.
  • the processing module is specifically further configured to: establish a mapping relationship between the preset operation of the extended area and the operation of the display area, so that the user can input the information in the extended area.
  • the preset operation mapping is the user's operation in the display area, that is, the preset operation input by the user in the extended area can be mapped to the operation in the display area by pre-setting the mapping relationship in the system. Operation, wherein the preset operation may be preset by the system, or may be set according to the user's own needs, so as to further enhance the user's interactive experience.
  • the display screen is a curved screen, a flat screen, or a folding screen
  • the expansion area is set at an edge area or a part of the curved screen or the flat screen
  • the expansion The area is set on one of the folding screens, or a part of the one screen, or the folding area of the folding screen after being folded.
  • the present application provides an electronic device including a processor and a memory, the memory stores instructions, and the processor is configured to read the instructions stored in the memory to execute the above-mentioned first One aspect of the method of embodiment.
  • the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is run by a processor, the processor executes the method of the above-mentioned embodiment of the first aspect.
  • Fig. 1 is a schematic structural diagram of an interactive system according to an embodiment of the application
  • FIG. 2 is a schematic structural diagram of the distribution of the expansion area and the display area on the display screen of the interactive system according to an embodiment of the application;
  • FIG. 3 is a schematic diagram of the software structure of an interactive system according to an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of a display screen of an interactive system according to an embodiment of the application.
  • FIG. 5 is a flowchart of an interactive system display method according to an embodiment of the application.
  • FIG. 6 is a schematic diagram of the architecture of the internal operating system of the interactive system according to an embodiment of the application.
  • FIG. 7 is a schematic structural diagram of a mapping scene 1 of a display method of an interactive system according to an embodiment of the application;
  • FIG. 8 is a schematic diagram of the position structure of a display screen, an extended area, and a display area in a preset coordinate system according to an embodiment of the application;
  • FIG. 9 is a schematic structural diagram of a second mapping scenario of a display method of an interactive system according to an embodiment of the application.
  • FIG. 10 is a schematic structural diagram of a mapping scenario 3 of a display method of an interactive system according to an embodiment of the application.
  • FIG. 11 is a schematic structural diagram of an interactive system according to an embodiment of the application.
  • FIG. 12 is a block diagram of a device according to some embodiments of the application.
  • FIG. 13 is a block diagram of a system on chip (SoC) according to some embodiments of the application.
  • SoC system on chip
  • the display method of an interactive system provided by the embodiments of the application can be used in electronic devices with display screens such as mobile phones, tablet computers, notebook computers, personal computers, netbooks, handheld computers, personal digital assistants, and wearable devices. This application does not make any restrictions.
  • FIG. 1 shows a schematic diagram of the structure of the mobile phone.
  • the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and user Identification module (subscriber identificat ion module, SIM) card interface 195 and so on.
  • SIM subscriber identification ion module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile phone 100.
  • the mobile phone 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc.
  • AP application processor
  • modem processor modem processor
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the different processing units may be independent devices or integrated in one or more processors.
  • the processor can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the processor 110 may include one or more interfaces.
  • Interfaces can include integrated circuit (I2C) interfaces, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interfaces, pulse code modulation (PCM) interfaces, universal asynchronous transmitters receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / Or Universal Serial Bus (USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART mobile industry processor interface
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB Universal Serial Bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may include multiple sets of I2C buses.
  • the processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100.
  • the MIPI interface can be used to connect the processor 110 with the display screen 194, the camera 193 and other peripheral devices.
  • the MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on.
  • the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100.
  • the processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the interface connection relationship between the modules illustrated in the embodiment of the present application is merely a schematic description, and does not constitute a structural limitation of the electronic device 100.
  • the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance).
  • the power management module 141 may also be provided in the processor 110.
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the mobile phone 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
  • the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied on the mobile phone 100.
  • the mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like.
  • the mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation via the antenna 1.
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • the wireless communication module 160 can provide applications on the mobile phone 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems. (global navigation satellite system, GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • WLAN wireless local area networks
  • BT Bluetooth
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication
  • IR infrared technology
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
  • the wireless communication module 160 may also receive a signal to be sent from the processor 110, perform frequency modulation, amplify, and convert it into electromagnetic waves to radiate through the antenna 2.
  • the antenna 1 of the mobile phone 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • Wireless communication technologies can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), and broadband code division. Multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , And/or IR technology, etc.
  • GSM global system for mobile communications
  • GPRS general packet radio service
  • CDMA code division multiple access
  • CDMA broadband code division. Multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , And/or IR technology, etc.
  • GNSS can include global positioning system (GP
  • the mobile phone 100 implements a display function through a GPU, a display screen 194, and an application processor.
  • the GPU is an image processing microprocessor, which is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations and is used for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 may include a display panel and a display, such as a touch sensor 180K.
  • the display is used to output display content to the user and display the display content on the display panel, such as displaying images, videos, etc.
  • the display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active matrix organic light-emitting diode active-matrix organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc.
  • the touch device is a touch sensor 180K.
  • the touch sensor 180K may be disposed on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”.
  • the touch sensor 180K is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • the visual output related to the touch event can be provided through the display screen 194.
  • the touch sensor 180K may also be disposed on the surface of the mobile phone 100, which is different from the position of the display screen 194.
  • the number of touch devices can be set to one or more according to the number of display areas or display screens.
  • the extended area and display area of the display screen can each correspond to a touch device to receive data from different areas. The user's operation and the touch event generated.
  • the mobile phone 100 can realize the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
  • the ISP is used to process the data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, which is converted into an image visible to the naked eye.
  • ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193.
  • the camera 193 is used to capture still images or videos.
  • the object generates an optical image through the lens and is projected to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the mobile phone 100 may include one or N cameras 193, and N is a positive integer greater than one.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the mobile phone 100 selects the frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the mobile phone 100 may support one or more video codecs. In this way, the mobile phone 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • MPEG moving picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, and so on.
  • NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • applications such as intelligent cognition of the mobile phone 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, and so on.
  • the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the mobile phone 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
  • the internal memory 121 may be used to store computer executable program code, and the executable program code includes instructions.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.
  • the processor 110 executes various functional applications and data processing of the mobile phone 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the mobile phone 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
  • the pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal.
  • the pressure sensor 180A may be provided on the display screen 194.
  • the capacitive pressure sensor may include at least two parallel plates with conductive materials.
  • the mobile phone 100 determines the intensity of the pressure according to the change of the capacitance.
  • a touch operation acts on the display screen 194, the mobile phone 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the mobile phone 100 may also calculate the touched position based on the detection signal of the pressure sensor 180A.
  • touch operations that act on the same touch position but have different touch operation strengths may correspond to different operation instructions. For example: when a touch operation whose intensity of the touch operation is less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.
  • the display screen 194 of the mobile phone 100 is divided into a display area and an extended area, where the display area and the extended area may belong to one physical display screen, or they may belong to two physical displays.
  • the screen such as one of the folding screens, can use one of the two screens as the display area and the other screen as the extended area.
  • the extended area can be provided with touch devices to receive user operations on the extended area, and A corresponding touch event is generated, for example, one or more of a touch screen touch event, a touch screen touch event for a long time, a touch screen sliding event, and a user-predetermined touch screen event.
  • the extended area can be edge areas set on both sides of a curved screen or a flat screen.
  • the user can operate on the extended areas on both sides, and the display area responds to the operation.
  • the extended area may be a part of the edge area set on one side of the curved screen or the flat screen, and the user can operate on the extended area on one side and respond to the operation in the display area.
  • the extended area may be one of the folding screens, such as a secondary screen.
  • the display area is the primary screen. The user can operate on the secondary screen and respond to the operation on the primary screen.
  • the extended area may be a foldable area in the folding screen, and the user can operate on the foldable area, and respond to the operation on the main screen or the secondary screen as the display area.
  • the extended area can receive a user's touch operation through a touch device, and the display area can be used as a conventional independent display screen and also used to respond to the user's touch operation from the extended area.
  • the extended area can be a black screen regardless of whether it is touched or not, or it can only be used to display time, weather, memo information, etc.
  • the above-mentioned software system of the mobile phone 100 may adopt a layered architecture, and the software system of the mobile phone 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiment of the present application takes an Android system with a layered architecture as an example to illustrate the software structure of the mobile phone 100.
  • FIG. 3 is a block diagram of the software structure of the mobile phone 100 according to an embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has its own role and division of labor. Communication between layers through software interface.
  • the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
  • the application layer can include a series of application packages.
  • the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer may include a window manager, a positioning strategy service, an extended interaction module, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.
  • the positioning strategy service and the extended interaction module may also be set in the underlying system, which is not the only limitation here.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • the system library can include multiple functional modules.
  • surface manager surface manager
  • media library Media Libraries
  • three-dimensional graphics processing library for example: OpenGL ES
  • 2D graphics engine for example: SGL
  • the system library and kernel layer below the application framework layer can also be referred to as the underlying system.
  • the underlying system includes the underlying display system for providing display services.
  • the underlying display system may include the display in the kernel layer.
  • Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.
  • the application layer and application framework layer run in a virtual machine.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 2D graphics engine is a drawing engine for 2D drawing.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.
  • the underlying display system can obtain the screen size of the display area, the screen size of the extended area, and the original screen size of the display screen, and inform the entire system that the area used for the display of the mobile phone 100 is the size of the display area.
  • the size of the screen can obtain the screen size of the display area, the screen size of the extended area, and the original screen size of the display screen, and inform the entire system that the area used for the display of the mobile phone 100 is the size of the display area. The size of the screen.
  • the original screen size of the display is A
  • the actual screen size of the display area is A1
  • the display configuration of the display driver changes the screen size A of the display to A1, that is, although the original screen size of the display is A
  • the underlying display system informs the entire system that the size of the screen used for display is A1 (display area)
  • the display configuration information command (config) of the Android system can be used for configuration, and the interface for obtaining the screen size for display can be modified.
  • the window manager is used to manage window programs.
  • the window manager can obtain the screen size A1 of the display area for display from the underlying display system, that is, call the interface of the screen size that has been modified by the underlying display system to obtain the screen size of the display area for display A1.
  • the user's operation at any position on the display screen can be responded to and displayed on the display area on the display screen of the mobile phone 100.
  • the extended area can receive and distribute the user's information in the extended area through the extended interactive module. operating.
  • FIG. 4 exemplarily shows the display screen and the extended area and the screen size of the display area, as shown in (a) and (b) in FIG. 4,
  • the underlying display system may Obtain the original screen size A of the display screen
  • the window manager obtains the original screen size A of the display screen from the underlying system.
  • the window manager may include a split-screen management service.
  • the split-screen management service can divide the screen size A of the display screen obtained from the underlying display system into the display area and the extended area for display, and obtain the screen size of the display area A1 and the size of the extended area A2, and use the size of the display area as the screen size of the mobile phone 100 for display.
  • the applications in the application package are divided into the display area, and the extended area is used as an area to be operated by the user. area.
  • the extended area can receive and distribute user operations in the extended area through an extended interactive module.
  • the extended interaction module receives event information corresponding to the operation used in the extended area.
  • the event information can be one of the touch screen event, touch screen touch event, touch screen sliding event, and user predetermined touch screen event.
  • the positioning strategy service can be used to receive event information transmitted by the extended interaction module, and obtain the screen size and position of the display area and the extended area from the underlying display system or window manager, and obtain the display area corresponding to the event information according to the mapping rules On the operation.
  • the mapping rule may be a preset operation that is pre-designed and stored in the system, for example, a mapping relationship is established between operation a in the extended area and operation b in the display area.
  • the positioning strategy service When the positioning strategy service is received, Expand the a operation of the area, and provide the corresponding b operation to the display system, so that the display system can operate according to the b operation provided by the positioning strategy service and drive the display area to display the b operation.
  • the preset operation may be a preset operation stored in advance in the system, or may be an operation set by itself according to the needs of the user.
  • the mapping rule can also be calculated based on the relationship between the coordinates of the original report point of the event information in the preset coordinate system, the display area and the screen size of the extended area, and the display corresponding to the event information is calculated. Operation of the area. For example, the screen size A1 of the display area used for display and the screen size A2 of the extended area are calculated to obtain the proportional relationship between A1 and A2. The coordinates of the original report point combined with the event information are calculated to obtain the corresponding event information Operate in the display area, and respond to the operation corresponding to the event information in the display area.
  • FIG. 5 shows a flowchart of a display method of an interactive system. As shown in FIG. 5, the method includes:
  • Step S501 Determine the display area and the extended area divided in the display screen, where the display area and the extended area can be two different areas belonging to the same physical display screen, for example, the display screen is a curved screen, a flat screen, or a flexible screen
  • the expansion area can be the expansion area set at the edge area or part of the area of the curved screen or flat screen, or the main screen or the auxiliary screen of the folding screen, or the folding area after the folding screen is folded.
  • the display screen is the display area.
  • the display area and the extended area can also be divided into two physical screens.
  • the division method can refer to the underlying display system in the above-mentioned embodiment for division, or the division of the management window of the window layer, which will not be repeated here.
  • the size of the two screens is determined and has been recorded by the system.
  • FIG. 6 is a schematic diagram of the architecture of the internal operating system of the interactive system according to an embodiment of the application.
  • the display area and the extended area can be connected through an extended interactive module to realize the transmission of event information in the extended area to the display area.
  • the extended area LCD2 is touched by the touch control.
  • the corresponding LCD2 event information is generated, and the extended interaction module sends the LCD2 event information to the input subsystem.
  • the input subsystem obtains the operation on the display area corresponding to the event information through the mapping rules of the positioning strategy service, and the display system and LCD Drive the LCD1 to power on and respond to the operation corresponding to the event information on the display area.
  • the positioning strategy service in this application can be set in the input subsystem or can exist independently of the input subsystem, which is not limited here.
  • step S502 event information is generated according to the user's operation on the extended area, where the event information includes one or more of touch screen events, long touch screen events, sliding touch screen events, and user-predetermined touch screen events.
  • Step S503 According to the event information, respond to the operation corresponding to the event information in the display area.
  • responding to the operation corresponding to the event information can include waking up the display area, or mapping the event information to the display area based on the relationship between the size of the display area and the extended area, and responding to the operation corresponding to the event information in the display area .
  • mapping the event information of the extended area to the display area will be described in detail in combination with multiple mapping scenarios.
  • Figure 7 shows a schematic diagram of the mapping scene 1.
  • the display screen of the mobile phone is exemplarily divided into the screen size A1 of the display area and the screen size A2 of the extended area.
  • the screen size of A1 and A2 It is a proportional relationship, but this application is not limited to this.
  • P2 is the user's click touch screen operation on the extended area. This operation can be received by the touch device and generate corresponding event information to determine that the operation is a click touch screen event and obtain the original report of the click touch screen event.
  • (a) in FIG. 8 shows the position of the display screen A in the preset coordinates
  • the display area A1 and the extended area A2 are in the Preset the position in the coordinate system, and then determine the coordinates of P2 in the preset coordinate system. Assuming that the coordinates of P2 are (W, 0.7H), the positioning strategy service is used to calculate P1 in the display area after P2 is mapped The coordinates of is (W, 0.2H), and the user influences the operation of the display area through the operation in the extended area.
  • Figure 9 shows a schematic diagram of the second mapping scenario.
  • the screen of the mobile phone is divided into display area A1 It is proportional to the screen size of the extended area A2, and only the A3 area in the extended area A2 is used for user input operations.
  • the specific mapping process can refer to the calculation method described in the mapping scenario 1.
  • FIG 10 shows a schematic diagram of the third mapping scenario.
  • the display screen of the mobile phone is exemplarily divided into a display area with a screen size of A1 and an extended area with a screen size of A2.
  • P1 is the user's click in the extended area
  • Touch operations can be mapped to sliding on the display area, such as sliding from P2 to P3, or the distance between P2 and P3 can be calculated by obtaining the time of the click and touch operation. For example, the more time the user clicks on the P1 position Longer, the longer the sliding distance from P2 to P3. It is also possible to map the click and touch operation in the extended area to the operation of simultaneously clicking on multiple points on the display area.
  • the mapping relationship may be a mapping rule pre-stored in the mobile phone and implemented by the positioning strategy service, for example,
  • the system stores the mapping rule that the event A from the extended area is equal to the event B in the display area.
  • the positioning strategy service receives the event A from the extended area, it will associate the event A with the event B in the display area according to the rules, and associate it with the event B in the display area.
  • the system combines the process of interactively affecting the display area by the extended area. For example, if the user clicks the touch operation P1 in the extended area, the positioning strategy service will respond to the operation of sliding the corresponding P2 to the P3 position in the display area according to the preset mapping rules. Among them, the user can also set preset operations according to needs.
  • the user can set a single-click touch operation equal to a double-click touch operation in the display area through the setting function of the extended area.
  • the mapping rule can be fully effective or partially effective.
  • the effective mode can also be set according to the needs of the user.
  • the overall effective means that the mapping rule is valid in any application and scenario, such as in any scenario.
  • the touch operation is clicked in the extended area, it will be mapped to the operation of sliding P2 to P3 in the display area, and it can also take effect in some scenes.
  • the user's touch operation in the extended area It can be used as a wake-up operation in the display area to light up the display area.
  • a single-click operation in the extended area is equivalent to a multi-key press in the game scene to achieve skill release, thereby enhancing the user's interactive experience.
  • the user can also set preset operations through the recording interface to achieve interactive definitions, for example, select common taps, long presses, swipes, two-finger sliding away from the extended area or directly in the extended area for a period of time.
  • the operation such as sliding a distance L on the screen.
  • the preset operation of the display area can be: tap, long press, slide, move away with two fingers, etc.
  • the button behavior of the display area can be recorded by the system, including all original button information such as coordinates, location, and duration.
  • the user can customize the operation of the extended area and the operation recorded in the display area to be established as a mapping rule.
  • the extended interactive module receives and distributes it to the positioning strategy service, and the positioning strategy service defines it
  • the preset operation is mapped to the preset operation recorded in the display area, and the display area responds to the operation recorded in the corresponding display area.
  • the interactive system of the present application includes a display screen 1101, a processing module 1102, and a generating module 1103.
  • the processing module 1102 is used to determine the display area and the extended area divided in the display screen, the generating module is used to generate event information according to the user's operation on the extended area, and the processing module is used to respond to events in the display area according to the event information.
  • the event information includes one or more of a touch screen tap event, a long touch screen event, a sliding touch screen event, a user-predetermined touch screen event, and screen recording.
  • the operation corresponding to the event information includes one or more of a touch screen event, a touch screen event, a touch screen sliding event, and a user-predetermined touch screen event.
  • the display area and the extended area are two different areas belonging to the same physical display screen.
  • the extended area is set on the curved screen or the flat screen.
  • the extended area is set on one of the folding screens, or the folding area of the folding screen after folding.
  • the display area and the extended area belong to two different physical display screens, for example, two folding screens, where one screen is used as the display area, and the other screen is used as the extended area.
  • the division of the display area and the extended area is determined by the underlying display system, the display screen is divided into the display area and the extended area, and the processing module 1102 is also used to: determine the original size and display area of the display screen The size of the display area and the screen size of the extended area; and notify the size of the display area as the screen size for system display, and use the display area for the display of the interactive system.
  • the division of the display area and the extended area is determined by the window layer, the display screen is divided into the display area and the extended area, and the processing module 1102 is also used to determine the original size and display area of the display screen. And the screen size of the extended area; and notify the screen size of the display area as the screen size for system display, and use the display area for the display of the interactive system.
  • the processing module 1102 is specifically configured to: determine the screen size of the display area and the extended area; map event information to the display area based on the relationship between the display area and the screen size of the extended area, and respond in the display area The operation corresponding to the event information.
  • the processing module 1102 is specifically further configured to: determine the position of the display area and the extended area in the preset coordinate system; based on the preset coordinate system, determine that the original report point of the event information is in the preset coordinate system The coordinates of the original report point and the display area and the extended area are determined; the coordinates of the original report point of the event information are mapped to the corresponding position of the display area according to the relationship between the coordinates, the screen size of the display area and the extended area.
  • the processing module 1102 is specifically further configured to: establish a mapping relationship between the preset operation of the extended area and the operation of the display area, so that the user inputting the preset operation in the extended area is mapped to the user's operation in the display area. .
  • the device 1200 may include one or more processors 1201 coupled to the controller hub 1203.
  • the controller hub 1203 is connected via a multi-branch bus such as a Front Side Bus (FSB), a point-to-point interface such as Quick Path Interconnect (QPI), or a similar connection 1206 communicates with the processor 1201.
  • the processor 1201 executes instructions that control general types of data processing operations.
  • the controller hub 1203 includes, but is not limited to, a graphics memory controller hub (Graphics Memory Controller Hub, GMCH) (not shown) and an input/output hub (Input Output Hub, IOH) (which may On a separate chip) (not shown), where the GMCH includes a memory and a graphics controller and is coupled to the IOH.
  • GMCH Graphics Memory Controller Hub
  • IOH input/output hub
  • the device 1200 may also include a coprocessor 1202 and a memory 1204 coupled to the controller hub 1203.
  • a coprocessor 1202 and a memory 1204 coupled to the controller hub 1203.
  • one or both of the memory and the GMCH may be integrated in the processor (as described in this application), and the memory 1204 and the coprocessor 1202 are directly coupled to the processor 1201 and the controller hub 1203.
  • the controller hub 1203 and IOH are in a single chip.
  • the memory 1204 may be, for example, a dynamic random access memory (Dynamic Random Access Memory, DRAM), a phase change memory (Phase Change Memory, PCM), or a combination of the two.
  • DRAM Dynamic Random Access Memory
  • PCM Phase Change Memory
  • the coprocessor 1202 is a dedicated processor, such as, for example, a high-throughput MIC processor (Many Integerated Core, MIC), a network or communication processor, a compression engine, a graphics processor, a general graphics processor (General Purpose Computing on GPU, GPGPU), or embedded processor, etc.
  • MIC Manufacturing Integerated Core
  • GPGPU General Purpose Computing on GPU
  • embedded processor etc.
  • the optional properties of the coprocessor 1202 are shown in dashed lines in FIG. 12.
  • the memory 1204 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions.
  • the memory 1204 may include any suitable non-volatile memory such as flash memory and/or any suitable non-volatile storage device, such as one or more hard disk drives (Hard-Disk Drive, HDD(s)), one or Multiple Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives.
  • any suitable non-volatile memory such as flash memory and/or any suitable non-volatile storage device, such as one or more hard disk drives (Hard-Disk Drive, HDD(s)), one or Multiple Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives.
  • HDD hard disk drives
  • CD Compact Disc
  • DVD Digital Versatile Disc
  • the device 1200 may further include a network interface (Network Interface Controller, NIC) 1206.
  • the network interface 1206 may include a transceiver, which is used to provide a radio interface for the device 1200 to communicate with any other suitable devices (such as a front-end module, an antenna, etc.).
  • the network interface 1206 may be integrated with other components of the device 1200.
  • the network interface 1206 can realize the function of the communication unit in the above-mentioned embodiment.
  • the device 1200 may further include an input/output (Input/Output, I/O) device 1205.
  • I/O 1205 may include: a user interface, which is designed to enable users to interact with the device 1200; the design of the peripheral component interface enables peripheral components to also interact with the device 1200; and/or a sensor is designed to determine the environment related to the device 1200 Conditions and/or location information.
  • FIG. 12 is only exemplary. That is, although the device 1200 shown in FIG. 12 includes multiple devices such as the processor 1201, the controller hub 1203, and the memory 1204, in actual applications, the devices using the methods of the present application may only include the devices of the device 1200. Some of the devices in, for example, may only include the processor 1201 and the NIC 1206. The properties of optional devices in Fig. 12 are shown by dashed lines.
  • the memory 1204 which is a computer-readable storage medium, stores instructions that when executed on a computer causes the system 1200 to execute the method of the interactive system in the above-mentioned embodiment.
  • the system 1200 stores instructions that when executed on a computer causes the system 1200 to execute the method of the interactive system in the above-mentioned embodiment.
  • the method is not repeated here.
  • SoC1300 includes: an interconnection unit 1350, which is coupled to an application processor 1310; a system agent unit 1380; a bus controller unit 1390; an integrated memory controller unit 1340; a group or one or more coprocessors 1320, which may include integrated graphics logic, image processors, audio processors, and video processors; a static random access memory (SRAM) unit 1330; and a direct memory access (DMA) unit 1360.
  • the coprocessor 1320 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.
  • the static random access memory (SRAM) unit 1330 may include one or more computer-readable media for storing data and/or instructions.
  • the computer-readable storage medium may store instructions, specifically, temporary and permanent copies of the instructions.
  • the instruction may include: when executed by at least one unit in the processor, the Soc1300 executes the display method of the interactive system according to the foregoing embodiment. For details, refer to the display method of the foregoing embodiment, and details are not described herein again.
  • the various embodiments of the mechanism disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods.
  • the embodiments of the present application can be implemented as a computer program or program code executed on a programmable system.
  • the programmable system includes at least one processor and a storage system (including volatile and non-volatile memory and/or storage elements) , At least one input device and at least one output device.
  • Program codes can be applied to input instructions to perform the functions described in this application and generate output information.
  • the output information can be applied to one or more output devices in a known manner.
  • the processing system includes any processor having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor. system.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • the program code can be implemented in a high-level programming language or an object-oriented programming language to communicate with the processing system.
  • assembly language or machine language can also be used to implement the program code.
  • the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.
  • the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof.
  • the disclosed embodiments can also be implemented as instructions carried by or stored on one or more transient or non-transitory machine-readable (eg, computer-readable) storage media, which can be executed by one or more processors Read and execute.
  • the instructions can be distributed through a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (for example, a computer), including, but not limited to, floppy disks, optical disks, optical disks, and compact disc read-only memory (Compact Disc Read Only Memory).
  • a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (for example, a computer).
  • each unit/module mentioned in each device embodiment of this application is a logical unit/module.
  • a logical unit/module can be a physical unit/module or a physical unit/ A part of the module can also be realized by a combination of multiple physical units/modules.
  • the physical realization of these logical units/modules is not the most important.
  • the combination of the functions implemented by these logical units/modules is the solution to this application.
  • the above-mentioned device embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems proposed by this application. This does not mean that the above-mentioned device embodiments do not exist. Other units/modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种交互系统的显示方法、交互系统及电子设备，所述交互系统包括显示屏，所述交互系统的显示方法包括：确定所述显示屏中划分出的显示区域和扩展区域（S501）；根据用户针对所述扩展区域的操作生成事件信息（S502）；根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作（S503）。

Description

交互系统的显示方法、交互系统及电子设备
本申请要求于2019年12月18日提交中国专利局、申请号为201911312368.0、申请名称为“交互系统的显示方法、交互系统及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人机交互技术领域，尤其涉及交互系统的显示方法、交互系统、电子设备及计算机可读存储介质。
背景技术
目前电子设备的超曲面屏，如柔性屏，折叠屏等产品形态，这些产品给用户带来的全新的交互方式。此外，为了提高用户的使用体验，屏幕的尺寸越来越大，在某些场景下，例如单手可能很难实现对屏幕的操作，或在某些应用场景下，用户需要同时触按多个位置，或多次触控屏幕，才能完成交互，交互的体验需要进一步提高。
发明内容
有鉴于此，本申请提供一种交互系统的显示方法、交互系统、电子设备及计算机可读存储介质，能够为用户带来全新的交互方式，提升用户的体验。
本申请的一些实施方式，提供了一种交互系统的显示方法。以下从多个方面介绍本申请，以下多个方面的实施方式和有益效果可互相参考。
第一方面，本申请提供一种交互系统的显示方法，应用于交互系统，该交互系统包括显示屏，该方法包括：确定所述显示屏中划分出的显示区域和扩展区域；根据用户针对所述扩展区域的操作生成事件信息，其中，对扩展区域的操作，可以是用户的触控行为，也可以是行为姿势等，事件信息可以包括点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作，其中，与所述事件信息对应的操作包括：点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
根据本申请的交互系统的显示方法，用户可以通过在扩展区域的操作，影响显示区域的操作，给用户带来全新的交互方式，提升用户的交互体验。
在上述第一方面的一种可能的实现中，所述显示区域和所述扩展区域为属于同一物理显示屏上的两个不同区域，例如，可以是柔性屏中的折叠屏的第一屏和第二屏，曲面屏上的平面部分和曲面部分，也可以是平面屏分为两部分等。
在上述第一方面的一种可能的实现中，所述显示区域和所述扩展区域的划分由底层显示系统确定，具体地，将所述显示屏划分为显示区域和扩展区域，包括：由底层显示系统确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的 屏幕大小，以及且底层显示系统将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
在上述第一方面的一种可能的实现中，所述显示区域和所述扩展区域的划分由窗口层确定，具体地，由窗口层将所述显示屏划分为显示区域和扩展区域，包括：确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小，可以通过用户分屏操作指令划定显示区域和扩展区域大小，进而确定显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小；以及将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
在上述第一方面的一种可能的实现中，所述显示区域和所述扩展区域为分别属于两块不同的物理显示屏，例如，物理上分割的折叠屏，将其中一块屏作为显示区域，另一块屏作为扩展区域，且两块屏的大小是固定的。
在上述第一方面的一种可能的实现中，根据所述事件信息响应与所述请求消息对应的操作，包括：确定显示区域和扩展区域的屏幕大小；基于所述显示区域和所述扩展区域的屏幕大小间的关系将所述事件信息映射至所述显示区域，并在所述显示区域响应与所述事件信息对应的操作。
在上述第一方面的一种可能的实现中，根据所述显示区域和所述扩展区域的尺寸大小间的关系将所述事件信息映射至所述显示区域，包括：确定所述显示区域、所述扩展区域在预设坐标系下的位置；基于所述预设坐标系，确定所述事件信息的原始报点在所述预设坐标系中的坐标，并确定所述原始报点与所述显示区域和扩展区域的相对位置；根据所述坐标、所述显示区域和扩展区域的尺寸大小间关系将所述事件信息的原始报点的坐标映射到所述显示区域对应的位置，也就是说，通过映射关系将所述事件信息的原始报点的坐标映射到所述显示区域对应的位置，进而可以实现通过扩展区域的操作影响显示区域的操作，给用户带来全新的交互方式。
在上述第一方面的一种可能的实现中，据所述事件信息，在所述显示区域响应与所述事件信息对应的操作，包括：将所述扩展区域的预设操作与所述显示区域的操作建立映射关系，以使用户在扩展区域输入所述预设操作映射为用户在所述显示区域的操作，也就是说，可以通过在系统内预先设置好映射关系，根据映射关系将用户在扩展区域内输入的预设操作映射为在显示区域的操作，其中，该预设操作可以是系统预先设定的，也可以是根据用户的自己的需求进行设定，进一步提升用户的交互体验。
在上述第一方面的一种可能的实现中，所述显示屏为曲面屏、平面屏或折叠屏，所述扩展区域设置在所述曲面屏或平面屏的边缘区域或部分区域，所述扩展区域设置在所述折叠屏的其中一块屏上，或所述一块屏的一部分，或所述折叠屏折叠后的折叠区域。
第二方面，本申请提供一种交互系统，包括显示屏、处理模块和生成模块，用于确定所述显示屏中划分出的显示区域和扩展区域，用于根据用户针对所述扩展区域的操作生成事件信息，其中，事件信息包括点击触屏事件、长按触屏事件、滑动触屏事件、用户预定触屏事件和屏幕录制中的一种或多种，所述处理模块用于根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作，其中，与所述事件信息对应 的操作包括点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
根据本申请实施例的交互系统，用户可以通过在扩展区域的操作，影响显示区域的操作，给用户带来全新的交互方式，提升用户的交互体验。。
在上述第二方面的一种可能的实现中，所述显示区域和所述扩展区域为属于同一物理显示屏上的两个不同区域，例如，可以是柔性屏中的折叠屏的第一屏和第二屏，曲面屏上的平面部分和曲面部分，也可以是平面屏分为两部分等。
在上述第二方面的一种可能的实现中，所述显示区域和所述扩展区域的划分由底层显示系统确定，将所述显示屏划分为显示区域和扩展区域，所述处理模块还用于：确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的大小；以及将所述显示区域的大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
在上述第二方面的一种可能的实现中，所述显示区域和所述扩展区域的划分由窗口层确定，将所述显示屏划分为显示区域和扩展区域，所述处理模块还用于：确定所述显示屏的屏幕原始尺寸大小、显示区域的大小以及扩展区域的屏幕大小；以及将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
在上述第二方面的一种可能的实现中，所述显示区域和所述扩展区域为分别属于两块不同的物理显示屏，例如，物理上分割的折叠屏，将其中一块屏作为显示区域，另一块屏作为扩展区域，且两块屏的屏幕大小是固定的。
在上述第二方面的一种可能的实现中，所述处理模块具体用于：确定显示区域和扩展区域的屏幕大小；基于所述显示区域和所述扩展区域的屏幕大小间的关系将所述事件信息映射至所述显示区域，并在所述显示区域响应与所述事件信息对应的操作。
在上述第二方面的一种可能的实现中，所述处理模块具体还用于：确定所述显示区域、所述扩展区域在预设坐标系下的位置；基于所述预设坐标系，确定所述事件信息的原始报点在所述预设坐标系中的坐标，并确定所述原始报点与所述显示区域和扩展区域的相对位置；根据所述坐标、所述显示区域和扩展区域的屏幕大小间关系将所述事件信息的原始报点的坐标映射到所述显示区域对应的位置。
在上述第二方面的一种可能的实现中，所述处理模块具体还用于：将所述扩展区域的预设操作与所述显示区域的操作建立映射关系，以使用户在扩展区域输入所述预设操作映射为用户在所述显示区域的操作，也就是说，可以通过在系统内预先设置好映射关系，根据映射关系将用户在扩展区域内输入的预设操作映射为在显示区域的操作，其中，该预设操作可以是系统预先设定的，也可以是根据用户的自己的需求进行设定，进一步提升用户的交互体验。
在上述第二方面的一种可能的实现中，所述显示屏为曲面屏、平面屏或折叠屏，所述扩展区域设置在所述曲面屏或平面屏的边缘区域或部分区域，所述扩展区域设置在所述折叠屏的其中一块屏上，或所述一块屏的一部分，或所述折叠屏折叠后的折叠区域。
第三方面，本申请提供一种的电子设备，包括处理器和存储器，所述存储器中存 储有指令，所述处理器，用于读取所述存储器中存储的所述指令，以执行上述第一方面实施例的方法。
第四方面，本申请提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，计算机程序被处理器运行时，使得处理器执行上述第一方面实施例的方法。
附图说明
图1为本申请一个实施例的交互系统的结构示意图；
图2为本申请一个实施例的交互系统的显示屏上的扩展区域和显示区域的分布的结构示意图；
图3为本申请一个实施例的交互系统的软件结构示意图；
图4为本申请一个实施例的交互系统的显示屏的结构示意图；
图5为本申请一个实施例的交互系统显示方法的流程图；
图6为本申请一个实施例的交互系统内部操作系统的架构示意图；
图7为本申请一个实施例的交互系统的显示方法的映射场景一的结构示意图；
图8为本申请一个实施例的显示屏、扩展区域和显示区域在预设坐标系中的位置结构示意图；
图9为本申请一个实施例的交互系统的显示方法的映射场景二的结构示意图；
图10为本申请一个实施例的交互系统的显示方法的映射场景三的结构示意图；
图11为本申请一个实施例的交互系统的结构示意图；
图12为本申请一些实施例的一种设备的框图；
图13为本申请一些实施例的一种片上系统(SoC)的框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面将结合附图对本申请的实施例作进一步地描述。
本申请实施例提供的一种交互系统的显示方法，可以用于手机、平板电脑、笔记本电脑、个人计算机、上网本、手持计算机、个人数字助理、可穿戴设备等具有显示屏的电子设备中，对此本申请不做任何限制。
下面以手机为例，对本申请的交互系统的显示方法进行描述，图1示出了手机的结构示意图。
手机100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber  identificat ion module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本申请实施例示意的结构并不构成对手机100的具体限定。在本申请另一些实施例中，手机100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
处理器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数 据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
手机100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。手机100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在手机100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
无线通信模块160可以提供应用在手机100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，手机100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code  division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
手机100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
在本申请的一个实施例中，显示屏194可以包括显示面板和显示器，如触摸传感器180K，显示器用于向用户输出显示内容，并将显示内容在显示面板上显示，如显示图像，视频等。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。
其中，触控器件为触摸传感器180K。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸事件相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于手机100的表面，与显示屏194所处的位置不同。
需要说明的是，触控器件的数量可以根据显示区域或显示屏的数量可以设置一个或多个，例如，显示屏的扩展区域和显示区域可以分别对应有一触控器件，以接收来自不同区域的用户的操作并生成的触摸事件。
手机100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图 像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，手机100可以包括1个或N个摄像头193，N为大于1的正整数。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当手机100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。手机100可以支持一种或多种视频编解码器。这样，手机100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现手机100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展手机100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储手机100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行手机100的各种功能应用以及数据处理。
手机100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。手机100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，手机100根据压力传感器180A检测触摸操作强度。手机100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。
在本申请的实施例中，手机100的显示屏194被划分为显示区域和扩展区域，其中，显示区域和扩展区域可以同属于物理上的一块显示屏，也可以是分别属于物理上 的两块屏，如折叠屏中的一种，可以将两块屏中的一个屏作为显示区域，另一个屏作为扩展区域，扩展区域可以设置有触控器件，用于接收用户针对扩展区域的操作，并生成对应的触摸事件，例如点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
如图2中的(a)所示，扩展区域可以是设置在曲面屏或平面屏的两侧的边缘区域，用户可以在两侧的扩展区域操作，而在显示区域响应该操作。
如图2中的(b)所示，扩展区域可以是设置在曲面屏或平面屏的一侧的部分边缘区域，用户可以在一侧的扩展区域操作，而在显示区域响应该操作。
如图2中的(c)所示，扩展区域可以是折叠屏中的其中一个屏，例如副屏，显示区域为主屏，用户可以在副屏上操作，而在主屏上响应该操作。
如图2中的(d)所示，扩展区域可以是折叠屏中可折叠的区域，用户可以在可折叠区域上操作，而在作为显示区域的主屏或副屏上响应该操作。
在本申请的实施例中，扩展区域可以通过触控器件来接收用户的触摸操作，而显示区域可以作为常规的独立的显示屏之外，还用于响应来自扩展区域的用户的触摸操作。扩展区域无论是有无触摸操作都可以是黑屏状态，或者是只用来显示时间、天气、备忘信息等。
上述手机100的软件系统可以采用分层架构，手机100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明手机100的软件结构。
图3是本申请实施例的手机100的软件结构框图。
分层架构将软件分成若干个层，每一层都有各自的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。
如图3所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图3所示，应用程序框架层可以包括窗口管理器，定位策略服务，扩展交互模块，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。本申请中其他实施例中，定位策略服务和扩展交互模块也可以设置在底层系统中，在此并不作为唯一限定。
如图3所示，内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
如图3所示，系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。
如图3所示，应用程序框架层以下的系统库和内核层还可称为底层系统，底层系 统中包括用于提供显示服务的底层显示系统，例如，底层显示系统可以包括内核层中的显示驱动，以及系统库中的表面管理器等。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。2D图形引擎是2D绘图的绘图引擎。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
根据本申请的一个实施例，底层显示系统可以获得显示区域的屏幕尺寸大小、扩展区域的屏幕尺寸大小和显示屏的屏幕原始尺寸大小，并且告知整个系统用于手机100显示的区域为显示区域的屏幕尺寸大小。
例如，以显示屏为LCD显示屏为例，如图4中的(a)和(b)所示，显示屏的屏幕原始尺寸大小为A，实际用于显示的显示区域的屏幕尺寸大小为A1，显示驱动的显示配置将显示屏的屏幕尺寸A改为A1，即，虽然显示屏的原始屏幕尺寸大小为A，但是底层显示系统告知整个系统的用于显示的屏幕的尺寸为A1(显示区域的屏幕尺寸)，其中，告知可以采用安卓系统的显示配置信息命令(config)进行配置，并且修改对获取用于显示的屏幕尺寸的接口。
窗口管理器用于管理窗口程序。窗口管理器可以从底层显示系统中获取的用于显示的显示区域的屏幕尺寸大小A1，即调用已经被底层显示系统修改后的屏幕尺寸的接口，以得到用于显示的显示区域的屏幕尺寸大小A1，进而，用户在显示屏上的任一位置的操作都可以在手机100的显示屏上的显示区域上响应并显示，其中，扩展区域可以通过扩展交互模块来接收和分发用户在扩展区域的操作。
根据本申请的另一个实施例，如图4示例性的示出了显示屏和扩展区域和显示区域的屏幕尺寸大小，如图4中的(a)和(b)所示，底层显示系统可以获得显示屏的屏幕原始尺寸大小A，且窗口管理器从底层系统中获取显示屏的屏幕原始尺寸大小A。窗口管理器可以包括分屏管理服务，分屏管理服务可以将从底层显示系统中获取的显示屏的屏幕尺寸大小A划分出用于显示的显示区域和扩展区域，以及获得显示区域的屏幕尺寸大小A1和扩展区域的尺寸大小A2，并将显示区域的尺寸大小作为手机100的用于显示的屏幕尺寸大小，将应用程序包内的应用划分到显示区域，而在扩展区域作为一个待用户操作的区域。其中，扩展区域可以通过扩展交互模块来接收和分发用户在扩展区域的操作。
扩展交互模块收到与用于在扩展区域的操作对应的事件信息，例如，事件信息可以为点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多 种，或者可以理解为用户的操作在扩展区域触摸时，原始报点的坐标和时间戳等。
定位策略服务可用于接收扩展交互模块传输的事件信息，并且从底层显示系统中或窗口管理器中获取显示区域和扩展区域的屏幕尺寸大小和位置，并根据映射规则得到与事件信息对应的显示区域上的操作。
根据本申请的一个实施例，映射规则可以是预先设计好的存储在系统中的预设操作，例如，将扩展区域的a操作与显示区域的b操作建立映射关系，当定位策略服务收到来至扩展区域的a操作,并向显示系统提供对应的b操作，以使显示系统根据定位策略服务提供的b操作并驱动显示区域显示b操作。其中，预设操作可以是系统内事先存储的预设操作，也可以是根据用于的需求自行设定的操作。
根据本申请的另一个实施例，映射规则也可以是根据事件信息的原始报点在预设坐标系中的坐标、显示区域和扩展区域的屏幕大小间关系计算得到与该事件信息对应的在显示区域的操作。例如，用于显示的显示区域的屏幕尺寸大小A1和扩展区域的屏幕尺寸大小A2，计算得到A1和A2的比例关系，在结合事件信息的原始报点的坐标通过计算得到与该事件信息对应的在显示区域的操作，并在显示区域响应与事件信息对应的操作。其中，A1和A2的比例关系包括等比例关系，即A1：A2＝1：1，或非等比例关系，即A1大于A2，或A1小于A2。
下面以手机作为交互系统为例，并结合附图详细的说明本申请提供的一种交互系统的显示方法。
在本申请的实施例中，图5示出了交互系统显示方法的流程图，如图5所示，该方法包括：
步骤S501，确定显示屏中划分出的显示区域和扩展区域，其中，显示区域和扩展区域可以为属于同一物理显示屏上的两个不同区域，例如，显示屏为曲面屏、平面屏或柔性屏的折叠屏时，其中扩展区域可以是扩展区域设置在曲面屏或平面屏的边缘区域或部分区域，或者，折叠屏的主屏或副屏，或者是折叠屏折叠后的折叠区域，除此之外的显示屏为显示区域。显示区域和扩展区域也可以分属于物理上的两块屏。
其中，显示区域和扩展区域同属于一块屏时，其划分方式可以参考上述实施例中的底层显示系统进行划分，或者窗口层的管理窗口器进行划分，在此不再赘述。
当显示区域和扩展区域为分属于物理上不同的两块屏时，两块屏的大小是确定的，且已经被系统记录。
图6为本申请一个实施例的交互系统内部操作系统的架构示意图。如图3所示和图6所示，显示区域与扩展区域可通过扩展交互模块连接，以实现将扩展区域的事件信息传输至显示区域，例如，扩展区域LCD2发生触屏的操作，由触摸控件生成对应的LCD2事件信息，并由扩展交互模块将LCD2事件信息发送至输入子系统，输入子系统通过定位策略服务的映射规则得到与事件信息对应的显示区域上的操作，进一步由显示系统和LCD驱动给LCD1上电，并在显示区域上响应与事件信息对应的操作，本申请中的定位策略服务可以设置与输入子系统中，也可以独立于输入子系统存在，在此并不作为限定。
步骤S502，根据用户针对扩展区域的操作生成事件信息，其中，事件信息包括点 击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
步骤S503，根据事件信息，在显示区域响应与事件信息对应的操作。其中，响应与事件信息对应的操作，可以包括唤醒显示区域，也可以是基于显示区域和扩展区域的尺寸大小间的关系将事件信息映射至显示区域，并在显示区域响应与事件信息对应的操作。
下面以手机为例，结合多种映射场景对扩展区域的事件信息映射至显示区域的过程进行详细的说明。
图7示出了映射场景一的示意图，如图7所示，手机的显示屏示例性地划分为显示区域的屏幕大小A1和扩展区域的屏幕大小A2，作为一具体的实例，A1与A2的为等比例关系，但是本申请不限于此。P2为用户在扩展区域上的单击触屏操作，该操作可以由触控器件接收并生成对应的事件信息，以确定该操作为单击触屏事件并获取该单击触屏事件的原始报点P2在预设坐标系中的位置，图8中的(a)示出了显示屏A在预设坐标中的位置，图8中的(b)示出了显示区域A1和扩展区域A2在预设坐标系中的位置，进而确定P2在预设坐标系中的坐标，假设P2的坐标为(W，0.7H)，则经过定位策略服务进行计算得到P2经过映射后在显示区域中的P1的坐标为(W，0.2H)，进而用户通过在扩展区域的操作影响显示区域的操作。
图9示出了映射场景二的示意图，扩展区域中也可以是只有部分区域用于用户的操作，其他部分用于设置虚拟按键等，如图9所示，手机的显示屏划分为显示区域A1和扩展区域A2的屏幕大小为等比例关系，而在扩展区域A2中只有A3区域用于用户的输入操作，具体的映射过程可参考映射场景一中描述的计算方法。此外，还可以结合A3与A1的屏幕大小比例关系，对A3上的操作进行等比例缩放，例如，A1：A3为3:1，当用户在A3上两个不同的位置点击P1和P2，可得到两点之间的距离L，并通过坐标计算后，在显示区域应当得到对应位置的两点坐标，将L等比例放大3倍后，并确定为映射在显示区域上最终确定的两点的位置P3和P4。需要说明的是，A3区域也可以直接作为扩展区域，其映射过程可参考场景二，在此不再赘述。
图10示出了映射场景三的示意图，如图10所示，手机的显示屏示例性地划分为显示区域的屏幕大小为A1和扩展区域的屏幕大小A2，P1为用户在扩展区域的单击触摸操作，可映射为在显示区域上的滑动，例如从P2滑到P3，也可以通过获取单击触摸操作的时间来计算P2到P3之间的距离，例如，用户在P1位置点击的时间越长，P2到P3的滑动距离越长。也可以通过在扩展区域的单击触摸操作映射为在显示区域上的多点位的同时点击的操作，该映射关系可以是预先存储在手机内的映射规则，并由定位策略服务实施，例如，系统中存储有来至扩展区域的A事件等于显示区域的B事件的映射规则，一旦定位策略服务接收到来自扩展区域的A事件，根据规则，将A事件关联显示区域的B事件，并与显示系统结合实现由扩展区域交互影响显示区域的过程，例如，用户在扩展区域单击触摸操作P1，定位策略服务根据预先设定的映射规则就会在显示区域响应对应的P2滑到P3位置的操作，其中，用户也可以根据需要设定的预设操作，例如，用户可以通过扩展区域的设置功能，设置单击触摸操作等于在显示区域的双击触摸操作。此外，该映射规则可以整体生效或局部生效的模式，该生效模式也可以根据用户的需要进行设定，其中，整体生效是指在任何应用和场景下该映射规 则都成立，如在任何场景下只要在扩展区域单击触摸操作都会映射为在显示区域的P2滑到P3位置的操作，也可以在部分场景下生效，例如，只有在显示区域为睡眠状态下，用户在扩展区域的触摸操作才可以作为在显示区域的唤醒操作，以点亮显示区域。又例如，在王荣耀游戏的场景中，在扩展区域的单击操作相当于游戏场景中的多键合按以实现技能释放，进而提升用户的交互体验。
根据本申请的一个实施例，用户也可以通过录制界面设置预设操作以实现交互定义，例如，在扩展区域选择常见的点击、长按、滑动、两指远离滑动或者直接在扩展区域进行一段时间的操作，如在屏幕滑动一段距离L。则显示区域的预设操作可以为：点击、长按、滑动、两指远离等。
在扩展区域的预设操作定义并录制完成后，在显示区域进行录制，显示区域的按键行为可以被系统记录，包括，坐标，位置，时长等一切原始按键信息。用户可以自定义将扩展区域的操作和显示区域录制的操作建立成为映射规则，当用户在扩展区域输入预设操作，由扩展交互模块接收和分发给定位策略服务，并由定位策略服务将该定义的预设操作映射为显示区域录制的预设操作，并在显示区域响应对应的显示区域录制的操作。
基于上述实施例的描述，具体描述本申请的交互系统，如图11所示，该交互系统包括显示屏1101、处理模块1102和生成模块1103。
具体地，处理模块1102用于确定显示屏中划分出的显示区域和扩展区域，生成模块用于根据用户针对扩展区域的操作生成事件信息，处理模块用于根据事件信息，在显示区域响应与事件信息对应的操作。其中，事件信息包括：点击触屏事件、长按触屏事件、滑动触屏事件、用户预定触屏事件和屏幕录制中的一种或多种。与事件信息对应的操作，包括：点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
根据本申请的一个实施例，显示区域和扩展区域为属于同一物理显示屏上的两个不同区域，例如，当显示屏为曲面屏、平面屏或折叠屏，扩展区域设置在曲面屏或平面屏的边缘区域或部分区域，扩展区域设置在折叠屏的其中一块屏上，或折叠屏折叠后的折叠区域。
根据本申请的一个实施例，显示区域和扩展区域为分别属于两块不同的物理显示屏，例如两块屏的折叠屏，其中一块屏作为显示区域，另一块屏作为扩展区域。
根据本申请的一个实施例，显示区域和扩展区域的划分由底层显示系统确定，将显示屏划分为显示区域和扩展区域，处理模块1102还用于：确定显示屏的屏幕原始尺寸大小、显示区域的大小以及扩展区域的屏幕大小；以及将显示区域的大小通知作为用于系统显示的屏幕大小，并且将显示区域用于交互系统的显示。
根据本申请的另一个实施例，显示区域和扩展区域的划分由窗口层确定，将显示屏划分为显示区域和扩展区域，处理模块1102还用于：确定显示屏的屏幕原始尺寸大小、显示区域的大小以及扩展区域的屏幕大小；以及将显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将显示区域用于交互系统的显示。
根据本申请的一个实施例，处理模块1102具体用于：确定显示区域和扩展区域的屏幕大小；基于显示区域和扩展区域的屏幕大小间的关系将事件信息映射至显示区域， 并在显示区域响应与事件信息对应的操作。
根据本申请的一个实施例，处理模块1102具体还用于：确定显示区域、扩展区域在预设坐标系下的位置；基于预设坐标系，确定事件信息的原始报点在预设坐标系中的坐标，并确定原始报点与显示区域和扩展区域的相对位置；根据坐标、显示区域和扩展区域的屏幕大小间关系将事件信息的原始报点的坐标映射到显示区域对应的位置。
根据本申请的一个实施例，处理模块1102具体还用于：将扩展区域的预设操作与显示区域的操作建立映射关系，以使用户在扩展区域输入预设操作映射为用户在显示区域的操作。
现在参考图12，所示为根据本申请的一个实施例的设备1200的框图。设备1200可以包括耦合到控制器中枢1203的一个或多个处理器1201。对于至少一个实施例，控制器中枢1203经由诸如前端总线(Front Side Bus，FSB)之类的多分支总线、诸如快速通道互连(Quick Path Interconnect，QPI)之类的点对点接口、或者类似的连接1206与处理器1201进行通信。处理器1201执行控制一般类型的数据处理操作的指令。在一实施例中，控制器中枢1203包括，但不局限于，图形存储器控制器中枢(Graphics Memory Controller Hub，GMCH)(未示出)和输入/输出中枢(Input Output Hub，IOH)(其可以在分开的芯片上)(未示出)，其中GMCH包括存储器和图形控制器并与IOH耦合。
设备1200还可包括耦合到控制器中枢1203的协处理器1202和存储器1204。或者，存储器和GMCH中的一个或两者可以被集成在处理器内(如本申请中所描述的)，存储器1204和协处理器1202直接耦合到处理器1201以及控制器中枢1203，控制器中枢1203与IOH处于单个芯片中。存储器1204可以是例如动态随机存取存储器(Dynamic Random Access Memory，DRAM)、相变存储器(Phase Change Memory，PCM)或这两者的组合。在一个实施例中，协处理器1202是专用处理器，诸如例如高吞吐量MIC处理器(Many Integerated Core，MIC)、网络或通信处理器、压缩引擎、图形处理器、通用图形处理器(General Purpose Computing on GPU，GPGPU)、或嵌入式处理器等等。协处理器1202的任选性质用虚线表示在图12中。
存储器1204作为计算机可读存储介质，可以包括用于存储数据和/或指令的一个或多个有形的、非暂时性计算机可读介质。例如，存储器1204可以包括闪存等任何合适的非易失性存储器和/或任何合适的非易失性存储设备，例如一个或多个硬盘驱动器(Hard-Disk Drive，HDD(s))，一个或多个光盘(Compact Disc，CD)驱动器，和/或一个或多个数字通用光盘(Digital Versatile Disc，DVD)驱动器。
在一个实施例中，设备1200可以进一步包括网络接口(Network Interface Controller，NIC)1206。网络接口1206可以包括收发器，用于为设备1200提供无线电接口，进而与任何其他合适的设备(如前端模块，天线等)进行通信。在各种实施例中，网络接口1206可以与设备1200的其他组件集成。网络接口1206可以实现上述实施例中的通信单元的功能。
设备1200可以进一步包括输入/输出(Input/Output，I/O)设备1205。I/O 1205可以包括：用户界面，该设计使得用户能够与设备1200进行交互；外围组件接口的设计使得外围组件也能够与设备1200交互；和/或传感器设计用于确定与设备1200相关 的环境条件和/或位置信息。
值得注意的是，图12仅是示例性的。即虽然图12中示出了设备1200包括处理器1201、控制器中枢1203、存储器1204等多个器件，但是，在实际的应用中，使用本申请各方法的设备，可以仅包括设备1200各器件中的一部分器件，例如，可以仅包含处理器1201和NIC1206。图12中可选器件的性质用虚线示出。
根据本申请的一些实施例，作为计算机可读存储介质的存储器1204上存储有指令，该指令在计算机上执行时使系统1200执行根据上述实施例中的交互系统的方法，具体可参照上述实施例的方法，在此不再赘述。
现在参考图13，所示为根据本申请的一实施例的SoC(System on Chip，片上系统)1300的框图。在图13中，相似的部件具有同样的附图标记。另外，虚线框是更先进的SoC的可选特征。在图13中，SoC1300包括：互连单元1350，其被耦合至应用处理器1310；系统代理单元1380；总线控制器单元1390；集成存储器控制器单元1340；一组或一个或多个协处理器1320，其可包括集成图形逻辑、图像处理器、音频处理器和视频处理器；静态随机存取存储器(Static Random Access Memory，SRAM)单元1330；直接存储器存取(DMA)单元1360。在一个实施例中，协处理器1320包括专用处理器，诸如例如网络或通信处理器、压缩引擎、GPGPU、高吞吐量MIC处理器、或嵌入式处理器等。
静态随机存取存储器(SRAM)单元1330中可以包括用于存储数据和/或指令的一个或多个计算机可读介质。计算机可读存储介质中可以存储有指令，具体而言，存储有该指令的暂时和永久副本。该指令可以包括：由处理器中的至少一个单元执行时使Soc1300执行根据上述实施例中的交互系统的显示方法，具体可参照上述实施例的显示方法，在此不再赘述。
本申请公开的机制的各实施例可以被实现在硬件、软件、固件或这些实现方法的组合中。本申请的实施例可实现为在可编程系统上执行的计算机程序或程序代码，该可编程系统包括至少一个处理器、存储系统(包括易失性和非易失性存储器和/或存储元件)、至少一个输入设备以及至少一个输出设备。
可将程序代码应用于输入指令，以执行本申请描述的各功能并生成输出信息。可以按已知方式将输出信息应用于一个或多个输出设备。为了本申请的目的，处理系统包括具有诸如例如数字信号处理器(Digital Signal Processor，DSP)、微控制器、专用集成电路(Application Specific Integrated Circuit，ASIC)或微处理器之类的处理器的任何系统。
程序代码可以用高级程序化语言或面向对象的编程语言来实现，以便与处理系统通信。在需要时，也可用汇编语言或机器语言来实现程序代码。事实上，本申请中描述的机制不限于任何特定编程语言的范围。在任一情形下，该语言可以是编译语言或解释语言。
在一些情况下，所公开的实施例可以以硬件、固件、软件或其任何组合来实现。所公开的实施例还可以被实现为由一个或多个暂时或非暂时性机器可读(例如，计算机可读)存储介质承载或存储在其上的指令，其可以由一个或多个处理器读取和执行。例如，指令可以通过网络或通过其他计算机可读介质分发。因此，机器可读介质可以 包括用于以机器(例如，计算机)可读的形式存储或传输信息的任何机制，包括但不限于，软盘、光盘、光碟、光盘只读存储器(Compact Disc Read Only Memory，CD-ROMs)、磁光盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(RAM)、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read Only Memory，EEPROM)、磁卡或光卡、闪存、或用于利用因特网以电、光、声或其他形式的传播信号来传输信息(例如，载波、红外信号数字信号等)的有形的机器可读存储器。因此，机器可读介质包括适合于以机器(例如，计算机)可读的形式存储或传输电子指令或信息的任何类型的机器可读介质。
在附图中，可以以特定布置和/或顺序示出一些结构或方法特征。然而，应该理解，可能不需要这样的特定布置和/或排序。而是，在一些实施例中，这些特征可以以不同于说明书附图中所示的方式和/或顺序来布置。另外，在特定图中包括结构或方法特征并不意味着暗示在所有实施例中都需要这样的特征，并且在一些实施例中，可以不包括这些特征或者可以与其他特征组合。
需要说明的是，本申请各设备实施例中提到的各单元/模块都是逻辑单元/模块，在物理上，一个逻辑单元/模块可以是一个物理单元/模块，也可以是一个物理单元/模块的一部分，还可以以多个物理单元/模块的组合实现，这些逻辑单元/模块本身的物理实现方式并不是最重要的，这些逻辑单元/模块所实现的功能的组合才是解决本申请所提出的技术问题的关键。此外，为了突出本申请的创新部分，本申请上述各设备实施例并没有将与解决本申请所提出的技术问题关系不太密切的单元/模块引入，这并不表明上述设备实施例并不存在其它的单元/模块。
需要说明的是，在本专利的示例和说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
虽然通过参照本申请的某些优选实施例，已经对本申请进行了图示和描述，但本领域的普通技术人员应该明白，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (24)

  1. 一种交互系统的显示方法，其特征在于，所述交互系统包括显示屏，所述方法包括：
    确定所述显示屏中划分出的显示区域和扩展区域；
    根据用户针对所述扩展区域的操作生成事件信息；
    根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作。
  2. 根据权利要求1所述的方法，其特征在于，所述显示区域和所述扩展区域为属于同一物理显示屏上的两个不同区域。
  3. 根据权利要求2所述的方法，其特征在于，所述显示区域和所述扩展区域的划分由底层显示系统确定，
    将所述显示屏划分出显示区域和扩展区域，包括：
    确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小；以及
    将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
  4. 根据权利要求2所述的方法，其特征在于，所述显示区域和所述扩展区域的划分由窗口层确定，
    将所述显示屏划分为显示区域和扩展区域，包括：
    确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小；以及
    将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
  5. 根据权利要求1所述的方法，其特征在于，所述显示区域和所述扩展区域为分别属于两块不同的物理显示屏。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作，包括：
    确定显示区域和扩展区域的屏幕尺寸大小；
    基于所述显示区域和所述扩展区域的屏幕大小间的关系将所述事件信息映射至所述显示区域，并在所述显示区域响应与所述事件信息对应的操作。
  7. 根据权利要求6所述的方法，其特征在于，根据所述显示区域和所述扩展区域的屏幕大小间的关系将所述事件信息映射至所述显示区域，包括：
    确定所述显示区域、所述扩展区域在预设坐标系下的位置；
    基于所述预设坐标系，确定所述事件信息的原始报点在所述预设坐标系中的坐标，并确定所述原始报点与所述显示区域和扩展区域的相对位置；
    根据所述坐标、所述显示区域和扩展区域的屏幕大小间关系将所述事件信息的原始报点的坐标映射到所述显示区域对应的位置。
  8. 根据权利要求1-5任一项所述的方法，其特征在于，据所述事件信息，在所述显示区域响应与所述事件信息对应的操作，包括：
    将所述扩展区域的预设操作与所述显示区域的操作建立映射关系，以使用户在扩展区域输入所述预设操作映射为用户在所述显示区域的操作。
  9. 根据权利要求1所述的方法，其特征在于，所述事件信息包括：
    与点击触屏事件、长按触屏事件、滑动触屏事件、用户预定触屏事件和操作录制对应的一种或多种。
  10. 根据权利要求9所述的方法，其特征在于，与所述事件信息对应的操作，包括：
    点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
  11. 根据权利要求1所述的方法，其特征在于，所述显示屏为曲面屏、平面屏或折叠屏，所述扩展区域设置在所述曲面屏或平面屏的边缘区域或部分区域，
    所述扩展区域设置在所述折叠屏的其中一块屏上，或所述折叠屏折叠后的折叠区域。
  12. 一种交互系统，其特征在于，包括：
    显示屏；
    处理模块，用于确定所述显示屏中划分出的显示区域和扩展区域；
    生成模块，用于根据用户针对所述扩展区域的操作生成事件信息；
    所述处理模块用于根据所述事件信息，在所述显示区域响应与所述事件信息对应的操作。
  13. 根据权利要求12所述的系统，其特征在于，所述显示区域和所述扩展区域为属于同一物理显示屏上的两个不同区域。
  14. 根据权利要求13所述的系统，其特征在于，所述显示区域和所述扩展区域的划分由底层显示系统确定，
    将所述显示屏划分为显示区域和扩展区域，所述处理模块还用于：
    确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小；以及
    将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
  15. 根据权利要求13所述的系统，其特征在于，所述显示区域和所述扩展区域的划分由窗口层确定，
    将所述显示屏划分为显示区域和扩展区域，所述处理模块还用于：
    确定所述显示屏的屏幕原始尺寸大小、显示区域的屏幕大小以及扩展区域的屏幕大小；以及
    将所述显示区域的屏幕大小通知作为用于系统显示的屏幕大小，并且将所述显示区域用于所述交互系统的显示。
  16. 根据权利要求12所述的系统，其特征在于，所述显示区域和所述扩展区域为分别属于两块不同的物理显示屏。
  17. 根据权利要求12-16任一项所述的系统，其特征在于，所述处理模块具体用于：
    确定显示区域和扩展区域的屏幕大小；
    基于所述显示区域和所述扩展区域的屏幕大小间的关系将所述事件信息映射至所述显示区域，并在所述显示区域响应与所述事件信息对应的操作。
  18. 根据权利要求17所述的系统，其特征在于，所述处理模块具体还用于：
    确定所述显示区域、所述扩展区域在预设坐标系下的位置；
    基于所述预设坐标系，确定所述事件信息的原始报点在所述预设坐标系中的坐标，并确定所述原始报点与所述显示区域和扩展区域的相对位置；
    根据所述坐标、所述显示区域和扩展区域的屏幕大小间关系将所述事件信息的原始报点的坐标映射到所述显示区域对应的位置。
  19. 根据权利要求12-16任一项所述的系统，其特征在于，所述处理模块具体还 用于：
    将所述扩展区域的预设操作与所述显示区域的操作建立映射关系，以使用户在扩展区域输入所述预设操作映射为用户在所述显示区域的操作。
  20. 根据权利要求12所述的系统，其特征在于，所述事件信息包括：
    点击触屏事件、长按触屏事件、滑动触屏事件、用户预定触屏事件和屏幕录制中的一种或多种。
  21. 根据权利要求20所述的系统，其特征在于，与所述事件信息对应的操作，包括：
    点击触屏事件、长按触屏事件、滑动触屏事件和用户预定触屏事件中的一种或多种。
  22. 根据权利要求12所述的系统，其特征在于，所述显示屏为曲面屏、平面屏或折叠屏，所述扩展区域设置在所述曲面屏或平面屏的边缘区域或部分区域，
    所述扩展区域设置在所述折叠屏的其中一块屏上，或所述折叠屏折叠后的折叠区域。
  23. 一种电子设备，其特征在于：包括处理器和存储器，
    所述存储器中存储有指令，
    所述处理器，用于读取所述存储器中存储的所述指令，以执行权利要求1-11任一项所述的方法。
  24. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器运行时，使得所述处理器执行权利要求1-11任一项所述的方法。
PCT/CN2020/136498 2019-12-18 2020-12-15 交互系统的显示方法、交互系统及电子设备 WO2021121223A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20901033.9A EP4060472A4 (en) 2019-12-18 2020-12-15 INTERACTIVE SYSTEM DISPLAY METHOD, INTERACTIVE SYSTEM AND ELECTRONIC DEVICE
US17/843,596 US20220317964A1 (en) 2019-12-18 2022-06-17 Display method for interactive system, interactive system, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911312368.0 2019-12-18
CN201911312368.0A CN110968252B (zh) 2019-12-18 2019-12-18 交互系统的显示方法、交互系统及电子设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/843,596 Continuation US20220317964A1 (en) 2019-12-18 2022-06-17 Display method for interactive system, interactive system, and electronic device

Publications (1)

Publication Number Publication Date
WO2021121223A1 true WO2021121223A1 (zh) 2021-06-24

Family

ID=70035118

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/136498 WO2021121223A1 (zh) 2019-12-18 2020-12-15 交互系统的显示方法、交互系统及电子设备

Country Status (4)

Country Link
US (1) US20220317964A1 (zh)
EP (1) EP4060472A4 (zh)
CN (1) CN110968252B (zh)
WO (1) WO2021121223A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110968252B (zh) * 2019-12-18 2021-10-22 华为技术有限公司 交互系统的显示方法、交互系统及电子设备
CN111782096B (zh) * 2020-06-29 2022-06-24 联想(北京)有限公司 一种目标物选择方法及装置
CN112235561B (zh) * 2020-10-16 2022-07-22 深圳市时代华影科技股份有限公司 Led显示屏、显示方法及装置、计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104516654A (zh) * 2013-09-26 2015-04-15 联想(北京)有限公司 操作处理方法和装置
CN105094654A (zh) * 2014-05-07 2015-11-25 中兴通讯股份有限公司 一种屏幕控制方法及装置
CN106775411A (zh) * 2016-12-24 2017-05-31 珠海市魅族科技有限公司 显示控制方法及系统
CN108920082A (zh) * 2018-06-28 2018-11-30 Oppo广东移动通信有限公司 操作控制方法、装置、存储介质及电子设备
CN110389802A (zh) * 2019-06-05 2019-10-29 华为技术有限公司 一种柔性屏幕的显示方法及电子设备
CN110968252A (zh) * 2019-12-18 2020-04-07 华为技术有限公司 交互系统的显示方法、交互系统及电子设备

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8479122B2 (en) * 2004-07-30 2013-07-02 Apple Inc. Gestures for touch sensitive input devices
US6545669B1 (en) * 1999-03-26 2003-04-08 Husam Kinawi Object-drag continuity between discontinuous touch-screens
KR100769783B1 (ko) * 2002-03-29 2007-10-24 가부시끼가이샤 도시바 표시 입력 장치 및 표시 입력 시스템
US20090322690A1 (en) * 2008-06-30 2009-12-31 Nokia Corporation Screen display
US8803816B2 (en) * 2008-09-08 2014-08-12 Qualcomm Incorporated Multi-fold mobile device with configurable interface
KR20130054071A (ko) * 2011-11-16 2013-05-24 삼성전자주식회사 다중 어플리케이션을 실행하는 모바일 장치 및 그 방법
CN103309594B (zh) * 2012-03-12 2016-12-14 联想(北京)有限公司 一种触摸屏输入方法及电子设备
US9019170B2 (en) * 2013-03-14 2015-04-28 Lg Electronics Inc. Display device and method for controlling the same
US9354742B2 (en) * 2013-04-10 2016-05-31 Samsung Electronics Co., Ltd Foldable electronic device and method of managing visible regions thereof
CN103593136A (zh) * 2013-10-21 2014-02-19 广东欧珀移动通信有限公司 单手操作大屏幕触控终端的方法、装置及触控终端
US10101772B2 (en) * 2014-09-24 2018-10-16 Dell Products, Lp Protective cover and display position detection for a flexible display screen
US10409542B2 (en) * 2016-01-04 2019-09-10 Rex HUANG Forming a larger display using multiple smaller displays
CN106354458B (zh) * 2016-08-26 2019-09-20 青岛海信移动通信技术股份有限公司 双屏显示方法及装置
CN108319422A (zh) * 2017-01-18 2018-07-24 中兴通讯股份有限公司 一种多屏互动触控显示方法、装置、存储介质和终端
KR102328102B1 (ko) * 2017-03-08 2021-11-17 삼성전자주식회사 전자 장치 및 전자 장치의 화면 표시방법
CN107577414A (zh) * 2017-09-07 2018-01-12 广东欧珀移动通信有限公司 分屏显示的方法和移动终端
EP3756084A1 (en) * 2018-02-23 2020-12-30 Cloudbanter Ltd. Phone operating system with split screen
CN108459790A (zh) * 2018-05-01 2018-08-28 苏州鸥鹄智能科技有限公司 一种电子设备触控交互方法
CN109145570A (zh) * 2018-09-06 2019-01-04 Oppo广东移动通信有限公司 解锁控制方法及相关装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104516654A (zh) * 2013-09-26 2015-04-15 联想(北京)有限公司 操作处理方法和装置
CN105094654A (zh) * 2014-05-07 2015-11-25 中兴通讯股份有限公司 一种屏幕控制方法及装置
CN106775411A (zh) * 2016-12-24 2017-05-31 珠海市魅族科技有限公司 显示控制方法及系统
CN108920082A (zh) * 2018-06-28 2018-11-30 Oppo广东移动通信有限公司 操作控制方法、装置、存储介质及电子设备
CN110389802A (zh) * 2019-06-05 2019-10-29 华为技术有限公司 一种柔性屏幕的显示方法及电子设备
CN110968252A (zh) * 2019-12-18 2020-04-07 华为技术有限公司 交互系统的显示方法、交互系统及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4060472A4

Also Published As

Publication number Publication date
CN110968252B (zh) 2021-10-22
EP4060472A1 (en) 2022-09-21
EP4060472A4 (en) 2023-01-25
US20220317964A1 (en) 2022-10-06
CN110968252A (zh) 2020-04-07

Similar Documents

Publication Publication Date Title
WO2020156269A1 (zh) 一种具有柔性屏幕的电子设备的显示方法及电子设备
WO2020187157A1 (zh) 一种控制方法和电子设备
WO2020177585A1 (zh) 一种手势处理方法及设备
WO2021000881A1 (zh) 一种分屏方法及电子设备
WO2021121223A1 (zh) 交互系统的显示方法、交互系统及电子设备
WO2020108356A1 (zh) 一种应用显示方法及电子设备
WO2021213164A1 (zh) 应用界面交互方法、电子设备和计算机可读存储介质
WO2020253758A1 (zh) 一种用户界面布局方法及电子设备
WO2021063237A1 (zh) 电子设备的控制方法及电子设备
WO2022042285A1 (zh) 一种应用程序界面显示的方法及电子设备
WO2021082564A1 (zh) 一种操作提示的方法和电子设备
WO2021147396A1 (zh) 图标管理方法及智能终端
WO2021057203A1 (zh) 一种操作方法和电子设备
US11995317B2 (en) Method and apparatus for adjusting memory configuration parameter
WO2021249281A1 (zh) 一种用于电子设备的交互方法和电子设备
WO2021051982A1 (zh) 调用硬件接口的方法及电子设备
WO2021057699A1 (zh) 具有柔性屏幕的电子设备的控制方法及电子设备
WO2021190524A1 (zh) 截屏处理的方法、图形用户接口及终端
WO2022135186A1 (zh) 设备控制方法和终端设备
WO2021000943A1 (zh) 一种指纹开关的管理方法及装置
CN111381996A (zh) 内存异常处理方法及装置
WO2021082911A1 (zh) 一种内容传输方法和终端设备
CN111524528B (zh) 防录音检测的语音唤醒方法及装置
CN111880661A (zh) 手势识别方法及装置
WO2023045806A1 (zh) 触控屏中的位置信息计算方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20901033

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020901033

Country of ref document: EP

Effective date: 20220614

NENP Non-entry into the national phase

Ref country code: DE