CN114089902A

CN114089902A - Gesture interaction method and device and terminal equipment

Info

Publication number: CN114089902A
Application number: CN202010748965.4A
Authority: CN
Inventors: 华文
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-07-30
Filing date: 2020-07-30
Publication date: 2022-02-25

Abstract

The embodiment of the application discloses a gesture interaction method, a gesture interaction device and terminal equipment, which are applied to the terminal equipment with a folding screen, wherein the folding screen comprises a first screen and a second screen, at least one application is displayed on the folding screen, and the folding screen is in a folding state to acquire gesture actions of a user; judging whether the gesture action is consistent with a preset command gesture; and when the gesture action is consistent with the instruction gesture, the application is moved to the first screen or the second screen. When the folding screen of the terminal equipment is in a folding state, the application can be quickly switched and the window can be moved between the two screens of the folding screen, and a user does not need to unfold the screens and drag the windows again. Meanwhile, the problem that when a user drags a window between two screens of the folding screen to pass through a crease area between the two screens, due to the fact that the contact area between the fingers and the folding screen is suddenly reduced, the application or the window is separated from the fingers and returns to the original position, and therefore the dragging experience of the user is poor is solved.

Description

Gesture interaction method and device and terminal equipment

[ technical field ] A method for producing a semiconductor device

The present application relates to the field of mobile communications, and in particular, to a gesture interaction method and apparatus, and a terminal device.

[ background of the invention ]

With the more and more various forms and more powerful functions of the terminal, the terminals in various product forms such as a folding screen and a special-shaped screen enter the market. As the user has more and more requirements on the functions of the terminal, the split screen scene of the terminal is more and more common. The user can watch the video and chat at the same time; while watching PPT (PowerPoint, presentation), taking notes; and the live broadcast is seen while the online shopping is carried out. For better multitask experience, users have the appeal of freely moving the position of the split-screen window. In such a scenario, the traditional man-machine interaction mode cannot meet the user requirements.

In the terminal equipment of the folding screen, a screen splitting line is usually positioned at the position of an axis of the folding screen, and in order to protect the axis of the folding screen, a control is prevented from being placed in a folding area where the axis of the folding screen is positioned; secondly, when the folding screen is in a folding state, the touch experience of the folding area is poor. For example, in a scene in which a user needs to drag a split-screen window, when a folding screen is in a folding state, the user drags one window from one screen to another screen and passes through a folding region, at this time, the contact area between the user's finger and the screen is suddenly reduced, which causes the finger to be separated from the screen.

[ summary of the invention ]

In view of this, embodiments of the present application provide a gesture interaction method and apparatus, and a terminal device, so as to solve the technical problem in the prior art that when a user drags a window, the dragging experience is not good when the user drags the window across a folding area of a folding screen.

In a first aspect, an embodiment of the present application provides a gesture interaction method, which is applied to a terminal device having a folding screen, where the folding screen includes a first screen and a second screen, at least one application is displayed on the folding screen, and the folding screen is in a folded state to obtain a gesture action of a user; judging whether the gesture action is consistent with a preset command gesture; moving the application to the first screen or the second screen when the gesture action coincides with the instruction gesture; the folded state is the state of the folded screen when an included angle is formed between the first screen and the second screen.

Through the scheme provided by the embodiment, when the folding screen of the terminal device is in the folding state, a user can input the gesture action matched with the preset instruction gesture, for example, the instruction gesture such as tapping the back of the terminal device or shaking the terminal device moves the position of the application or carries out folding and unfolding operations on the device.

In a preferred embodiment, before determining the gesture action of the user, the method further comprises the steps of: and detecting the using state of the folding screen.

Through the scheme that this embodiment provided, adopt different application position mobile control logic when terminal equipment's folding screen is in fold condition or fold condition, avoid the folding screen still to execute this method when not in fold condition and control the application and lead to the chaotic problem of application control logic.

In a preferred embodiment, before detecting the use state of the folding screen, the method further comprises the steps of: detecting a display function used by the folding screen; and determining the moving mode of the application according to the instruction gesture, the display function and the number of the applications.

According to the scheme provided by the embodiment, the optimized application window moving mode which is more consistent with the current display function can be matched and set according to different display functions of the terminal equipment.

In a preferred embodiment, if the display function is a split-screen display function, the application is moved in the following manner: one or more applications displayed in the first screen are interchanged with one or more applications displayed in the second screen; or one or more applications displayed in the first screen or the second screen are moved to the second screen or the first screen.

Through the scheme provided by the embodiment, in a split-screen use scene, the positions of the application in the first screen and the application in the second screen can realize various modes such as overall interchange, local multiple interchange, single interchange, overall movement, local multiple movement, single movement and the like, so that the degree of freedom of use of a user is enhanced, and the functions of the terminal equipment are more powerful.

In a preferred embodiment, if the display function is a floating window display function, the application moves in the following manner: one or more applications that are hovered displayed on the first screen or the second screen are moved to the second screen or the first screen; or one or more applications simultaneously floating on the first screen and the second screen are moved to the first screen or the second screen.

Through the scheme provided by the embodiment, in the use scene of the floating window, the positions of the applications floating on the first screen and the second screen can be completely moved to one of the screens, the applications can be divided into two application groups to be respectively moved to the two screens, and the like, so that the more complex use requirements of users are met, and the functions of the terminal equipment are more powerful.

In a preferred embodiment, if the display function is a split screen display function and a floating window display function, only the application displayed in a floating manner moves in response to the gesture motion, and the application moves in a manner that: one or more applications that are hovered displayed on the first screen or the second screen are moved to the second screen or the first screen; or one or more applications simultaneously floating on the first screen and the second screen are moved to the first screen or the second screen.

Through the scheme provided by the embodiment, in the use scene with the coexistence of the split-screen display function and the floating window display function, the gesture action of the user only controls the floating window presented on the upper layer, so that the application movement control of the terminal device is ordered.

In a preferred embodiment, the split-screen displayed application moves in response to the gesture motion when the hover displayed application is closed.

Through the scheme that this embodiment provided, in the use scene that split screen display function and suspension window display function coexisted, the user just can control the application that moves the split screen and show if want the control to present the split screen display's of the lower floor in suspension window application, then need close suspension window earlier, the advantage of this kind of design can avoid user's gesture action to appear the control confusion when the application that control suspension window or split screen show, control the not good problem of experience.

In a preferred embodiment, if the display function is a hover ball display function, the application moves in the following manner: one or more applications displayed on the first screen or the second screen in a hover ball form are moved to the second screen or the first screen; or one or more applications simultaneously displayed on the first screen and the second screen in a hover ball form are moved to the first screen or the second screen.

Through the scheme provided by the embodiment, in the use scene of the suspended window, the positions of the applications suspended on the first screen and the second screen can be all moved to one of the screens, so that the users can click conveniently, the more complex use requirements of the users are met, and the functions of the terminal equipment are more powerful.

In a preferred embodiment, if the display function is a hover ball display function and a hover window display function, a hover ball display function and a split screen display function, or a hover ball display function and a hover window display function and a split screen display function coexist, an application displayed only in the hover window mode moves in response to the gesture motion, and the application moves in a manner that: one or more applications of a floating window displayed on the first screen or the second screen are moved to the second screen or the first screen; or one or more applications simultaneously displayed on the first screen and the second screen by the floating window are moved to the first screen or the second screen.

Through the scheme provided by the embodiment, in the use scene in which the floating ball display function coexists with the split screen display function and the floating window display function, the gesture action of the user only controls the floating window presented on the upper layer, so that the application movement control of the terminal device appears orderly.

In a preferred embodiment, when the using state of the folding screen is in an unfolded state, judging the gesture action of a user; an application displayed on the fold screen moves in response to the gesture motion.

Through the scheme provided by the embodiment, when the folding screen of the terminal equipment is in a folding state, the user can move the application window by adopting the gesture interaction method disclosed by the application, and when the folding screen of the terminal equipment is not in the folding state, the user can complete the movement of the application window by adopting the mode disclosed in the prior art.

In a preferred embodiment, the gesture action comprises dragging a window of the application, clicking a split screen line, double clicking a folded screen.

Through the scheme that this embodiment provided, when terminal equipment's folding screen was in fold condition, the user adopted conventional, convenient gesture action to control can.

In a second aspect, an embodiment of the present application provides a gesture interaction apparatus, including: the detection unit is used for detecting the using state of the folding screen and detecting the display function of the folding screen; the judging unit is used for judging whether the gesture action of the user is consistent with a preset command gesture; a processing unit to move the application to the first screen or the second screen when the gesture action coincides with the instruction gesture.

Through the scheme provided by the embodiment, when the folding screen of the terminal device is in the folding state, the user moves the position of the application by inputting the gesture action matched with the preset instruction gesture, for example, the instruction gesture of knocking the back of the terminal device or shaking the terminal device is input.

In a third aspect, an embodiment of the present application provides a terminal device, including: a memory and a processor: the memory for storing a computer program; the processor is configured to execute the computer program stored in the memory to cause the terminal device to perform the method according to the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, comprising a program or instructions, which when run on a computer, performs the method according to the first aspect.

Compared with the prior art, the technical scheme at least has the following beneficial effects:

according to the gesture interaction method and device and the terminal device, when the folding screen of the terminal device is in the folding state, the application can be rapidly switched and the window can be moved between the two screens of the folding screen, and a user does not need to unfold the screens and drag the window. Meanwhile, the problem that when a user drags a window between two screens of the folding screen to pass through a crease area between the two screens, due to the fact that the contact area between the fingers and the folding screen is suddenly reduced, the application or the window is separated from the fingers and returns to the original position, and therefore the dragging experience of the user is poor is solved.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a terminal device provided in embodiment 1 of the present application;

FIG. 2 is a schematic flowchart of a gesture interaction method provided in embodiment 2 of the present application;

fig. 3a to fig. 3f are diagrams illustrating a moving manner of an application when a folding screen uses a split screen display function in a gesture interaction method provided in embodiment 2 of the present application;

fig. 4a to 4c are diagrams illustrating a moving manner of an application when a floating window display function is used in a foldable screen in a gesture interaction method provided in embodiment 2 of the present application;

fig. 5a to 5c are diagrams illustrating a moving manner of an application when a folding screen simultaneously uses a split screen display function and a floating window display function in a gesture interaction method provided in embodiment 2 of the present application;

fig. 6a to 6f are diagrams illustrating a moving manner of an application when a hover ball function is used in a foldable screen in the gesture interaction method provided in embodiment 2 of the present application;

fig. 7 is a schematic structural diagram of a gesture interaction apparatus provided in embodiment 3 of the present application.

Reference numerals:

1-an antenna;

2-an antenna;

100-a terminal device; 110-a processor; 120-external memory interface; 121-internal memory; 130-universal serial bus interface; 140-a charge management module; 141-power management module; 142-a battery; 150-a mobile communication module; 160-a wireless communication module; 170-an audio module; 170A-speaker; 170B-receiver; 170C-microphone; 170D-headset interface; 180-a sensor module; 180A-pressure sensor; 180B-a gyroscope sensor; 180C-air pressure sensor; 180D-magnetic sensor; 180E-acceleration sensor; 180F-distance sensor; 180G — low beam sensor; 180H-fingerprint sensor; 180J-temperature sensor; 180K-touch sensor; 180L-ambient light sensor; 180M-bone conduction sensor; 190-key press; 191-a motor; 192-an indicator; 193-camera; 194-a display screen; 195-a subscriber identity module card interface;

1000-gesture interaction means; 1001-detection unit; 1002-a judging unit; 1003-processing unit.

[ detailed description ] embodiments

For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.

It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Embodiments of a terminal device and a method for implementing the terminal device are described below, where the terminal device may be a mobile phone (also called an intelligent terminal device), a tablet personal computer (tablet personal computer), a personal digital assistant (personal digital assistant), an electronic book reader (e-book reader), or a virtual reality interactive device (virtual reality interactive device), and the terminal device may be accessed into various types of communication systems, for example: long Term Evolution (LTE) systems, future fifth Generation (5th Generation, 5G) systems, new radio access technology (NR), and future communication systems, such as 6G systems; but also Wireless Local Area Networks (WLANs) and the like.

For convenience of description, in the following embodiments, an intelligent terminal device is taken as an example for description.

Example 1

Fig. 1 shows a schematic structural diagram of a terminal device disclosed in embodiment 1 of the present application, where the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identity Module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the terminal device 100. In other embodiments of the present application, terminal device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processor (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. In one embodiment, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In one embodiment, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In one embodiment, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the terminal device 100.

The I2S interface may be used for audio communication. In one embodiment, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In one embodiment, the audio module 170 may transmit audio signals to the wireless communication module 160 through the I2S interface, so as to receive phone calls through a bluetooth headset.

The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment, the audio module 170 and the wireless communication module 160 may be coupled by a PCM bus interface. In one embodiment, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment, a UART interface is generally used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In an embodiment, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.

MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a display screen serial interface (DSI), and the like. In one embodiment, the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the terminal device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In one embodiment, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the terminal device 100, and may also be used to transmit data between the terminal device 100 and a peripheral device. And the earphone can also be used for connecting an earphone and playing audio through the earphone. The interface may also be used to connect other terminal devices, such as AR devices and the like.

It should be understood that the interface connection relationship between the modules according to the embodiment of the present invention is only an exemplary illustration, and does not limit the structure of the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In one wired charging embodiment, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In one wireless charging embodiment, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the terminal device 100. The charging management module 140 may also supply power to the terminal device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In one embodiment, the power management module 141 may also be disposed in the processor 110. In another embodiment, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied on the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In one embodiment, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In one embodiment, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In one embodiment, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the terminal device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

In one embodiment, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150 and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with a network and other devices through a wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou satellite navigation system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

The terminal device 100 implements a display function by the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used for displaying images, videos, and the like, wherein the display screen 194 includes a display panel, the display screen may specifically include a folding screen, a special-shaped screen, and the display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (flex-emitting diode, FLED), a miniature, a Micro-o led, a quantum dot light-emitting diode (QLED), and the like. In one embodiment, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In one embodiment, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In one embodiment, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform fourier transform or the like on the frequency point energy.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record video in a plurality of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. The NPU can implement applications such as intelligent recognition of the terminal device 100, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, a phonebook, etc.) created during use of the terminal device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like. The processor 110 executes various functional applications of the terminal device 100 and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal device 100 may implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In one embodiment, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert the audio electrical signal into an acoustic signal. The terminal device 100 can listen to music through the speaker 170A, or listen to a handsfree call.

The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. When the terminal device 100 answers a call or voice information, it is possible to answer a voice by bringing the receiver 170B close to the human ear.

The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 170C by speaking the user's mouth near the microphone 170C. The terminal device 100 may be provided with at least one microphone 170C. In other embodiments, the terminal device 100 may be provided with two microphones 170C, which may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the terminal device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headphone interface 170D is used to connect a wired headphone. The headset interface 170D may be the USB interface 130, or may be an Open Mobile Terminal Platform (OMTP) standard interface of 3.5mm, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In one embodiment, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure from the change in the capacitance. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The terminal device 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In one embodiment, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the terminal device 100. In one embodiment, the angular velocity of the terminal device 100 about three axes (i.e., x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. Illustratively, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the terminal device 100, calculates the distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the terminal device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The air pressure sensor 180C is used to measure air pressure. In one embodiment, the terminal device 100 calculates an altitude from the barometric pressure measured by the barometric pressure sensor 180C, and assists in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The terminal device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In one embodiment, when the terminal device 100 is a folder, the terminal device 100 may detect the opening and closing of the folder according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

The acceleration sensor 180E can detect the magnitude of acceleration of the terminal device 100 in various directions (generally, three axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. The method can also be used for recognizing the posture of the terminal equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The terminal device 100 may measure the distance by infrared or laser. In one embodiment, the terminal device 100 may take a picture of a scene and may range using the distance sensor 180F to achieve fast focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light to the outside through the light emitting diode. The terminal device 100 detects infrared reflected light from a nearby object using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can utilize the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the ambient light level. The terminal device 100 may adaptively adjust the brightness of the display screen 194 according to the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket, in order to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The terminal device 100 can utilize the collected fingerprint characteristics to realize fingerprint unlocking, access to an application lock, fingerprint photographing, fingerprint incoming call answering and the like.

The temperature sensor 180J is used to detect temperature. In one embodiment, the terminal device 100 executes a temperature processing strategy using the temperature detected by the temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the terminal device 100 heats the battery 142 when the temperature is below another threshold to avoid the terminal device 100 being abnormally shut down due to low temperature. In other embodiments, when the temperature is lower than a further threshold, the terminal device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also called a "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on the surface of the terminal device 100, different from the position of the display screen 194.

In one embodiment, the touch screen composed of the touch sensor 180K and the display screen 194 may be located in a side area or a folding area of the terminal device 100, and is used for determining a position touched by a user and a gesture touched by the user when the user touches the touch screen with a hand; for example, when the user holds the terminal device, the user can click any position on the touch screen with a thumb, the touch sensor 180K can detect the click operation of the user and transmit the click operation to the processor, and the processor determines the click operation according to the click operation to wake up the screen.

The bone conduction sensor 180M may acquire a vibration signal. In one embodiment, the bone conduction sensor 180M may acquire a vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M may also contact the human pulse to receive the blood pressure pulsation signal. In one embodiment, the bone conduction sensor 180M may also be disposed in a headset, integrated into a bone conduction headset. The audio module 170 may analyze a voice signal based on the vibration signal of the bone mass vibrated by the sound part acquired by the bone conduction sensor 180M, so as to implement a voice function. The application processor can analyze heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The terminal device 100 may receive a key input, and generate a key signal input related to user setting and function control of the terminal device 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the terminal device 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. The same SIM card interface 195 can be inserted with multiple cards at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In one embodiment, the terminal device 100 employs eSIM, namely: an embedded SIM card. The eSIM card may be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

When the terminal device adopts the special-shaped screen or the folding screen, the touch display screen of the terminal device may include a plurality of touch display areas, for example, the folding screen of the terminal device includes a folding area in a folded state, and the folding area may also implement touch response. However, in the prior art, the operation of the terminal device on a specific touch display area is limited to a relatively large extent, and no relevant operation is specifically performed on the specific touch display area, and based on this, an embodiment of the present application provides a gesture interaction method, where a touch response area exists in a side area or a folding area of the terminal device in the gesture interaction method, and the terminal device may obtain an input event of the touch response area, and in response to the input event, trigger the terminal device to execute an operation instruction corresponding to the input event, so as to implement a gesture operation on the side area or the folding area of the terminal device, and improve the operation experience of the terminal device.

In the terminal device disclosed in embodiment 1 of the present application, the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the terminal device executes the method described in embodiment 2 of the present application.

Example 2

Fig. 2 is a schematic diagram illustrating steps of a gesture interaction method provided in embodiment 2 of the present application.

The gesture interaction method is applied to terminal equipment with a folding screen, wherein the folding screen comprises a first screen and a second screen, and at least one application is displayed on the folding screen.

The gesture interaction method specifically comprises the following steps:

step 101: detecting a display function used by the folding screen;

step 102: determining the moving mode of the application according to the command gesture, the display function and the number of the applications;

step 103: detecting the use state of the folding screen;

step 104: acquiring gesture actions of a user;

step 105: judging whether the gesture action is consistent with a preset command gesture;

step 106: and when the gesture action is consistent with the instruction gesture, the application is moved to the first screen or the second screen.

The folding state refers to a state of folding the screen when an included angle is formed between the first screen and the second screen, and the included angle is larger than 0 degree and smaller than 180 degrees.

Specifically, in one implementation, if the display function is a split-screen display function, the application moving method is as follows: one or more applications displayed in the first screen are interchanged with one or more applications displayed in the second screen; or one or more applications displayed in the first screen or the second screen are moved to the second screen or the first screen.

As shown in fig. 3a to 3f, when the user uses the split-screen display function in the folded state of the folding screen and the system determines that the user uses the preset command gesture that is satisfied, the system interchanges the position of the application a displayed in the first screen and the application B displayed in the second screen (see fig. 3a and 3B). Sometimes, the applications displayed in the split screen may be three, four or more (see fig. 3c), and when the preset command gesture is determined, the system interchanges the positions of all the applications displayed in the first screen and all the applications displayed in the second screen (see fig. 3 d). If there is a split application window displayed in both the first screen and the second screen, the split application window is not moved (see application C in fig. 3 d). If the split application windows are both displayed in the first screen and the second screen (see FIG. 3 e), application A is moved to the first screen display and application B is moved to the second screen display. (see FIG. 3f for an illustration.)

When the user uses the split-screen display function in the unfolded state of the folding screen, the system judges that the user uses the preset command gesture in the meeting, and the system does not make any reaction. The user still needs to use the method in the prior art to perform the window moving operation.

The preset command gesture comprises: knocking or shaking of the device, folding or unfolding of the device, etc. Taking the tapping of the device as an example, when the user taps the back of the first screen or the second screen of the device in the folded state of the folding screen, the application of the first screen and the application of the second screen complete position switching.

In a split-screen use scene, the positions of the application in the first screen and the application in the second screen can realize various modes such as overall interchange, local multiple interchange, single interchange, overall movement, local multiple movement, single movement and the like, so that the degree of freedom of use of a user is enhanced, and the functions of the terminal equipment are more powerful.

In one implementation, if the display function is a floating window display function, the application moving method is as follows: moving one or more applications displayed on the first screen or the second screen in a floating manner to the second screen or the first screen; or one or more applications simultaneously floating and displayed on the first screen and the second screen are moved to the first screen or the second screen.

As shown in fig. 4a to 4c, when the user uses the floating window in the folded state of the folding screen, the system determines that the user uses the preset command gesture, determines the moving position of the preset command gesture, and then moves the position of the floating window. Taking the tap gesture as an example, if the user taps the back of the first screen while application a is displayed within the first screen, application a is moved to another screen, i.e., the second screen (fig. 4b), otherwise no movement is made.

If the floating window is displayed on both the first screen and the second screen (fig. 4a), tapping the back of the second screen will move the window into the first screen and only display it in the first screen (fig. 4 c). If the back of the first screen is tapped, the window will move to the second screen and be displayed only within the second screen (fig. 4 b).

Sometimes, the number of the floating windows exceeds one, and if the preset instruction gesture is judged, the moving position of the preset instruction gesture is judged, and all the floating windows in a certain screen are moved.

If the user uses the combination of the split screen window and the floating window, the preset command gesture only operates the position of the floating window, if the position of the split screen window needs to be controlled, all the floating windows need to be closed, and then the preset command gesture is triggered.

In a floating window use scene, the positions of the applications floating on the first screen and the second screen can be completely moved to one of the screens, the applications can be divided into two application groups and respectively moved to the two screens, and the like, so that the more complex use requirements of users are met, and the functions of the terminal equipment are more powerful.

In one implementation, if the display function is a split screen display function and a floating window display function, only the application displayed in a floating manner moves in response to a gesture motion, and the application moves in the following manner: moving one or more applications displayed on the first screen or the second screen in a floating manner to the second screen or the first screen; or one or more applications simultaneously floating on the first screen and the second screen are moved to the first screen or the second screen (see fig. 5a, 5b, and 5 c).

In a use scene with the coexistence of the split-screen display function and the floating window display function, the gesture action of the user only controls the floating window presented on the upper layer, so that the application movement control of the terminal device is ordered.

In one implementation, when the hover displayed application is closed, the split-screen displayed application moves in response to the gesture motion.

In the use scene that split screen display function and suspension window display function coexisted, the user just can control the application that removes the split screen display if want the control to present the split screen display's of the lower floor in the suspension window application, then need close the suspension window earlier, the advantage of this kind of design can avoid user's gesture action to appear controlling the confusion, control the not good problem of experience when the application of suspension window or split screen display is controlled.

In one implementation, if the display function is an application hover ball and split screen display function or a hover window display function, only the split screen display application or the hover window display application moves in response to the gesture motion.

In a use scene in which the application floating ball and the split-screen display function or the floating window display function coexist, the gesture action of the user only controls the floating window or the split screen which is presented on the upper layer and has more instant use significance, so that the application control of the terminal device appears to be ordered and meaningful (see fig. 6e and 6 f).

In one implementation, if the application displayed in the split screen and the application displayed in the floating window are closed, the application floating ball moves in response to the gesture motion.

In the application scene where the floating ball function and the split-screen display function or the floating window display function coexist, if a user wants to control an application floating ball without instant use significance at the moment, the user needs to close the application displayed in the split-screen mode or close the floating window to control the mobile application floating ball, and the design has the advantage that the problems of control confusion and poor control experience when the user performs gesture actions to control the floating window or the split-screen display application can be solved (see fig. 6a and 6 b).

Similarly, the system functions floating on the application interface like applying the hover ball function also include hovering the navigation ball, side bars, and bubble notification, etc. (see fig. 6c and 6 d).

When the folding screen is in a folding state, the moving direction of a preset instruction gesture can be judged, and the split screen window or the floating window is moved and displayed on a specified screen.

By adopting the gesture interaction method in the embodiment of the application, when the folding screen of the terminal device is in the folding state, a user can input the gesture action matched with the preset instruction gesture, for example, the instruction gesture such as knocking the back of the terminal device or shaking the terminal device moves the application position.

Step103 is adopted before Step104, that is, before the gesture action of the user is obtained, the use state of the folding screen is detected, different application position movement control logics can be adopted when the folding screen of the terminal device is in a folding state or an unfolded state, and the problem that the application control logics are disordered when the folding screen is not in the folding state due to the fact that the method is still executed to control the application is solved.

Step101 and Step102 are adopted before Step103, namely, before the use state of the folding screen is detected, the display function used by the folding screen is detected; and determining the moving mode of the application according to the command gesture, the display function and the number of the applications. According to different display functions of the terminal equipment, a more optimized application window moving mode which is more consistent with the current display function can be set in a matching manner.

In one implementation, when the use state of the folding screen is in the unfolded state, Step104 and Step105 are also executed, that is, the gesture action of the user is acquired and judged, and the application displayed on the folding screen moves in response to the gesture action.

When the terminal device with the folding screen is in the unfolded state, the user can complete the movement of the application window using the manner disclosed in the prior art. When the window is folded, the user uses a preset command gesture to complete the movement of the window.

When the folding screen of the terminal device is in the folding state, the user can use the gesture interaction method disclosed by the application to move the application window, and when the folding screen of the terminal device is not in the folding state, the user can use the mode disclosed in the prior art to complete the movement of the application window.

In one implementation, the gesture actions include dragging a window of the application, clicking on a split screen line, double clicking on a collapsed screen.

When the folding screen of the terminal equipment is in a folding state, a user can control the folding screen by adopting conventional and convenient gesture actions.

Example 3

Fig. 6 is a schematic structural diagram of a gesture interaction apparatus 1000 provided in embodiment 3 of the present application, where the gesture interaction apparatus 1000 may be used to execute the gesture interaction method shown in fig. 2. The gesture interaction device 1000 comprises: a detection unit 1001 for detecting a use state of the folding screen and detecting a display function of the folding screen; the judging unit 1002 is configured to judge whether a gesture action of a user is consistent with a preset instruction gesture; and the processing unit 1003 is configured to move the application to the first screen or the second screen when the gesture motion matches the instruction gesture.

By using the gesture interaction device 1000 provided in embodiment 3 of the present application, when the folding screen of the terminal device is in the folded state, the user can input a gesture motion corresponding to a preset instruction gesture, for example, an instruction gesture such as tapping the back of the terminal device or shaking the terminal device moves the application position.

In one implementation, the gesture operation includes one or more of double-clicking the screen, shaking the device, tapping the back of the screen, and the like.

The apparatus in the above embodiments may be a terminal device, or may be a chip applied in the terminal device, or other combined devices and components having the above terminal function.

Example 4

Embodiment 4 of the present application provides a computer-readable storage medium, which includes a program or instructions, when the program or instructions are run on a computer, the method according to the first aspect is performed.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Video Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed.

Claims

1. A gesture interaction method is applied to a terminal device with a folding screen, wherein the folding screen comprises a first screen and a second screen, at least one application is displayed on the folding screen,

the folding screen is in a folding state, and gesture actions of a user are acquired;

judging whether the gesture action is consistent with a preset command gesture;

moving the application to the first screen or the second screen when the gesture action coincides with the instruction gesture;

the folded state is the state of the folded screen when an included angle is formed between the first screen and the second screen.

2. The gesture interaction method according to claim 1, characterized in that before determining the gesture action of the user, the method further comprises the steps of:

and detecting the using state of the folding screen.

3. The gesture interaction method according to claim 2, characterized in that before detecting the use state of the folding screen, the method further comprises the steps of:

detecting a display function used by the folding screen;

and determining the moving mode of the application according to the instruction gesture, the display function and the number of the applications.

4. The gesture interaction method according to claim 3,

if the display function is a split screen display function, the application moving mode is as follows:

one or more applications displayed in the first screen are interchanged with one or more applications displayed in the second screen; or

One or more applications displayed in the first screen or the second screen are moved to the second screen or the first screen.

5. The gesture interaction method according to claim 3,

if the display function is a floating window display function, the application moving mode is as follows:

one or more applications that are hovered displayed on the first screen or the second screen are moved to the second screen or the first screen; or

One or more applications simultaneously floating on the first screen and the second screen are moved to the first screen or the second screen.

6. The gesture interaction method according to claim 3,

if the display function is a split screen display function and a floating window display function, only the application displayed in a floating mode moves in response to the gesture action, and the application moves in a mode that:

7. The gesture interaction method according to claim 6, wherein when the hover displayed application is closed, the split-screen displayed application moves in response to the gesture action.

8. The gesture interaction method according to claim 3,

if the display function is a floating ball display function, the application moving mode is as follows:

one or more applications displayed on the first screen or the second screen in a hover ball form are moved to the second screen or the first screen; or

One or more applications simultaneously displayed on the first screen and the second screen in the form of a hover ball are moved to the first screen or the second screen.

9. The gesture interaction method according to claim 3,

if the display function is a floating ball display function, a floating window display function, a floating ball display function and a split screen display function or the floating ball display function, the floating window display function and the split screen display function coexist, only the application displayed in the form of the floating window responds to the gesture action to move, and the application moving mode is as follows:

one or more applications of a floating window displayed on the first screen or the second screen are moved to the second screen or the first screen; or one or more applications simultaneously displayed on the first screen and the second screen by the floating window are moved to the first screen or the second screen.

10. The gesture interaction method according to claim 2, wherein when the using state of the folding screen is in an unfolded state, the gesture action of the user is judged;

an application displayed on the fold screen moves in response to the gesture motion.

11. The gesture interaction method according to claim 10, wherein the gesture action includes dragging a window of the application, clicking a split screen line, double clicking a folded screen.

12. A gesture interaction apparatus, the apparatus comprising:

the detection unit is used for detecting the using state of the folding screen and detecting the display function of the folding screen;

the judging unit is used for judging whether the gesture action of the user is consistent with a preset command gesture;

a processing unit to move the application to the first screen or the second screen when the gesture action coincides with the instruction gesture.

13. A terminal device, comprising: a memory and a processor:

the memory for storing a computer program;

the processor configured to execute the computer program stored in the memory to cause the terminal device to perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium comprising a program or instructions for performing the method of any one of claims 1 to 11 when the program or instructions are run on a computer.