CN115665317A

CN115665317A - Application control method and device of virtual screen and computer readable storage medium

Info

Publication number: CN115665317A
Application number: CN202211316220.6A
Authority: CN
Inventors: 罗晓明
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2023-01-31

Abstract

The invention discloses an application control method, application control equipment and a computer readable storage medium of a virtual screen, wherein the method comprises the following steps: taking over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through an input device connected with the outside and operation events transmitted through wireless connection with other terminals; and converting the operation event collected by the event acquisition module into the touch event supported by the selected application through the event conversion module according to the virtual screen parameter of the virtual screen, and modifying the touch parameter and the mapping relation of the touch event according to the operation requirement of the operation event. The humanized application control scheme of the virtual screen is realized, the application operation experience of the virtual screen is synchronously promoted for multi-party users such as an operator, a sharing party and a watching party, and the application function of the virtual screen and the expansion experience of the virtual application are greatly enriched.

Description

Application control method and device of virtual screen and computer readable storage medium

Technical Field

The present invention relates to the field of mobile communications, and in particular, to a method and an apparatus for controlling an application of a virtual screen, and a computer-readable storage medium.

Background

In the prior art, smart terminals such as smart phones are often connected with an external display device in a wired or wireless screen projection manner, so as to share content in devices such as mobile phones or provide better display experience than the screen of the mobile phones. For example, a common screen projection manner of a mobile phone is mirror projection, that is, the display content of an external display is completely consistent with the display content of a display screen of the mobile phone. The method is convenient for application operation, a user only needs to operate the application on the touch screen of the mobile phone, and the change of the display content can be synchronously reflected on the external display.

However, the above method has the following disadvantages:

first, even if the resolution of the external display is much higher than the mobile phone screen, the final display also has the effect of amplifying the content of the mobile phone screen after mirroring, and the real improvement of the image quality cannot be brought.

Secondly, the aspect ratio of the external display is generally different from that of the screen of the mobile phone, and at the moment, black edges are left above and below or on the left and right of the external display, so that the full screen cannot be fully displayed, and the visual experience is influenced.

Thirdly, the application operation experience is consistent with that of the mobile phone, the user operates the application on the mobile phone, and for the application needing frequent operation, for example, for the game application, the application can only be shared with the spectator, but the experience of the user cannot be improved.

Therefore, how to synchronously enhance the application operation experience of the virtual screen for multi-party users such as an operating party, a sharing party, a viewing party, and the like becomes a technical problem to be solved urgently at present.

Disclosure of Invention

In order to solve the technical defects in the prior art, the invention provides an application control method of a virtual screen, which is applied to an intelligent terminal comprising an operation processing central control, wherein the operation processing central control comprises a virtual screen management module, an event acquisition module, an event conversion module and an event transmission module, and the method comprises the following steps:

creating a virtual screen through the virtual screen management module, and running a selected application into the virtual screen;

taking over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through input equipment connected with the outside and operation events transmitted through wireless connection with other terminals through the event acquisition module;

converting the operation event collected by the event acquisition module into a touch event supported by the selected application through the event conversion module according to the virtual screen parameter of the virtual screen, and modifying the touch parameter and the mapping relation of the touch event according to the operation requirement of the operation event;

dispatching, by the event delivery module, the modified touch event to a processing process of the selected application in the virtual screen.

Optionally, the intelligent terminal further includes a setting module and an operation interface module, and the method further includes:

receiving a selection instruction transmitted to the operation or equipment of the selected application by a user through the setting module, and setting a mapping relation between the operation or the equipment and the processable event of the selected application;

and displaying an operation interface through the operation interface module, and receiving an operation instruction of the touch screen sent by a user in the operation interface.

Optionally, the method further comprises:

acquiring the event type and/or the event attribute of the processable event through the setting module;

establishing, by the setting module, the mapping relationship between one or more of a first operation instruction of the touch screen and/or the sensor, a second operation instruction of the input device, and a third operation instruction of the other terminal and the processable event according to the event type and/or the event attribute.

Optionally, the method further comprises:

receiving size setting parameters and density setting parameters of a user for the virtual screen through the setting module, or determining the size setting parameters and the density setting parameters according to equipment parameters of external display equipment connected with the intelligent terminal;

and transmitting the size setting parameters and the density setting parameters to the virtual screen management module through the setting module.

Optionally, the method further comprises:

receiving a selection instruction of a user for the selected application through the setting module and/or receiving a switching instruction of the user for the selected application in the operation interface through the operation interface module;

and loading and displaying the resources according to the size setting parameters, the density setting parameters, the selection instructions and/or the switching instructions through the virtual screen management module.

Optionally, the method further comprises:

after the virtual screen is created, if the operation interface is called to be displayed, acquiring the operation event and converting the touch event through the operation processing center control;

after the virtual screen is created, if the operation interface is closed or hidden, the operation processing central control unit stops receiving the operation event corresponding to the selected application.

Optionally, the method further comprises:

when the virtual screen is displayed on the screen of the intelligent terminal, displaying the display content of the virtual screen through a first mapping area controlled in the operation interface in the operation processing;

and amplifying the touch event of the first mapping area in an equal proportion through the operation processing center control, and transmitting the amplified touch event to the selected application of the virtual screen.

Optionally, the method further comprises:

when the virtual screen is displayed on an external screen of the intelligent terminal, displaying the display content of the virtual screen and/or displaying a touch panel in the operation interface through a second mapping area controlled in the operation interface in the operation processing;

and amplifying the touch event of the second mapping area according to an equal proportion and transmitting the amplified touch event to the selected application of the virtual screen through the operation processing center control, and/or converting the touch event of the touch panel according to panel attributes and transmitting the converted touch event to the selected application of the virtual screen.

The invention also provides an application control device of a virtual screen, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of the application control method of the virtual screen when being executed by the processor.

The present invention further provides a computer-readable storage medium, on which an application manipulation program of a virtual screen is stored, and when being executed by a processor, the application manipulation program of the virtual screen implements the steps of the application manipulation method of the virtual screen according to any one of the above items.

By implementing the application control method, the device and the computer readable storage medium of the virtual screen, the virtual screen is created through the virtual screen management module, and the selected application is operated to the virtual screen; the event acquisition module takes over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through input equipment connected with the outside and operation events transmitted through wireless connection with other terminals; converting the operation event collected by the event acquisition module into a touch event supported by the selected application through the event conversion module according to the virtual screen parameter of the virtual screen, and modifying the touch parameter and the mapping relation of the touch event according to the operation requirement of the operation event; dispatching, by the event delivery module, the modified touch event to a processing process of the selected application in the virtual screen. The humanized application control scheme of the virtual screen is realized, the application operation experience of the virtual screen is synchronously promoted for multi-party users such as an operator, a sharing party and a watching party, and the application function of the virtual screen and the expansion experience of the virtual application are greatly enriched.

Drawings

The invention will be further described with reference to the accompanying drawings and examples, in which:

fig. 1 is a schematic diagram of a hardware structure of a mobile terminal according to the present invention;

FIG. 2 is a first flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 3 is a second flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 4 is a third flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 5 is a fourth flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 6 is a fifth flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 7 is a sixth flowchart of an application manipulation method of the virtual screen of the present invention;

FIG. 8 is a seventh flowchart of an application manipulation method of the virtual screen of the present invention;

fig. 9 is an eighth flowchart of an application manipulation method of a virtual screen of the present invention;

FIG. 10 is a schematic view of a virtual module of the application manipulation method of the virtual screen according to the present invention;

FIG. 11 is a diagram illustrating a first event transition of the application manipulation method of the virtual screen according to the present invention;

fig. 12 is a second event transition diagram of the application manipulation method of the virtual screen according to the present invention;

FIG. 13 is a schematic non-external display diagram illustrating an application manipulation method for a virtual screen according to the present invention;

fig. 14 is an external display diagram of the application manipulation method of the virtual screen according to the present invention.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in itself. Thus, "module", "component" or "unit" may be used mixedly.

The terminal may be implemented in various forms. For example, the terminal described in the present invention may include mobile terminals such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and fixed terminals such as a Digital TV, a desktop computer, and the like.

The following description will be given by way of example of a mobile terminal, and it will be understood by those skilled in the art that the construction according to the embodiment of the present invention can be applied to a fixed type terminal, in addition to elements particularly used for mobile purposes.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal for implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (Radio Frequency) unit 101, wiFi module 102, audio output unit 103, a/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111. Those skilled in the art will appreciate that the mobile terminal architecture shown in fig. 1 is not intended to be limiting of mobile terminals, which may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile terminal in detail with reference to fig. 1:

the radio frequency unit 101 may be configured to receive and transmit signals during information transmission and reception or during a call, and specifically, receive downlink information of a base station and then process the downlink information to the processor 110; in addition, uplink data is transmitted to the base station. Typically, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can also communicate with a network and other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000 ), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division duplex Long Term Evolution), and TDD-LTE (Time Division duplex Long Term Evolution).

WiFi belongs to short-distance wireless transmission technology, and the mobile terminal can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 102, and provides wireless broadband internet access for the user. Although fig. 1 shows the WiFi module 102, it is understood that it does not belong to the essential constitution of the mobile terminal, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is used to receive audio or video signals. The a/V input Unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042, the Graphics processor 1041 Processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphic processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 may receive sounds (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, or the like, and may be capable of processing such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 101 in case of the phone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the gesture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.

The display unit 106 is used to display information input by a user or information provided to the user. The Display unit 106 may include a Display panel 1061, and the Display panel 1061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect a touch operation performed by a user on or near the touch panel 1071 (e.g., an operation performed by the user on or near the touch panel 1071 using a finger, a stylus, or any other suitable object or accessory), and drive a corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel 1071 may be implemented in various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like, and are not limited to these specific examples.

Further, the touch panel 1071 may cover the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although the touch panel 1071 and the display panel 1061 are shown in fig. 1 as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, and is not limited herein.

The interface unit 108 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and external devices.

The memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 109 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, performs various functions of the mobile terminal and processes data by operating or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby integrally monitoring the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The mobile terminal 100 may further include a power supply 111 (e.g., a battery) for supplying power to various components, and preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described in detail herein.

Based on the hardware structure of the mobile terminal, the embodiments of the method of the invention are provided.

Fig. 2 is a first flowchart of an application manipulation method of the virtual screen according to the present invention. The embodiment provides an application control method of a virtual screen, which is applied to an intelligent terminal including an operation processing center control, wherein the operation processing center control includes a virtual screen management module, an event acquisition module, an event conversion module and an event transmission module, and the method includes:

s1, creating a virtual screen through a virtual screen management module, and running a selected application into the virtual screen;

s2, taking over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through input equipment connected with the outside and operation events transmitted through wireless connection with other terminals through the event acquisition module;

s3, converting the operation event collected by the event acquisition module into a touch event supported by the selected application through the event conversion module according to the virtual screen parameter of the virtual screen, and modifying the touch parameter and the mapping relation of the touch event according to the operation requirement of the operation event;

and S4, dispatching the modified touch event to the processing process of the selected application in the virtual screen through the event delivery module.

Optionally, in this embodiment, taking a mobile phone as an example, screen projection and an operation method thereof are optimized, specifically: firstly, a virtual screen is created in a mobile phone, and an application runs in the virtual screen; then, an operation processing center control is designed corresponding to the virtual screen and is used for collecting user operation behaviors of various sources; finally, the behaviors are converted into events which can be processed by the application and are transmitted to the application, so that the application is operated. When the screen needs to be projected, the whole virtual screen is mirrored to the external equipment for displaying.

Optionally, in this embodiment, please refer to the schematic diagram of the virtual module shown in fig. 10. Wherein, this intelligent terminal includes following software module: the virtual screen management module is used for creating a management virtual screen and running the selected application into the virtual screen; the event acquisition module is used for taking over operation events transmitted by a touch screen and a sensor of the mobile phone, an external connection input device and other smart phones through wireless connection; the event conversion module is used for converting the events collected by the event acquisition module into events supported by the application program, wherein the events are generally touch screen events, and parameters and mapping relations are modified as required, and the conversion module is used for controlling the conversion of coordinates according to the size of the virtual screen because the conversion of the touch screen events is related to the size of the screen; and the event transmission module is used for sending the event to the application process in the virtual screen.

Optionally, in this embodiment, the selected application is run in a virtual screen on the mobile phone, and the presence of the virtual screen is unrelated to the connection of the external display device, so that when the user disconnects the external screen, the application in the virtual screen still continues to run, and the application can continue to be displayed when the external screen is connected next time.

Optionally, in this embodiment, the parameter of the virtual screen may be set in a self-defined manner according to a user requirement, for example, the displayed resolution may be consistent with the external device, based on which, even if the mobile phone screen itself is not a screen with a resolution of 4K, if the resolution of the virtual screen is set to be consistent with the resolution of 4K, the application on the mobile phone may also run with the resolution of 4K, and at this time, when the virtual screen is displayed on the external device, the real image quality of 4K may be achieved, and the full screen is fully covered.

Optionally, in this embodiment, an operation behavior of a mobile phone screen, a mobile phone sensor, an external device connected to the mobile phone, or even another mobile phone may be collected, and even if the application itself does not support the operation behavior, the operation behavior may be converted into a general event that can be supported by the selected application in the operation process of this embodiment, so as to operate the application, thereby bringing about a rich experience effect.

The method has the advantages that the virtual screen is created through the virtual screen management module, and the selected application is operated to the virtual screen; the event acquisition module takes over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through input equipment connected with the outside and operation events transmitted through wireless connection with other terminals; converting the operation event collected by the event acquisition module into a touch event supported by the selected application through the event conversion module according to the virtual screen parameter of the virtual screen, and modifying the touch parameter and the mapping relation of the touch event according to the operation requirement of the operation event; dispatching, by the event delivery module, the modified touch event to a processing process of the selected application in the virtual screen. The humanized application control scheme of the virtual screen is realized, the application operation experience of the virtual screen is synchronously promoted for multi-party users such as an operator, a sharing party and a watching party, and the application function of the virtual screen and the expansion experience of the virtual application are greatly enriched.

Fig. 3 is a second flowchart of the application control method for a virtual screen according to the present invention, and based on the foregoing embodiment, optionally, the intelligent terminal further includes a setting module and an operation interface module, and the method further includes:

s01, receiving a selection instruction which is transmitted to the operation or equipment of the selected application and is sent by a user through the setting module, and setting a mapping relation between the operation or the equipment and the processable event of the selected application;

and S02, displaying an operation interface through the operation interface module, and receiving an operation instruction of the touch screen sent by a user in the operation interface.

Optionally, in this embodiment, as shown in fig. 10, a setting module is configured to select an operation or a device that needs to be delivered to the application by a user, and a mapping relationship between the operation and the application may process an event; and the operation interface module is used for interface display, and a user can operate the touch screen in the operation interface module.

Fig. 4 is a third flowchart of an application manipulation method of a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s03, acquiring the event type and/or the event attribute of the processable event through the setting module;

and S04, establishing the mapping relation between one or more of a first operation instruction of the touch screen and/or the sensor, a second operation instruction of the input equipment and a third operation instruction of the other terminal and the processable event according to the event type and/or the event attribute through the setting module.

Optionally, in this embodiment, the setting of the operation event source and the event mapping relationship includes that the user may selectively obtain the operation event from the sensor, the peripheral device, the mobile phone touch screen, another mobile phone, and the like in the graph to control the virtual on-screen application; the external case is taken as an example, normally the peripheral only acts on applications running on the screen of the handset itself, where peripheral event transitions can be selected. Therefore, when the virtual screen is in a working state, the peripheral events are intercepted by the operation processing control module and are transmitted to the operation processing control module to control the virtual screen application.

Optionally, in this embodiment, for the operation event source and the event mapping relationship, the user may further redefine the event of each source, map the event as a new event, and control an application of the virtual screen with the new event; for example, a click of a key in the keyboard may be relocated to a touch screen click event at a specified screen location; for example, the acceleration sensor may detect an upward movement of the mobile phone, and the movement may be redefined as a click event of a touch screen button for controlling the character jump in a certain game application, so that the character jump may be controlled by lifting the mobile phone in the game.

Optionally, in this embodiment, please refer to a first event conversion schematic diagram shown in fig. 11, wherein the event acquisition module collects events from different sources, and the events are converted and combined to collectively control the virtual screen application. Specifically, since the mobile phone application does not support many events, in order to process events from different sources, the events need to be converted into common touch screen events and then transmitted to the application. Taking a touch screen event and a gyro sensor to control a shooting game as an example, as shown in fig. 11, for the shooting game of a mobile phone, generally, a touch area representing a visual field in a sliding screen is used to move the visual field, and in this embodiment, a character moving direction and a shooting action may be controlled through the touch screen in an operation interface; and detecting the rotation direction of the mobile phone by using a gyroscope of the mobile phone. Based on the method, when the mobile phone slightly rotates along the Y axis in a counterclockwise way, the mobile phone is converted into left sliding of a visual field area in the virtual screen application in the event conversion module, and the visual field is controlled to move left; when the mobile phone slightly rotates clockwise along the Y axis, the rotation is converted into right sliding of the visual field area, and the visual field is controlled to move rightwards; when the mobile phone slightly rotates along the X axis anticlockwise, the mobile phone is converted into an upward slide of the visual field area, and the visual field is controlled to move upwards; when the mobile phone slightly rotates clockwise along the X axis, the mobile phone is converted into the downward sliding of the visual field area, and the visual field is controlled to move downward. It can be seen that, in the embodiment, in the event conversion module, the touch screen event from the operation interface is modified in coordinates, and is converted into the view area sliding event from the gyroscope, so as to synthesize the multi-finger touch screen event, and then the multi-finger touch screen event is transferred to the game application to control the game.

Optionally, in this embodiment, please refer to a second event transition diagram shown in fig. 12, wherein, for the control, sharing and watching requirements of the multi-party user, two mobile phones may be connected through WiFi, and an application is controlled in turn or together, as shown in fig. 12, a virtual screen is created by the mobile phone a, a game application runs in the virtual screen, the mobile phone a is connected to the external screen, and the content of the virtual screen is projected to the external screen. The mobile phone A controls the application A through the operation interface, the mobile phone B is connected with the mobile phone A through WiFi, the operation interface layout is shared by the mobile phone A and the user of the mobile phone B can use the same operation interface, a generated touch screen event is transmitted to the event acquisition module of the mobile phone A through WiFi, and is transmitted to a game through the event conversion module to control the game to run. It can be seen that, in this embodiment, when the event conversion module processes only one event of the mobile phone at the same time, the game is controlled by turns for a and B; when the event conversion module combines the events of the two mobile phones together, the game is controlled by the A and the B together, for example, the A controls the movement of characters, and the B controls the visual field and shooting. Therefore, the users A and B operate own interfaces and watch game pictures of the external screen together, and the effects of coaching other people to play games or playing games together can be achieved. It should be noted that the above experience effect is not achieved by the original single-hand game. Similarly, the operation processing center control of the embodiment may convert and synthesize events of a plurality of event sources and different event source combinations, thereby further realizing richer user operation, sharing and viewing experiences.

Fig. 5 is a fourth flowchart of the application control method for a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s11, receiving size setting parameters and density setting parameters of a user for the virtual screen through the setting module, or determining the size setting parameters and the density setting parameters according to equipment parameters of external display equipment connected with the intelligent terminal;

and S12, transmitting the size setting parameters and the density setting parameters to the virtual screen management module through the setting module.

Optionally, in this embodiment, manual settings of the user may be received in the setting interface, including various parameters of the virtual screen, such as length, width, screen density, and the like; an automatic detection method can also be selected, for example, when the mobile phone is connected with an external display, the parameters of the external display are detected, and a virtual screen consistent with the parameters is created.

Optionally, in this embodiment, the parameters of the virtual screen are synchronized to the event conversion module.

Fig. 6 is a fifth flowchart of an application manipulation method of a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s21, receiving a selection instruction of a user for the selected application through the setting module and/or receiving a switching instruction of the user for the selected application in the operation interface through the operation interface module;

and S22, loading resources and displaying the resources in a layout mode through the virtual screen management module according to the size setting parameters, the density setting parameters, the selection instructions and/or the switching instructions.

Optionally, in this embodiment, an application that needs to be started is selected in the setting module, or the application is switched in the operation interface.

Optionally, in this embodiment, the specified application loads the resources and lays out the display according to the size and parameters of the virtual screen.

Fig. 7 is a sixth flowchart of an application manipulation method of a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s31, after the virtual screen is created, if the operation interface is called to be displayed, acquiring the operation event and converting the touch event through the operation processing center control;

and S32, after the virtual screen is created, if the operation interface is closed or hidden, stopping receiving the operation event corresponding to the selected application by the operation processing central control.

Optionally, in this embodiment, after the virtual screen is created, the operation interface may be invoked to display on the mobile phone at any time; when the operation interface is displayed, the operation processing center control carries out event acquisition and conversion; when the operation interface is closed, the mobile phone processes the event according to the original flow of the mobile phone, and the virtual screen application does not receive the event any more.

Fig. 8 is a seventh flowchart of an application manipulation method of a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s41, when the virtual screen is displayed on the screen of the intelligent terminal, displaying the display content of the virtual screen through a first mapping area controlled in the operation interface in the operation processing;

and S42, amplifying the touch event of the first mapping area in an equal proportion through the operation processing control, and transmitting the amplified touch event to the selected application of the virtual screen.

Optionally, in this embodiment, please refer to a non-external display schematic diagram shown in fig. 13. When no external display screen is connected, the operation interface can map the display content in the virtual screen, wherein the mapping area is a similar rectangle of the virtual screen and displays the application picture in the virtual screen. And after the touch screen operation of the user in the mapping area is carried out and the coordinate is amplified in equal proportion, the coordinate can be directly transmitted to the application in the virtual screen.

Fig. 9 is an eighth flowchart of an application manipulation method of a virtual screen according to the present invention, and based on the foregoing embodiment, the method further includes:

s43, when the virtual screen is displayed on an external screen of the intelligent terminal, displaying the display content of the virtual screen and/or displaying a touch panel in the operation interface through a second mapping area controlled in the operation interface in the operation processing;

s44, amplifying the touch event of the second mapping area according to an equal proportion through the operation processing central control, and transmitting the amplified touch event to the selected application of the virtual screen, and/or converting the touch event of the touch panel according to panel attributes and transmitting the converted touch event to the selected application of the virtual screen.

Optionally, in this embodiment, please refer to an external display schematic diagram shown in fig. 14. When the mobile phone is connected with the external screen, a user can see the external screen, in order to save power consumption, the operation interface can enter a user-defined touch panel mode instead of displaying the content of the virtual screen, wherein an area in the operation interface corresponds to each area of the corresponding number in the application interface of the virtual screen, in order to realize the operation experience of blind operation, the effective touch area in the general operation interface is larger, and the position is adjusted according to the habit of the user. And the coordinate of the touch screen point actually operated by the user is reduced and then transmitted to the application in the virtual screen.

Based on the foregoing embodiment, the present invention further provides an application control device for a virtual screen, where the device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the application control method for a virtual screen according to any one of the foregoing embodiments.

It should be noted that the apparatus embodiment and the method embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment, and technical features in the method embodiment are applicable in the apparatus embodiment, which is not described herein again.

Based on the above embodiment, the present invention further provides a computer readable storage medium, where an application manipulation program of a virtual screen is stored, and when being executed by a processor, the application manipulation program of the virtual screen implements the steps of the application manipulation method of the virtual screen according to any one of the above embodiments.

It should be noted that the media embodiment and the method embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment, and technical features in the method embodiment are correspondingly applicable in the media embodiment, which is not described herein again.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An application control method of a virtual screen is applied to an intelligent terminal comprising an operation processing central control, wherein the operation processing central control comprises a virtual screen management module, an event acquisition module, an event conversion module and an event transmission module, and the method comprises the following steps:

the event acquisition module takes over one or more of operation events of a touch screen and/or a sensor of the intelligent terminal, operation events transmitted through input equipment connected with the outside and operation events transmitted through wireless connection with other terminals;

2. The application control method of the virtual screen according to claim 1, wherein the intelligent terminal further comprises a setting module and an operation interface module, and the method further comprises:

3. The application manipulation method of the virtual screen according to claim 2, further comprising:

4. The application manipulation method of the virtual screen according to claim 3, further comprising:

5. The application manipulation method of the virtual screen according to claim 4, further comprising:

and loading resources and displaying the resources in a layout way through the virtual screen management module according to the size setting parameters, the density setting parameters, the selection instructions and/or the switching instructions.

6. The application manipulation method of the virtual screen according to claim 5, further comprising:

7. The application manipulation method of the virtual screen according to claim 6, further comprising:

8. The application manipulation method of the virtual screen according to claim 7, further comprising:

9. An application manipulation device for a virtual screen, characterized in that the device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the application manipulation method for a virtual screen according to any one of claims 1 to 8.

10. A computer-readable storage medium, on which an application manipulation program of a virtual screen is stored, the application manipulation program of the virtual screen implementing the steps of the application manipulation method of the virtual screen according to any one of claims 1 to 8 when executed by a processor.