CN118042040A

CN118042040A - Multi-terminal expansion display method, device and system of cloud mobile phone

Info

Publication number: CN118042040A
Application number: CN202410175126.6A
Authority: CN
Inventors: 陈有权; 王嘉; 党戈; 黄鹤羽; 丁小波; 赵鹏; 李悦; 梁小龙; 叶耀基; 许晓燕
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Internet Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Internet Co Ltd
Priority date: 2024-02-07
Filing date: 2024-02-07
Publication date: 2024-05-14

Abstract

The application provides a multi-terminal expansion display method, device and system of a cloud mobile phone. The method applied to the cloud mobile phone server comprises the following steps: determining actual position information and relative position information of a screen display area of each accessed terminal device according to screen size information of a plurality of accessed terminal devices at present in response to successful access of the first terminal device; cutting the cloud mobile phone video stream data according to the relative position information to obtain partitioned video stream data corresponding to each accessed terminal device; the partitioned video stream data and the actual position information are respectively sent to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on the screen of the accessed terminal equipment based on the actual position information of the accessed terminal equipment, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the screen of the terminal can be broken through, and the requirement for amplifying and displaying the picture of the cloud mobile phone is met.

Description

Multi-terminal expansion display method, device and system of cloud mobile phone

Technical Field

The application relates to the technical field of cloud mobile phones, in particular to a multi-terminal expansion display method, device and system of a cloud mobile phone.

Background

The cloud mobile phone refers to a virtualized mobile phone service based on a cloud computing technology. By placing core resources such as a mobile phone operating system and application programs on a cloud server, a user can access and use mobile phone functions of the cloud through network connection from terminal equipment (such as a smart phone, a tablet computer and a computer).

In the related art, the display scheme of the cloud mobile phone supports two forms of single terminal display and simultaneous display of multiple terminals, but when the multiple terminals are simultaneously displayed, the same content can only be displayed on the multiple terminals, and the requirement on amplifying and displaying the content of the cloud mobile phone can not be met.

Disclosure of Invention

In order to solve the problems, the application provides a multi-terminal expansion display method, device and system of a cloud mobile phone.

According to a first aspect of the present application, there is provided a multi-terminal extension display method of a cloud mobile phone, applied to a cloud mobile phone server, including:

Responding to successful access of the first terminal equipment to the cloud mobile phone server, and determining actual position information and relative position information of a screen display area of each accessed terminal equipment according to screen size information of a plurality of accessed terminal equipment;

cutting the cloud mobile phone video stream data according to the relative position information to obtain partition video stream data corresponding to each accessed terminal device;

and respectively sending the partitioned video stream data and the actual position information to corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on a screen of the accessed terminal equipment based on the actual position information of the accessed terminal equipment.

According to a second aspect of the present application, there is provided a multi-terminal extension display method of a cloud mobile phone, applied to a terminal device, including:

Receiving actual position information of a screen display area of the terminal equipment and partition video stream data corresponding to the terminal equipment, wherein the actual position information is sent by a cloud mobile phone server; the partitioned video stream data are obtained by cutting the cloud mobile phone video stream data by the cloud mobile phone server based on the relative position information of the screen display area of each accessed terminal device; the actual position information and the relative position information are determined by the cloud mobile phone server based on screen size information of a plurality of accessed terminal devices;

And displaying the partitioned video stream data on a screen of the terminal equipment according to the actual position information.

According to a third aspect of the present application, there is provided a multi-terminal expansion display device of a cloud mobile phone, applied to a cloud mobile phone server, including:

The first determining module is used for responding to successful access of the first terminal equipment to the cloud mobile phone server and determining actual position information and relative position information of a screen display area of each accessed terminal equipment according to screen size information of a plurality of accessed terminal equipment;

the cutting module is used for cutting the cloud mobile phone video stream data according to the relative position information to obtain partition video stream data corresponding to each accessed terminal device;

And the first sending module is used for respectively sending the partitioned video stream data and the actual position information to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on a screen of the accessed terminal equipment based on the actual position information of the accessed terminal equipment.

According to a fourth aspect of the present application, there is provided a multi-terminal expansion display device of a cloud mobile phone, applied to a terminal device, including:

The first receiving module is used for receiving the actual position information of the screen display area of the terminal equipment and the partitioned video stream data corresponding to the terminal equipment, which are sent by the cloud mobile phone server; the partitioned video stream data are obtained by cutting the cloud mobile phone video stream data by the cloud mobile phone server based on the relative position information of the screen display area of each accessed terminal device; the actual position information and the relative position information are determined by the cloud mobile phone server based on screen size information of a plurality of accessed terminal devices;

and the display module is used for displaying the partitioned video stream data on a screen of the terminal equipment according to the actual position information.

According to a fifth aspect of the present application, there is provided a multi-terminal extended display system of a cloud mobile phone, including:

the cloud mobile phone server is used for executing the method of the first aspect;

a terminal device configured to perform the method according to the second aspect.

According to a sixth aspect of the present application, there is provided an electronic device comprising: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to implement the method of the first aspect and/or to implement the method of the second aspect.

According to a seventh aspect of the present application, there is provided a computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the method of the first aspect and/or to perform the method of the second aspect.

According to the technical scheme, after the first terminal equipment is successfully accessed, the cloud mobile phone server determines the actual position information and the relative position information of the screen display area of each accessed terminal equipment according to the screen size information of the current accessed terminal equipment; cutting the cloud mobile phone video stream data according to the relative position information to obtain partitioned video stream data corresponding to each accessed terminal device; the partitioned video stream data and the actual position information are respectively sent to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on a screen based on the actual position information, the capability of displaying the cloud mobile phone picture by combining multiple terminals can be realized, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the terminal screen is broken through, and the requirement of amplifying and displaying the cloud mobile phone picture is met.

Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.

Drawings

The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

fig. 1 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to an embodiment of the present application;

fig. 2 is an exemplary diagram of a multi-terminal extended display of a cloud mobile phone according to an embodiment of the present application;

fig. 3 is a flowchart of another multi-terminal extended display method of a cloud mobile phone according to an embodiment of the present application;

Fig. 4 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to another embodiment of the present application;

FIG. 5 is an exemplary diagram of determining an optimal screen stitching scheme in an embodiment of the present application;

fig. 6 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to another embodiment of the present application;

Fig. 7 is an exemplary diagram of determining actual position information and relative position information of a screen display area according to an embodiment of the present application;

Fig. 8 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to another embodiment of the present application;

Fig. 9 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to another embodiment of the present application;

Fig. 10 is a flowchart of a multi-terminal extended display method of a cloud mobile phone according to another embodiment of the present application;

Fig. 11 is a structural block diagram of a multi-terminal extended display system of a cloud mobile phone according to an embodiment of the present application;

fig. 12 is a block diagram of a multi-terminal expansion display device of a cloud mobile phone according to an embodiment of the present application;

fig. 13 is a block diagram of a multi-terminal expansion display device of another cloud mobile phone according to an embodiment of the present application;

Fig. 14 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.

In the technical scheme of the application, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated. The user personal information involved is acquired, stored and applied with consent of the user.

It should be noted that, the cloud mobile phone refers to a virtualized mobile phone service based on a cloud computing technology. By placing core resources such as a mobile phone operating system and application programs on a cloud server, a user can access and use mobile phone functions of the cloud through network connection from terminal equipment (such as a smart phone, a tablet computer and a computer). The working principle of the cloud mobile phone is to simulate and operate a complete mobile phone environment on a cloud server, wherein the complete mobile phone environment comprises an operating system, application programs, storage and computing resources and the like. The user establishes a connection with the cloud handset through a terminal device and network connection, such as in a mobile application or browser, as if a real handset were used.

In the related art, the display scheme of the cloud mobile phone supports two forms of single terminal display and simultaneous display of multiple terminals. The single terminal is displayed independently, namely, one cloud mobile phone is only displayed on one terminal, the interface of the cloud mobile phone is updated to the terminal in real time through a video stream form, and meanwhile, a user can operate the terminal and control the cloud mobile phone through uploading an operation instruction. The simultaneous display of a plurality of terminals means that one cloud mobile phone is simultaneously displayed on a plurality of terminals, the content displayed by each terminal is the same, each terminal can control the cloud mobile phone, the scene is called as screen interaction, and the method is suitable for scenes such as game playing, file sharing and the like. However, when a plurality of terminals are simultaneously displayed, the same content can only be displayed on the plurality of terminals, and the requirement for amplifying and displaying the content of the cloud mobile phone can not be met.

Fig. 1 is a flowchart of a multi-terminal expansion display method of a cloud mobile phone according to an embodiment of the present application. It should be noted that, the multi-terminal extension display method of the cloud mobile phone in the embodiment of the application is applied to a cloud mobile phone server. The multi-terminal expansion display method of the cloud mobile phone provided by the embodiment of the application can be used for the multi-terminal expansion display device of the cloud mobile phone in the embodiment of the application, and the device can be configured in electronic equipment. As shown in fig. 1, the method may include the steps of:

Step 101, in response to the first terminal device successfully accessing the cloud mobile phone server, determining actual position information and relative position information of a screen display area of each accessed terminal device according to screen size information of the current accessed terminal devices.

The first terminal device may be a terminal device such as a tablet, a smart phone, or a computer, and the successful access of the first terminal device to the cloud mobile phone server means that the first terminal device successfully accesses to the cloud mobile phone server in an extension mode.

As an example, the main device can access to the cloud mobile phone server through the cloud mobile phone client installed by the main device, and the cloud mobile phone server successfully accesses to the cloud mobile phone server through authentication and instance distribution of the main device; the main equipment is terminal equipment of a first access cloud mobile phone instance; when the expansion display is needed, the main equipment can share the access two-dimension code with the first terminal equipment, the first terminal equipment sends an access request to the cloud mobile phone server through describing the access two-dimension code, and the cloud mobile phone server pairs, authenticates and connects the cloud mobile phone instance connected with the main equipment after receiving the access request.

As a possible implementation manner, after the host device accesses the cloud mobile phone server, the device information of the host device may be actively uploaded to the cloud mobile phone server, for example: model, operating system, resolution, etc. The cloud mobile phone server can send the video stream in real time, and the main equipment is rendered into a real-time picture. In addition, the cloud mobile phone server can also send information such as a login token and a cloud mobile phone address to the main equipment, and when the expansion display is needed, the main equipment can generate an access two-dimensional code based on the information such as the login token and the cloud mobile phone address and share the access two-dimensional code to the first terminal equipment so as to be accessed by the first terminal equipment through code scanning.

In some embodiments of the present application, the current plurality of accessed terminal devices are terminal devices that have been successfully accessed to the cloud mobile phone server, including the first terminal device, the master device, and other terminal devices that have been accessed to the cloud mobile phone server. The screen size information of each accessed terminal device can comprise the screen length and width of the terminal device, and the screen size information of each accessed terminal device can be directly obtained by the cloud mobile phone server through a corresponding interface according to the model of the accessed terminal device, or can be sent by the accessed terminal device to the cloud mobile phone server.

The screen display area of each accessed terminal device refers to an area for displaying a cloud mobile phone picture in the screen of each accessed terminal device. The actual position information of the screen display area of the accessed terminal device a refers to the position information of the screen display area of the accessed terminal device a for the entire screen of the accessed terminal device a. The relative position information of the screen display area of the accessed terminal device a refers to the position information of the screen display area of the accessed terminal device a with respect to the quadrangle corresponding to the cloud mobile phone display area determined based on the current plurality of accessed terminal devices.

In some embodiments of the present application, determining the actual location information and the relative location information of the screen display area of each of the accessed terminal devices based on the screen size information of the current plurality of accessed terminal devices may comprise: according to the screen size information of a plurality of accessed terminal devices, based on a preset splicing scheme, the screen of each accessed terminal device is regarded as one quadrangle, and all quadrangles are spliced; determining a quadrangle corresponding to the cloud mobile phone display area based on the spliced graph and a preset length-width ratio; determining the area of the quadrangle corresponding to the screen of each accessed terminal equipment in the quadrangle corresponding to the mobile phone display area as the screen display area of each accessed terminal equipment; determining actual position information of a screen display area of each accessed terminal device based on the screen size information of each terminal device and the spliced direction; and establishing a rectangular coordinate system by taking a certain angle in a quadrangle corresponding to the cloud mobile phone display area as an origin, determining four-corner coordinates of the screen display area of each accessed terminal device under the coordinate system, and taking the four-corner coordinates as relative position information of the screen display area of the accessed terminal device.

In other embodiments of the present application, the screen of the accessed terminal devices may be regarded as a plurality of quadrilaterals to be spliced in different manners according to the screen size information of the accessed terminal devices, and the size of each quadrilaterals is consistent with the screen size of the corresponding accessed terminal device; determining an optimal screen splicing scheme from different splicing modes based on the quadrangle corresponding to the cloud mobile phone display area determined by each splicing mode; based on the optimal splicing scheme, determining a quadrangle corresponding to the cloud mobile phone display area, determining the screen display area of each accessed terminal device, and determining the actual position information and the relative position information of the screen display area of each accessed terminal device.

And 102, cutting the cloud mobile phone video stream data according to the relative position information to obtain the partitioned video stream data corresponding to each accessed terminal device.

That is, according to the relative position information of the screen display areas of the accessed terminal devices, the cloud mobile phone video stream data can be cut into a plurality of pieces to obtain the partition video stream data corresponding to each accessed terminal device, and the number of the partition video stream data obtained after cutting is the same as the number of the accessed terminal devices.

As an example, as shown in fig. 2, if the number of accessed terminal devices is 3, namely, terminal device 1, terminal device 2 and terminal device 3, the screen display area of terminal device 1 is area a, the screen display area of terminal device 2 is area b, the screen display area of terminal device 3 is area c, and areas a, b and c constitute a cloud mobile phone display area. For example, a rectangular coordinate system can be established by taking the upper left corner of the display area of the cloud mobile phone as the origin, so as to determine the relative position information of each screen display area, and each frame of data in the video stream data of the cloud mobile phone is cut according to the relative position information, so that the partitioned video stream data corresponding to each accessed terminal device is obtained.

Step 103, the partitioned video stream data and the actual position information are respectively sent to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on the screen of the accessed terminal equipment based on the actual position information of the accessed terminal equipment.

That is, the obtained partitioned video stream data and the actual position information of the screen display area of each accessed terminal device are respectively sent to the corresponding accessed terminal device by cutting the video stream data of the cloud mobile phone, so that each accessed terminal device displays the corresponding partitioned video stream data in the corresponding screen display area, and multi-terminal expansion display of the cloud mobile phone is realized.

If the number of accessed terminal devices is 3, namely the terminal device 1, the terminal device 2 and the terminal device 3, the actual position information of the screen display area of the terminal device 1 and the partitioned video stream data corresponding to the terminal device 1 are sent to the terminal device 1, the actual position information of the screen display area of the terminal device 2 and the partitioned video stream data of the terminal device 2 are sent to the terminal device 2, and the actual position information of the screen display area of the terminal device 3 and the partitioned video stream data of the terminal device 3 are sent to the terminal device 3. As shown in fig. 2, when only one terminal device accesses the cloud mobile phone server, the cloud mobile phone server sends the complete video stream data to the terminal device, so that the terminal device displays the picture of the cloud mobile phone. When three terminal devices are connected to the cloud mobile phone server, the terminal devices respectively receive the video stream data subjected to the processing partition, namely, the three terminal devices respectively display a part of the cloud mobile phone picture, and the three terminal devices are spliced into an expanded cloud mobile phone interface, so that multi-terminal expansion display of the cloud mobile phone is realized.

That is, after the new terminal device is successfully accessed, the steps 101-103 are executed, if no new terminal device is accessed, the cloud mobile phone video stream data is still cut according to the last determined relative position information of the screen display area of each accessed terminal device, the obtained partitioned video stream data and the actual position information of each screen display area are sent to the corresponding accessed terminal device, the accessed terminal device is updated until the new terminal device is successfully accessed, the actual position information and the relative position information of the screen display area of each accessed terminal device are redetermined, and the cutting scheme is updated.

In some embodiments of the present application, the partitioned video stream data may be video encoded and compressed before being transmitted to the corresponding accessed terminal device.

In order to improve the picture display effect, the first terminal equipment can also send equipment information such as machine type, resolution ratio and the like to the cloud mobile phone server, and after receiving an access request of the first terminal equipment, the cloud mobile phone server carries out login authentication on the first terminal equipment and also carries out adaptation verification on the first terminal equipment. For example, because the screen size difference between the terminal devices such as a flat panel, a television, a watch and a smart phone is large, in order to improve the display effect, the cloud mobile phone server can check the model of the first terminal device, if the model of the first terminal device is consistent with the model of the main device, access is allowed, otherwise, access is not allowed; for another example, the display effect is also affected by performing extended display on the terminal devices with different resolutions, the cloud mobile phone server can compare the resolution of the first terminal device with the resolution of the main device, if the two are consistent, access is allowed, otherwise, the access of the first terminal device fails.

In addition, in order to reduce invalid screen areas and influence the picture continuity, the cloud mobile phone server can judge the full screen mode of the first terminal equipment when determining the access result of the first terminal equipment, for example, the android system can acquire the available area of the screen based on windowInsets interfaces provided by the system, if the first terminal equipment is in a non-full screen mode, a switching notification can be sent to the first terminal equipment, if the first terminal equipment is switched to the full screen mode within a preset time period, the access is successful, otherwise the access is failed.

According to the multi-terminal expansion display method of the cloud mobile phone, after the first terminal equipment is successfully accessed, the cloud mobile phone server determines the actual position information and the relative position information of the screen display area of each accessed terminal equipment according to the screen size information of the current accessed terminal equipment; cutting the cloud mobile phone video stream data according to the relative position information to obtain partitioned video stream data corresponding to each accessed terminal device; the partitioned video stream data and the actual position information are respectively sent to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on a screen based on the actual position information, the capability of displaying the cloud mobile phone picture by combining multiple terminals can be realized, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the terminal screen is broken through, and the requirement of amplifying and displaying the cloud mobile phone picture is met.

Fig. 3 is a flowchart of another multi-terminal extension display method of a cloud mobile phone according to an embodiment of the present application. As shown in fig. 3, based on the above embodiment, the implementation of step 101 in fig. 1 may include the following steps:

step 301, determining an optimal screen splicing scheme of a plurality of accessed terminal devices according to the screen size information of each accessed terminal device.

It will be appreciated that, according to the screen size information of a plurality of accessed terminal devices, the screen of the accessed terminal device is regarded as quadrangles with corresponding sizes, and the quadrangles are spliced, and a plurality of splicing schemes are possible. In order to enhance the extended display effect, an optimal screen stitching scheme may be determined from among a plurality of stitching schemes. For example, the display area can be determined based on the size of the area for cloud mobile phone interface display obtained after the splicing, and also can be determined based on the area ratio of the display area obtained after the splicing.

In some embodiments of the present application, a plurality of splicing schemes may be preset based on screen size information of different terminals, and according to the screen size information of each accessed terminal device, splicing and combining are performed in a quadrilateral form with the same size, so as to obtain a plurality of spliced graphs; because the cloud mobile phone is preset with the screen display proportion, a largest quadrangle meeting the display proportion of the cloud mobile phone can be determined in the spliced graph, the largest quadrangle is used as an area for expanding and displaying the interface of the cloud mobile phone, and the splicing scheme with the largest area of the obtained quadrangle is determined to be the optimal screen splicing scheme.

In other embodiments of the present application, all the splicing combinations may be traversed for the quadrangles corresponding to the screens of the accessed terminal devices based on the screen size information of the accessed terminal devices based on a greedy algorithm, a quadrangle meeting the aspect ratio of the cloud mobile phone is determined based on the splicing pattern obtained by each splicing mode, the area ratio of each obtained quadrangle and the corresponding splicing pattern, that is, the area ratio of the display area, is determined, and the splicing scheme corresponding to the largest area ratio is determined as the optimal screen splicing scheme.

Step 302, determining actual location information and relative location information of a screen display area of each accessed terminal device based on an optimal screen stitching scheme.

It can be understood that, based on the optimal screen splicing scheme, the size of the area of the cloud mobile phone interface expansion display and the size of the quadrangle corresponding to the screen of each accessed terminal device falling in the area can be determined, so that the screen display area and the cloud mobile phone display area of each accessed terminal device can be determined, based on the screen size information of the accessed terminal device, the actual position information of the screen display area can be determined, and based on the position relationship between the screen display area of the accessed terminal device and the cloud mobile phone display area, the relative position information of the screen display area of each accessed terminal device can be determined.

According to the multi-terminal expansion display method of the cloud mobile phone, in response to the success of the first terminal access, an optimal screen splicing scheme of a plurality of accessed terminal devices is determined according to the screen size information of the current plurality of accessed terminal devices, the actual position information and the relative position information of the screen display area of each accessed terminal device are determined based on the optimal screen splicing scheme, then the cloud mobile phone video stream data are cut based on the relative position information, a plurality of subarea video stream data are obtained, each subarea video stream data and the corresponding actual position information are respectively sent to the corresponding accessed terminal devices, so that expansion display of content when the plurality of terminal devices are connected with the cloud mobile phone is achieved, and the expansion display effect can be improved.

Fig. 4 is a flowchart of a multi-terminal extension display method of a cloud mobile phone according to another embodiment of the present application. As shown in fig. 4, based on the above embodiment, the implementation procedure of step 301 in fig. 3 may include:

And step 401, according to the screen size information of each accessed terminal device, arranging and combining the screens of a plurality of accessed terminal devices to determine all screen splicing schemes.

That is, according to the screen size information of each accessed terminal device, the screen of each accessed terminal device is regarded as a quadrangle with a corresponding size, and then a plurality of quadrangles are spliced in a permutation and combination mode, so that all screen splicing schemes are obtained.

Step 402, determining a maximum quadrilateral display area corresponding to each screen splicing scheme based on a preset display proportion.

It can be understood that the cloud mobile phone server presets the display proportion of the cloud mobile phone, so that the display proportion of the spliced cloud mobile phone display area needs to be met.

In some embodiments of the present application, determining the largest quadrilateral display area corresponding to each screen stitching scheme is equivalent to determining the first largest quadrilateral area in the stitching area corresponding to each screen stitching scheme, where the aspect ratio of the largest quadrilateral area is consistent with a preset display scale.

In step 403, the area of each of the largest quadrilateral display areas is determined.

And 404, determining a screen splicing scheme corresponding to the largest quadrilateral display area with the largest area as an optimal screen splicing scheme.

For example, as shown in fig. 5, if the number of accessed terminal devices is 3, namely, terminal device 1, terminal device 2 and terminal device 3, and the aspect ratio of the screens of the three terminal devices is 16:9, the display ratio of the cloud mobile phone is 16:9, based on the first screen splicing scheme, the obtained maximum quadrilateral display area is 27 and the width is 15.18, based on the second screen splicing scheme, the obtained maximum quadrilateral display area is 25:14.06, and since the area of the maximum quadrilateral display area determined by the second screen splicing scheme is smaller than the maximum quadrilateral display area determined by the first screen splicing scheme, and the areas of the maximum quadrilateral display areas determined by the other screen splicing schemes are smaller than the maximum quadrilateral display area determined by the first screen splicing scheme, the first screen splicing scheme is determined as the optimal screen splicing scheme.

According to the multi-terminal extension display method of the cloud mobile phone, based on the display proportion, the largest quadrilateral display area corresponding to each screen splicing scheme is determined, the area of each largest quadrilateral display area is determined, and the screen splicing scheme corresponding to the largest quadrilateral display area with the largest area is determined to be the optimal screen splicing scheme. According to the scheme, the splicing scheme with the largest area of the determined largest quadrilateral display area is determined to be the optimal screen splicing scheme, so that the cloud mobile phone display area can be maximized, the display size of a cloud mobile phone picture can be maximized, and the expansion display effect can be improved.

Next, a detailed description will be given of an implementation procedure for determining actual position information and relative position information of a screen display area of each accessed terminal device.

Fig. 6 is a flowchart of a multi-terminal extension display method of a cloud mobile phone according to another embodiment of the present application. As shown in fig. 6, based on the above embodiment, the implementation procedure of step 302 in fig. 3 may include the following steps:

Step 601, determining a cloud mobile phone display area corresponding to the optimal screen splicing scheme and a screen display area of each accessed terminal device.

The cloud mobile phone display area corresponding to the optimal screen splicing scheme is a largest quadrilateral area which meets a preset display proportion and is determined from the splicing area obtained by the optimal screen splicing scheme. The screen display area of each accessed terminal device refers to an area in which a quadrilateral area represented by a screen of the accessed terminal device falls in the cloud mobile phone display area after being spliced.

For example, if the number of accessed terminal devices is 3, which is terminal device 1, terminal device 2 and terminal device 3 respectively, the screen splicing mode in fig. 7 is the optimal screen splicing mode, and the cloud mobile phone display area is the display area a in fig. 7. The screen display area of each accessed terminal device is a shadow area corresponding to each accessed terminal device, and taking the terminal device 2 as an example, the screen display area of the terminal device 2 is a display area b in fig. 7.

At step 602, the actual location information of the screen display area of each accessed terminal device is determined.

In some embodiments of the present application, the actual location information of the screen display area of each accessed terminal device may include screen direction information and actual coordinate information. The screen direction information refers to the screen direction of the accessed terminal equipment in the optimal splicing scheme, for example, the screen direction is distinguished by four numbers of 0, 1,2 and 3, wherein 0 represents the vertical screen direction, 1 represents the horizontal screen direction, 2 represents the reverse vertical screen direction, and 3 identifies the reverse horizontal screen direction. The actual coordinate information of the screen display area can be determined by constructing the self coordinate system of the accessed terminal device.

As one possible implementation manner, determining the actual location information of the screen display area of each accessed terminal device includes: determining screen direction information of each accessed terminal device in an optimal screen combination scheme; determining actual coordinate information of a screen display area of each accessed terminal device based on the screen direction information and the screen size information; screen direction information and actual coordinate information are determined as actual position information.

As shown in fig. 7, a rectangular coordinate system may be established at the upper left corner corresponding to the screen direction of the accessed terminal device, for example, a rectangular coordinate system may be established at the upper left corner of the screen area corresponding to the terminal device 2, and coordinate information of the screen display area of the terminal device 2 in the rectangular coordinate system may be determined and used as actual coordinate information of the screen display area of the terminal device 2. The actual coordinate information of the screen display area of the terminal device 2 is (Xb 1, yb1, xb2, yb 2).

And step 603, determining the relative position information of the screen display area of each accessed terminal device in the cloud mobile phone display area based on the optimal screen splicing scheme.

In some embodiments of the present application, a rectangular coordinate system may be constructed based on the determined cloud mobile phone display area, and coordinate information of each screen display area under the coordinate system may be determined as relative position information of each screen display area in the cloud mobile phone display area.

As shown in fig. 7, a rectangular coordinate system may be established with the upper left corner of the display area of the cloud mobile phone as the origin, so as to obtain (Xa 1, ya1, xa2, ya 2) the relative position information of the screen display area of the terminal device 1, (Xa 1, ya2, xa2, ya 3) the relative position information of the screen display area of the terminal device 2, and (Xa 1, ya3, xa2, ya 4) the relative position information of the screen display area of the terminal device 3.

In some embodiments of the present application, after the accessed terminal device displays the corresponding partitioned video stream data, the cloud mobile phone may be further locally controlled based on the displayed partitioned video stream data.

Fig. 8 is a flowchart of a multi-terminal extension display method of a cloud mobile phone according to another embodiment of the present application. As shown in fig. 8, based on the above embodiment, the multi-terminal extended display method of the cloud mobile phone of the present application may further include the following steps:

step 801, receiving operation instruction information sent by a second terminal device; the operation instruction information comprises an event type for triggering operation and first trigger position information; wherein the second terminal device is any one of a plurality of accessed terminal devices.

That is, when the user of the accessed terminal device triggers the displayed partitioned video stream data, the accessed terminal device determines the event type (such as a sliding event, a clicking event, etc.) of the triggering operation and the position information of the triggering operation, that is, the first triggering position information, and then sends the operation instruction information including the event type of the triggering operation and the first triggering position information to the cloud mobile phone server. The first trigger position information may be coordinate information of a position of the trigger operation in a coordinate system of a screen direction of the terminal device.

Step 802, determining an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal device.

As one possible implementation, the implementation procedure of step 802 may include: determining second trigger position information corresponding to the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal device; and determining an event to be operated according to the event type and the second trigger position information.

The first trigger position information can be actual coordinate information of the trigger operation position in a coordinate system corresponding to the screen direction of the second terminal equipment, and the cloud mobile phone server stores the actual position information and the relative position information of the screen display area of the second terminal equipment, so that the coordinate information of the trigger operation position in the cloud mobile phone display area can be obtained through conversion according to the first trigger position information, the position information of the trigger operation in the cloud mobile phone display area, namely the second trigger position information, can be determined, and then an event to be operated is determined according to the second trigger position information and the event type.

Step 803, processing the event to be operated, obtaining an event processing result, and sending the event processing result to a plurality of accessed terminal devices respectively, so that each accessed terminal device displays the event processing result.

That is, each of the accessed terminal devices may interoperate on a respective displayed screen, and each of the accessed terminal devices may synchronously receive a result of the interoperation.

The event processing result may be feedback notification data, and may also be video stream data. For example, when the event to be processed is sliding left, the event processing result is the sliding video stream data, so when the event processing result is sent to a plurality of accessed terminal devices, the video stream data is divided based on the dividing mode determined by the above embodiment, and the obtained divided video stream data and the corresponding actual position information are sent to each accessed terminal device.

According to the multi-terminal expansion display method of the cloud mobile phone, the cloud mobile phone server can receive the operation instruction information sent by the second terminal equipment, determine an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal equipment, process the event to be operated, and send the event processing result to each accessed terminal equipment so that each accessed terminal equipment can display the event processing result. That is, the cloud mobile phone can be controlled by each accessed terminal device, and user experience is further improved.

Next, a multi-terminal extended display method for a cloud mobile phone on a terminal device side will be described.

Fig. 9 is a flowchart of a multi-terminal extension display method of a cloud mobile phone according to another embodiment of the present application. It should be noted that, the multi-terminal extension display method of the cloud mobile phone according to the embodiment of the present application may be applied to a terminal device, where the terminal device may be a current accessed terminal device. As shown in fig. 9, the method may include:

Step 901, receiving actual position information of a screen display area of a terminal device and partition video stream data corresponding to the terminal device, which are sent by a cloud mobile phone server; the partitioned video stream data are obtained by cutting the cloud mobile phone video stream data by the cloud mobile phone server based on the relative position information of the screen display area of each accessed terminal device; the actual position information and the relative position information are determined by the cloud mobile phone server based on screen size information of a plurality of accessed terminal devices.

And step 902, displaying the partitioned video stream data on a screen of the terminal equipment according to the actual position information.

In some embodiments of the present application, the actual location information may include actual coordinate information and screen direction information of a screen display area of the terminal device. For example, the display direction of the divided video stream data may be determined according to the screen direction information, and the divided video stream data may be displayed in the screen display area based on the actual coordinate information and the display direction.

It should be noted that, the screen of each accessed terminal device displays a part of video stream data, and the content displayed by each accessed terminal device is different, and the screens of a plurality of accessed terminal devices are spliced to form a large screen, so as to obtain the display interface of the whole cloud mobile phone, that is, the user can splice the screens of a plurality of accessed terminal devices, so as to realize the expansion display of the cloud mobile phone interface.

According to the multi-terminal expansion display method of the cloud mobile phone, the terminal equipment receives the actual position information of the screen display area of the terminal equipment, which is sent by the cloud mobile phone server, and the partitioned video stream data corresponding to the terminal equipment, and displays the partitioned video stream data on the screen of the terminal equipment according to the actual position information. According to the scheme, when the multi-terminal is accessed to the cloud mobile phone server, all the accessed terminal equipment displays a part of content of a cloud mobile phone interface, and a large screen is formed by splicing a plurality of terminal equipment, so that the capability of displaying the cloud mobile phone pictures in a multi-terminal combined way can be realized, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the terminal screen is broken through, and the requirement of amplifying and displaying the cloud mobile phone pictures is met.

In other embodiments of the present application, as shown in fig. 10, based on the above embodiment, the method may further include the steps of:

Step 1001, in response to a triggering operation of the partitioned video stream data, sending operation instruction information to a cloud mobile phone server based on the triggering operation; the operation instruction information comprises an event type for triggering operation and first trigger position information; the cloud mobile phone server determines an event to be operated based on the actual position information and the relative position information of the screen display area of the terminal equipment, processes the event to be operated, and determines an event processing result.

Step 1002, receiving an event processing result sent by the cloud mobile phone server, and displaying the event processing result on a screen of the terminal device.

According to the multi-terminal expansion display method of the cloud mobile phone, the terminal equipment responds to triggering operation of the partitioned video stream data, sends operation instruction information to the cloud mobile phone server based on the triggering operation, receives event processing results sent by the cloud mobile phone server, and displays the event processing results on a screen of the terminal equipment. That is, the cloud mobile phone can be controlled by each accessed terminal device, and user experience is further improved.

In order to achieve the above embodiment, the embodiment of the present application further provides a multi-terminal extended display system of a cloud mobile phone.

Fig. 11 is a structural block diagram of a multi-terminal extended display system of a cloud mobile phone according to an embodiment of the present application. As shown in fig. 11, the system includes a cloud mobile phone server 1101 and a plurality of terminal devices, and fig. 11 illustrates that the number of terminal devices is 3, for example: a first terminal device 1102, a second terminal device 1103 and a third terminal device 1104. The plurality of terminal devices are all terminal devices that have been connected to the cloud mobile phone server 1101. The first terminal device 1102, the second terminal device 1103 and the third terminal device 1104 may be smart phones, computers, tablets, smart watches, televisions and the like.

In some embodiments of the present application, the cloud mobile phone server 1101 is configured to perform the multi-terminal extended display method of the cloud mobile phone applied to the cloud mobile phone server in the above embodiments, and the first terminal device 1102, the second terminal device 1103, and the third terminal device 1104 are configured to perform the multi-terminal extended display method of the cloud mobile phone applied to the cloud mobile phone server of the terminal device in the above embodiments. Next, an interaction between the cloud handset server 1101 and the first terminal device 1102, the second terminal device 1103, and the third terminal device 1104 will be described:

(1) If the first terminal device 1102 is not already connected to the cloud mobile phone server 1101, both the second terminal device 1103 and the third terminal device 1104 are already connected to the cloud mobile phone server 1101. Before the first terminal device 1102 accesses the cloud mobile phone server 1101, an access request may be sent to the cloud mobile phone server 1101 through the two-dimension code shared by the code scanning master device, where the access request may include access information, and the access request is used by the cloud mobile phone server 1101 to verify the access of the first terminal device 1102. In addition, the first terminal device 1102 may also send device information to the cloud mobile phone server 1101 during the access process, and when the cloud mobile phone server 1101 performs verification on the access of the first terminal device 1102, the verification may also be performed based on the received device information.

(2) After the first terminal device 1102 is successfully accessed, the cloud mobile phone server 1101 determines the actual position information and the relative position information of the screen display area of each accessed terminal device according to the screen size information of the current accessed terminal devices; cutting the cloud mobile phone video stream data according to the relative position information to obtain partitioned video stream data corresponding to each accessed terminal device; corresponding partition video stream data and actual position information are transmitted to the first terminal device 1102, the second terminal device 1103, and the third terminal device 1104, respectively.

(3) The first terminal device 1102, the second terminal device 1103 and the third terminal device 1104 respectively receive the actual position information of the screen display area of the corresponding terminal device sent by the cloud mobile phone server 1101 and the partitioned video stream data corresponding to each terminal device; and displaying the partitioned video stream data on a screen of the terminal device according to the actual position information.

(4) If any one of the first terminal device 1102, the second terminal device 1103, and the third terminal device 1104 receives a trigger operation for the partitioned video stream data, the operation instruction information may be sent to the cloud mobile phone server 1101 based on the trigger operation.

(5) The cloud mobile phone server 1101 receives the operation instruction information, and determines an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the corresponding terminal device; the event to be operated is processed, an event processing result is obtained, and the event processing result is sent to the first terminal device 1102, the second terminal device 1103, and the third terminal device 1104, respectively.

(6) The first terminal device 1102, the second terminal device 1103, and the third terminal device 1104 receive the event processing result sent by the cloud mobile phone server 1101, and display the event processing result on the screen of the terminal device.

According to the multi-terminal expansion display system of the cloud mobile phone, a scene of a cloud mobile phone server can be accessed to a plurality of terminal devices, so that a part of a cloud mobile phone picture is displayed on a screen by each accessed terminal device, and the screens of the plurality of accessed terminal devices are spliced to form an entire cloud mobile phone interface, thereby realizing the capability of multi-terminal combined display of the cloud mobile phone picture, breaking through the limitation that a cloud mobile phone display area cannot be enlarged due to the size of the terminal screen, and meeting the requirement of amplifying and displaying the cloud mobile phone picture.

In order to achieve the above embodiment, the present application provides a multi-terminal expansion display device of a cloud mobile phone.

Fig. 12 is a structural block diagram of a multi-terminal expansion display device of a cloud mobile phone according to an embodiment of the present application. It should be noted that the device is applied to cloud mobile phone servers. As shown in fig. 12, the apparatus may include:

A first determining module 1210, configured to determine, in response to the first terminal device successfully accessing the cloud mobile phone server, actual location information and relative location information of a screen display area of each of the accessed terminal devices according to screen size information of the current plurality of accessed terminal devices;

The cutting module 1220 is configured to perform cutting processing on the cloud mobile phone video stream data according to the relative position information, so as to obtain partition video stream data corresponding to each accessed terminal device;

The first sending module 1230 is configured to send the partitioned video stream data and the actual location information to respective corresponding accessed terminal devices, so that each accessed terminal device displays the corresponding partitioned video stream data on a screen of the accessed terminal device based on the actual location information of the accessed terminal device.

In some embodiments of the present application, the first determining module 1210 includes:

a first determining unit 1211, configured to determine an optimal screen splicing scheme of a plurality of accessed terminal devices according to screen size information of each accessed terminal device;

a second determining unit 1212 for determining actual position information and relative position information of the screen display area of each accessed terminal device based on the optimal screen splicing scheme.

As a possible implementation manner, the first determining unit 1211 is specifically configured to:

According to the screen size information of each accessed terminal device, arranging and combining the screens of a plurality of accessed terminal devices, and determining all screen splicing schemes;

determining a maximum quadrilateral display area corresponding to each screen splicing scheme based on a preset display proportion;

determining the area of each largest quadrilateral display area;

And determining a screen splicing scheme corresponding to the largest quadrilateral display area with the largest area as an optimal screen splicing scheme.

In some embodiments of the present application, the second determining unit 1212 is specifically configured to:

determining a cloud mobile phone display area corresponding to the optimal screen splicing scheme and a screen display area of each accessed terminal device;

determining actual position information of a screen display area of each accessed terminal device;

And determining the relative position information of the screen display area of each accessed terminal device in the cloud mobile phone display area based on the optimal screen splicing scheme.

As an example, the second determination unit 1212 is also to:

determining screen direction information of each accessed terminal device in an optimal screen combination scheme;

Determining actual coordinate information of a screen display area of each accessed terminal device based on the screen direction information and the screen size information;

Screen direction information and actual coordinate information are determined as actual position information.

In some embodiments of the application, the apparatus further comprises:

A second receiving module 1240, configured to receive the operation instruction information sent by the second terminal device; the operation instruction information comprises an event type for triggering operation and first trigger position information; wherein the second terminal device is any one of a plurality of accessed terminal devices;

A second determining module 1250, configured to determine an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal device;

the second sending module 1260 is configured to process the event to be operated, obtain an event processing result, and send the event processing result to the plurality of accessed terminal devices respectively, so that each accessed terminal device displays the event processing result.

The second determining module 1250 specifically is configured to:

Determining second trigger position information corresponding to the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal device;

And determining an event to be operated according to the event type and the second trigger position information.

According to the multi-terminal expansion display device of the cloud mobile phone, after the first terminal equipment is successfully accessed, the cloud mobile phone server determines the actual position information and the relative position information of the screen display area of each accessed terminal equipment according to the screen size information of the current accessed terminal equipment; cutting the cloud mobile phone video stream data according to the relative position information to obtain partitioned video stream data corresponding to each accessed terminal device; the partitioned video stream data and the actual position information are respectively sent to the corresponding accessed terminal equipment, so that each accessed terminal equipment displays the corresponding partitioned video stream data on a screen based on the actual position information, the capability of displaying the cloud mobile phone picture by combining multiple terminals can be realized, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the terminal screen is broken through, and the requirement of amplifying and displaying the cloud mobile phone picture is met.

Fig. 13 is a block diagram of a multi-terminal expansion display device of another cloud mobile phone according to an embodiment of the present application. It should be noted that, the multi-terminal expansion display device of the cloud mobile phone in the embodiment of the application is applied to terminal equipment. As shown in fig. 13, the apparatus may include:

a first receiving module 1301, configured to receive actual position information of a screen display area of a terminal device sent by a cloud mobile phone server, and partition video stream data corresponding to the terminal device; the partitioned video stream data are obtained by cutting the cloud mobile phone video stream data by the cloud mobile phone server based on the relative position information of the screen display area of each accessed terminal device; the actual position information and the relative position information are determined by the cloud mobile phone server based on screen size information of a plurality of accessed terminal devices;

and the display module 1302 is used for displaying the partitioned video stream data on a screen of the terminal device according to the actual position information.

In some embodiments of the present application, the actual position information includes actual coordinate information and screen direction information of a screen display area of the terminal device; the display module 1302 is specifically configured to:

Determining the display direction of the partitioned video stream data according to the screen direction information;

based on the actual coordinate information and the display direction, the partitioned video stream data is displayed in a screen display area.

In some embodiments of the application, the apparatus further comprises:

The third sending module 1303 is configured to respond to a triggering operation of the partitioned video stream data, and send operation instruction information to the cloud mobile phone server based on the triggering operation; the operation instruction information comprises an event type for triggering operation and first trigger position information; the cloud mobile phone server determines an event to be operated based on the actual position information and the relative position information of a screen display area of the terminal equipment, processes the event to be operated, and determines an event processing result;

And the third receiving module 1304 is configured to receive an event processing result sent by the cloud mobile phone server, and display the event processing result on a screen of the terminal device.

According to the terminal expansion display device of the cloud mobile phone, the terminal equipment receives the actual position information of the screen display area of the terminal equipment, which is sent by the cloud mobile phone server, and the partitioned video stream data corresponding to the terminal equipment, and displays the partitioned video stream data on the screen of the terminal equipment according to the actual position information. According to the scheme, when the multi-terminal is accessed to the cloud mobile phone server, all the accessed terminal equipment displays a part of content of a cloud mobile phone interface, and a large screen is formed by splicing a plurality of terminal equipment, so that the capability of displaying the cloud mobile phone pictures in a multi-terminal combined way can be realized, the limitation that the display area of the cloud mobile phone cannot be enlarged due to the size of the terminal screen is broken through, and the requirement of amplifying and displaying the cloud mobile phone pictures is met.

It should be noted that, the explanation of the foregoing description embodiment of the multi-terminal extended display method of the cloud mobile phone is also applicable to the multi-terminal extended display device of the cloud mobile phone of the embodiment, which is not described herein again.

According to embodiments of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

Fig. 14 shows a schematic block diagram of an example electronic device 1400 that may be used to implement an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.

As shown in fig. 14, the apparatus 1400 includes a computing unit 1401 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1402 or a computer program loaded from a storage unit 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the device 1400 can also be stored. The computing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input/output (I/O) interface 1405 is also connected to the bus 1404.

Various components in device 1400 are connected to I/O interface 1405, including: an input unit 1406 such as a keyboard, a mouse, or the like; an output unit 1407 such as various types of displays, speakers, and the like; a storage unit 1408 such as a magnetic disk, an optical disk, or the like; and a communication unit 1409 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the device 1400 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 1401 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1401 performs the respective methods and processes described above, for example, a multi-terminal extended display method of a cloud cellular phone. For example, in some embodiments, the multi-terminal extended display method of a cloud handset may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 1400 via the ROM 1402 and/or the communication unit 1409. When the computer program is loaded into the RAM 1403 and executed by the computing unit 1401, one or more steps of the multi-terminal extended display method of the cloud cellular phone described above may be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform the multi-terminal extended display method of a cloud handset in any other suitable way (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, so long as the desired result of the technical solution of the present application is achieved, and the present application is not limited herein.

The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims

1. The multi-terminal expansion display method of the cloud mobile phone is characterized by being applied to a cloud mobile phone server and comprising the following steps:

2. The method of claim 1, wherein said determining actual location information and relative location information for the screen display area of each of the accessed terminal devices based on screen size information for the current plurality of accessed terminal devices comprises:

Determining an optimal screen splicing scheme of the plurality of accessed terminal devices according to the screen size information of each accessed terminal device;

And determining the actual position information and the relative position information of the screen display area of each accessed terminal device based on the optimal screen splicing scheme.

3. The method of claim 2, wherein said determining an optimal screen stitching scheme for said plurality of accessed terminal devices based on screen size information for each of said accessed terminal devices comprises:

according to the screen size information of each accessed terminal device, arranging and combining the screens of the accessed terminal devices to determine all screen splicing schemes;

Determining the area of each largest quadrilateral display area;

And determining a screen splicing scheme corresponding to the largest quadrilateral display area with the largest area as the optimal screen splicing scheme.

4. The method of claim 2, wherein the determining actual location information and relative location information of the screen display area of each of the accessed terminal devices based on the optimal screen stitching scheme comprises:

5. The method of claim 4, wherein said determining actual location information for the screen display area of each of said accessed terminal devices comprises:

determining screen direction information of each accessed terminal device in the optimal screen combination scheme;

and determining the screen direction information and the actual coordinate information as the actual position information.

6. The method as recited in claim 1, further comprising:

Receiving operation instruction information sent by second terminal equipment; the operation instruction information comprises an event type for triggering operation and first trigger position information; wherein the second terminal device is any one of the plurality of accessed terminal devices;

Determining an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal equipment;

And processing the event to be operated to obtain an event processing result, and respectively sending the event processing result to the plurality of accessed terminal devices so that each accessed terminal device displays the event processing result.

7. The method according to claim 6, wherein the determining an event to be operated according to the event type and the first trigger position information based on the actual position information and the relative position information of the screen area of the second terminal device comprises:

and determining the event to be operated according to the event type and the second trigger position information.

8. The multi-terminal expansion display method of the cloud mobile phone is characterized by being applied to terminal equipment and comprising the following steps of:

9. The method according to claim 8, wherein the actual position information includes actual coordinate information and screen direction information of a screen display area of the terminal device; the step of displaying the partitioned video stream data on a screen of the terminal device according to the actual position information comprises the following steps:

and displaying the partitioned video stream data in the screen display area based on the actual coordinate information and the display direction.

10. The method as recited in claim 8, further comprising:

Responding to the triggering operation of the partitioned video stream data, and transmitting operation instruction information to the cloud mobile phone server based on the triggering operation; the operation instruction information comprises an event type of the triggering operation and first triggering position information; the cloud mobile phone server is used for determining an event to be operated based on actual position information and relative position information of a screen display area of the terminal equipment, processing the event to be operated and determining an event processing result;

and receiving the event processing result sent by the cloud mobile phone server, and displaying the event processing result on a screen of the terminal equipment.

11. The utility model provides a cloud cell-phone's multi-terminal extension display device which characterized in that is applied to cloud cell-phone server, includes:

12. The multi-terminal expansion display device of the cloud mobile phone is characterized by being applied to terminal equipment and comprising:

13. The multi-terminal expansion display system of the cloud mobile phone is characterized by comprising:

A cloud handset server for performing the method of any of the preceding claims 1-7;

terminal device for performing the method of any of the preceding claims 8-10.

14. An electronic device, comprising:

a processor;

A memory for storing executable instructions of the processor;

Wherein the processor is configured to execute the instructions to implement the method of any one of claims 1-7 and/or to implement the method of any one of claims 8-10.

15. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of claims 1-7 and/or to perform the method of any one of claims 8-10.