WO2018107604A1 - Method and system for controlling display content in multiple display device environment and mobile terminal - Google Patents

Abstract

A method and system for controlling display content in a multiple display device environment, and a mobile terminal. The method comprises: apparatus connection (S11): selecting one among a plurality of display devices as a master apparatus and the other display devices as slave apparatuses, and connecting the slave apparatuses and the mobile terminal to the master apparatus; display device position calibration (S12): using a mobile terminal direction sensor to perform position calibration on the display devices, and storing position information of the display devices in a location associated with the display devices; projection data display (S18): the mobile terminal determining a target display device according to the position information provided by the mobile terminal direction sensor, and transmitting projection data via the master apparatus to the target display device for display. The method allows distribution of multiple pages of content to different display devices for display, and at the same time supports both static and dynamic projection.

Description

Method and system for controlling display content in multi-display device environment and mobile terminal Technical field

The present invention relates to a screen transmission technology, and in particular to a method and system for controlling display content in a multi-display device environment and a mobile terminal.

Background technique

Screening, or screencasting, as the name implies, displays the content displayed on one screen to another screen, such as displaying the content displayed on the computer screen to the TV screen, or displaying the content displayed on the screen of the mobile phone to the television. On the screen of the machine, the screen can be realized by wired means, or the screen can be realized by wireless means. There are multiple display screens in a general office or home, such as a computer screen, a mobile phone screen, a television screen, and a pad screen. These display devices can realize wireless connection with each other in the local area network, thereby displaying the content displayed on one screen. Displayed on other screens, the displayed content includes various multimedia files such as pictures, audio and video.

The current screen transmission technology can simultaneously display the content displayed on one screen to another screen or screens, but when the mobile phone has multiple pages of content that need to be screened on multiple screens for display, the current transmission Screen technology can not meet this demand; in addition, after a display screen on the mobile phone screen is projected to one or more screens, if the display screen is desired to be fixedly displayed on the one or more screens, the content display content does not change with the mobile phone end. With the change, the current screen technology can not meet this demand.

Summary of the invention

In order to solve the above technical problem, the present invention provides a control display content in a multi display device environment. The method includes the following steps:

Step S1: Device connection: select one display device as a master device among the plurality of display devices, and connect the other display devices as slave devices, the slave device and the mobile terminal to the master device;

Step S2: Proofreading display device orientation: the mobile terminal performs orientation proofreading on each display device by using the mobile terminal direction sensor, and saves the orientation information of the display device in association with the display device;

Step S3: Display screencast data: the mobile terminal determines the target display device according to the orientation information provided by the mobile terminal direction sensor, and transmits the screencast data to the target display device for display by the master device.

Further, in step S3, the target display device displays the projection screen data according to a projection mode, where the projection mode includes a static projection mode and a dynamic projection mode, and the static projection mode refers to a target display device display trigger. The screen displayed on the screen of the mobile terminal when screening, and the screen is always displayed when no new screening instruction is received; the dynamic screen mode means that the screen displayed by the target display device changes as the display screen of the mobile terminal changes.

Further, in step S2, the orientation proofing comprises the following steps:

Step S21, using the mobile terminal direction sensor to perform orientation verification on the master device, and recording the orientation data provided by the mobile terminal direction sensor, and storing the mapping relationship between the orientation data and the MAC address of the master device in the display device orientation information list; After the device orientation, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the master device;

Step S22: After receiving the orientation confirmation command, the master device sends a proofreading instruction to a slave device, and the slave device that receives the proofreading instruction feeds back the acknowledgement information to the master device.

Step S23, after receiving the receiving confirmation information fed back from the device, the master device notifies the mobile terminal to perform orientation proofreading on the slave device;

Step S24, using the mobile terminal direction sensor to perform orientation verification on the slave device, and recording the orientation data provided by the mobile terminal direction sensor, and storing the mapping relationship between the orientation data and the MAC address of the slave device in the display device orientation information list; After confirming the orientation of the slave device, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the slave device; the master device receives the orientation confirmation finger of the slave device. After that, the process returns to step S22 to continue the execution until the orientation proofing of all the slave devices is completed.

Further, the step S3 includes the following sub-steps:

Step S31: Monitoring whether there is a projection indication: the mobile terminal monitors whether there is a projection indication, and if yes, the projection operation is triggered, acquiring the orientation data of the current mobile terminal orientation sensor, and continuing to step S32; if not, repeating step S31 ;

Step S32: determining whether the trigger is valid: detecting whether the orientation data of the current mobile terminal direction sensor matches the saved display device orientation information, and if yes, the mobile terminal points to a determined target display device, and the trigger is valid, and then proceeds to step S33. If not, the mobile terminal does not point to a certain target display device, this time the trigger is invalid, returning to step S31;

Step S33, selecting a screen shooting mode;

Step S34: packaging the projection screen data: packaging the projection screen data according to the selected projection mode;

Step S35: transmitting the screencast data: the mobile terminal transmits the packaged screencast data to the master device, and then the master device forwards the packaged screencast data to the target display device;

Step S36: Display the projection screen data: after receiving the packaged projection screen data, the target display device displays the projection screen data according to the selected projection mode.

Further, in the step S32, the detecting whether the matching refers to whether the current direction sensor orientation data is within an error range of the orientation data of a display device in the display device orientation information list.

Further, in the step S3, when the static projection mode is selected, the mobile terminal intercepts the screen displayed on the screen of the mobile terminal when the screen is triggered, and the intercepted screen is used as the projection screen data, and the mobile terminal packages the intercepted screen, the header The information includes the MAC address of the target display device. When the dynamic projection mode is selected, the mobile terminal continuously encodes and packs the current screen data, and the packet header information includes the MAC address of the target display device.

The present invention further provides a system for implementing the above method, the system comprising: a mobile terminal and a plurality of display devices, wherein one of the plurality of display devices is a master device, and the other display devices are slave devices. The mobile terminal and all slave devices are connected to the master device.

The present invention also provides a mobile terminal for controlling display content in a multi-display device environment. One display device of the multi-display device is a master device, and other display devices are slave devices, and the mobile terminal and all slave devices are connected to the master device. The mobile terminal includes: a proofreading device orientation module and a transmitting module;

The proofreading device orientation module is configured to perform orientation proofreading on each display device by using a mobile terminal direction sensor, and save orientation information of all display devices associated with the display device;

The screen projection data sending module determines the target display device according to the orientation information provided by the mobile terminal direction sensor, and transmits the projection screen data to the target display device for display by the master device.

Further, the screen projection data sending module includes a monitoring module, a determining module, a selecting module, a packaging module, and a transmitting module, where

The monitoring module is configured to monitor whether there is a projection indication, and if yes, the projection operation is triggered to acquire the orientation data of the mobile terminal orientation sensor; if not, the monitoring is always performed;

The determining module is configured to detect whether the orientation data of the mobile terminal direction sensor matches the saved display device orientation information, and if yes, the mobile terminal points to a certain target display device, and the current trigger is valid; if not, the mobile terminal Did not point to a certain target display device, this trigger is invalid, notify the monitoring module to continue listening;

The selection module is configured to select a screening mode;

The packaging module is configured to package the projection screen data according to the selected screening mode;

The transmitting module is configured to transmit the packaged screencast data to the master device to forward the packaged screencast data to the target display device by the master device.

Further, the detecting whether the matching refers to whether the current direction sensor orientation data is within an error range of the orientation data of a display device in the display device orientation information list.

The beneficial effects of the invention:

Compared with the prior art, the present invention realizes distributing multiple pages of content to different display devices for display, and simultaneously supports static projection and dynamic projection.

DRAWINGS

1 is a flow chart of a method for controlling display content in a multi-display device environment proposed by the present invention;

2 is a schematic structural diagram of a system for controlling display content in a multi-display device environment according to the present invention;

FIG. 3 is a schematic structural diagram of a mobile terminal for controlling display content in a multi-display device environment according to the present invention.

detailed description

The present invention will be further described in detail below with reference to the specific embodiments of the invention. However, it is to be understood by those skilled in the art that the present invention is not limited to the drawings and the following embodiments.

The invention provides a method for controlling display content in a multi-display device environment. The flow chart is as shown in FIG. 1 and includes the following steps:

Step S11: Device connection: select one display device as the master device among the plurality of display devices, and the other display device as the slave device, the master device can open its own wireless hotspot, and the other slave devices and the mobile terminal connect to the hotspot, thereby forming a local area network composed of a mobile terminal and a plurality of display devices, the master device may obtain a list of all slave devices, the list including a MAC address of the slave device;

Step S12: Proofreading display device orientation: the mobile terminal enters a proofreading device orientation mode, and in the proofreading device orientation mode, the user performs orientation proofreading on each display device according to the prompt of the proofreading device orientation interactive interface displayed on the display screen of the mobile terminal, and Making a mapping relationship between the orientation information of all the display devices and the MAC address thereof, and storing the display device orientation information list on the mobile terminal;

The operation process of the orientation proofing is as follows:

Step S121, displaying, on the orientation interface of the proofreading device of the mobile terminal, performing orientation proofreading on the master device, and the user points the head of the mobile terminal to the orientation of the master device in a correct manner, and the orientation device module of the mobile terminal obtains the direction of the mobile terminal. The azimuth data provided by the sensor, and the mapping relationship between the acquired azimuth data and the MAC address of the master device is saved in the display device orientation information list. Here azimuth indicates the azimuth, the value is the angle between the magnetic north pole and the Y axis when the device is horizontal, the range is 0° to 360°, 0°=north, 90°=east, 180°=south, 270°=west; confirm the main After the device orientation, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the master device;

Step S122, the master device sends a proofreading instruction to a slave device, and the slave device receiving the proofreading instruction feeds back the receiving confirmation information to the master device, and prompts the user in a certain discoverable manner (for example, lighting the display screen of the slave device) , even the information such as “azimuth proofreading” may be displayed on the screen, and the receiving confirmation information includes the MAC address of the slave device;

Step S123: After receiving the receiving confirmation information fed back from the device, the master device sends the MAC address information of the slave device to the mobile terminal, and notifies the mobile terminal to perform the orientation proofreading on the slave device.

Step S124, the user points the mobile terminal head to the orientation of the slave device in a correct manner, and the calibration device orientation module of the mobile terminal acquires the azimuth data provided by the mobile terminal direction sensor, and obtains the azimuth data and the MAC address of the slave device. The mapping relationship of the address is saved in the display device orientation information list; after confirming the slave device orientation, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the slave device; the master device receives the orientation confirmation command of the slave device Thereafter, the process returns to step S122 to continue the execution until the orientation proofing of all the slave devices is completed.

Step S13: Monitor the screencasting action: the mobile terminal monitors whether the global floating button set on the orientation interface of the calibration device of the mobile terminal is clicked, and if yes, the screening operation is triggered to acquire the azimuth data of the mobile terminal direction sensor, and then proceeds to step S14. If not, repeat step S13 to monitor the screen shooting action;

Step S14: determining whether the trigger is valid: detecting whether the mobile terminal direction sensor azimuth data matches the locally saved display device orientation information list, and determining whether the matching refers to whether the value of the current direction sensor azimuth is locally stored in a display device orientation value. Within a certain error range; if yes, the mobile terminal points to a certain target display device, the trigger is valid, and the process proceeds to step S15; if not, the mobile terminal does not point to a certain target display device, and the trigger is invalid. Returning to step S13;

Step S15: Selecting a screen-casting mode: the mobile terminal enters a screen-casting selection mode, in which the user selects a screen-casting method corresponding to the screen-casting data according to the prompt of the screen-casting selection interactive interface displayed on the display screen of the mobile terminal. The screen projection method includes a static projection mode and a dynamic projection mode, and the static projection mode refers to a screen displayed on the screen of the mobile terminal when the target display device displays the trigger projection, and when a new projection indication is not received. The screen is always displayed; the dynamic screen mode means that the screen displayed by the target display device changes as the display screen of the mobile terminal changes;

Step S16: Packing the screen data: When the static screen shooting mode is selected, the mobile terminal intercepts the screen displayed on the screen of the mobile terminal when the screen is triggered, and the screen is displayed as the screen displayed on the target display device, and the screen is intercepted by the mobile terminal. The packet header information includes the MAC address of the target display device. When the dynamic screen is selected, the mobile terminal continuously encodes and packs the current screen data and transmits the data in the stream. The header information includes the MAC address of the target display device.

Step S17: transmitting the projection screen data: the mobile terminal transmits the packaged projection screen data to the main device, and after receiving the packaged projection screen data, the main device performs unpacking, and determines the transmission according to the MAC address of the target display device in the packet header information. The target display device transmits the projection data to the target display device to complete the transmission of the data packet;

Step S18: Display the projection screen data: after receiving the projection screen data, the target display device first determines whether the type of the projection mode corresponding to the projection screen data is a static projection mode or a dynamic projection mode, and then performs the projection screen data according to the projection mode. Rendering, if it is a static projection mode, directly convert the projection data into a picture for rendering; if it is a dynamic projection mode, use the android rendering control to render the interface of the projection screen data.

The present invention also provides a system for implementing the above method. As shown in FIG. 2, the system includes: a mobile terminal and a plurality of display devices, wherein one of the plurality of display devices serves as a master device, and the other display devices serve as slave devices. The device can open its own wireless hotspot, and the other slave devices and the mobile terminal connect to the hotspot, so that the mobile terminal and the plurality of display devices form a local area network, and the master device can obtain a list of all the slave devices, and the list includes the slave device. MAC address.

The present invention also provides a mobile terminal for controlling display content in a multi-display device environment, the mobile terminal and the multi-display device form a local area network, and one of the plurality of display devices has a display device as a master device, and other display devices serve as From the device, the master device can obtain a list of all slave devices, the list including the MAC address of the slave device.

As shown in FIG. 3, the mobile terminal includes: a proofreading device orientation module 31, a listening module 32, a judging module 33, a selecting module 34, a packing module 35, and a transmitting module 36.

The proofreading device orientation module 31 is configured to: provide a proofreading device orientation mode after the mobile terminal completes network interconnection with the main device, and in the proofreading device orientation mode, the user displays the proofreading device orientation interaction interface according to the display screen of the mobile terminal Prompting to perform orientation proofreading on each display device, and to make a mapping relationship between the orientation information of all the display devices and the MAC address thereof, and save the display device position information list locally.

Among them, the operation process of the orientation proofing is as follows:

In step S311, the orientation verification of the master device is performed on the orientation interface of the proofreading device of the mobile terminal, and the user points the head of the mobile terminal to the orientation of the master device in a correct manner, and the orientation module of the mobile terminal obtains the direction of the mobile terminal. The azimuth data provided by the sensor, and the mapping relationship between the acquired azimuth data and the MAC address of the master device is stored in the display device orientation information list, where the azimuth indicates the orientation, and the value is the angle between the magnetic north pole and the Y axis when the device is horizontal. The range is 0° to 360°, 0°=North, 90°=East, 180°=South, 270°=West; after confirming the orientation of the master device, the mobile terminal sends a position confirmation command to the master device to complete the orientation proofreading of the master device. ;

Step S312, the device sends a proofreading instruction to a slave device, and the slave device receiving the proofreading instruction feeds back the receiving confirmation information to the master device, and prompts the user in a certain discoverable manner (for example, lighting the display screen of the slave device, It is even possible to display information such as "azimuth proofreading" on the screen, and the receiving confirmation information includes the MAC address of the slave device;

Step S313, after receiving the receiving confirmation information fed back from the device, the master device sends the MAC address information of the slave device to the mobile terminal, and notifies the mobile terminal to perform the orientation proofreading on the slave device.

Step S314, the user points the mobile terminal head to the orientation of the slave device in a correct manner, and the calibration device orientation module of the mobile terminal acquires the azimuth data provided by the mobile terminal direction sensor, and obtains the azimuth data and the MAC address of the slave device. The mapping relationship of the address is saved in the display device orientation information list; after confirming the slave device orientation, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the slave device; the master device receives the orientation confirmation command of the slave device Thereafter, the process returns to step S122 to continue the execution until the orientation proofing of all the slave devices is completed.

The monitoring module 32 is configured to: set a global floating button on the calibration device orientation interface of the mobile terminal for the user to trigger the projection operation; the monitoring module 32 monitors whether the global floating button is clicked, and if yes, the projection operation is triggered. Obtaining the azimuth data of the mobile terminal direction sensor; if not, the screen projection action is always monitored;

The determining module 33 is configured to: detect whether the mobile terminal direction sensor azimuth data matches the locally saved display device orientation information list, and determine whether the matching refers to whether the value of the current direction sensor azimuth is a certain local display device orientation value. Within the error range; if yes, the mobile terminal points to a certain target display device, and the trigger is valid; if not, the mobile terminal does not point to a certain target display device, the trigger is invalid, and the monitoring module continues to monitor;

The selection module 34 is configured to: when the determining module determines that the trigger is valid, providing a screen selection mode, wherein in the screen selection mode, the user selects a corresponding screening mode according to the prompt of the screen selection interactive interface displayed on the display screen of the mobile terminal The screen shooting mode includes static screen shooting and dynamic screen shooting. The static screen shooting mode refers to the screen displayed by the target display device when the screen is triggered when the screen is triggered, and the screen is always displayed when a new screen shooting instruction is not received. The screen; the dynamic screen mode means that the screen displayed by the target display device changes as the display screen of the mobile terminal changes;

The packaging module 35 is configured to: when the static projection mode is selected, the mobile terminal intercepts a screen displayed on the screen of the mobile terminal when the screen is triggered, and the captured screen is used as a screen displayed on the target display device, and the packaging module 35 performs the captured image on the intercepted screen. Packing, the header information includes the MAC address of the target display device; when dynamic screening is selected, the mobile terminal continuously encodes and packs the current screen data and transmits the packet in a stream. The header information includes the MAC address of the target display device;

The transmitting module 36 is configured to: transmit the screencast data packaged by the packaging module 35 to the master device.

After receiving the packaged projection data, the main device performs unpacking, determines the transmitted target display device according to the MAC address of the target display device in the packet header information, and then transmits the projection screen data to the target display device to complete the transmission of the data packet. After receiving the projection data, the target display device first determines whether the type of the projection mode corresponding to the projection screen data is a static projection mode or a dynamic projection mode, and then renders the projection screen data according to the projection mode, if it is a static projection screen In the way, the screen data is directly converted into a picture for rendering; if it is a dynamic screen mode, the android rendering control is used to render the interface of the screen data.

The mobile terminal can be a device with an orientation proofing function such as a mobile phone or a tablet computer.

The embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiment. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims (10)

1. A method for controlling display content in a multi-display device environment, comprising the steps of:

   Step S1: Device connection: select one display device as a master device among the plurality of display devices, and connect the other display devices as slave devices, the slave device and the mobile terminal to the master device;

   Step S2: Proofreading display device orientation: the mobile terminal performs orientation proofreading on each display device by using the mobile terminal direction sensor, and saves the orientation information of the display device in association with the display device;

   Step S3: Display screencast data: the mobile terminal determines the target display device according to the orientation information provided by the mobile terminal direction sensor, and transmits the screencast data to the target display device for display by the master device.
2. The method according to claim 1, wherein in the step S3, the target display device displays the projection screen data according to a projection mode, and the projection screen mode comprises a static projection mode and a dynamic projection mode. The static projection mode refers to the screen displayed by the target display device on the screen of the mobile terminal when the screen is triggered, and the screen is always displayed when the new projection indication is not received; the dynamic projection mode refers to the display of the target display device. The screen changes as the display of the mobile terminal changes.
3. The method according to claim 1 or 2, wherein in the step S2, the orientation proofing comprises the following steps:

   Step S21, using the mobile terminal direction sensor to perform orientation verification on the master device, and recording the orientation data provided by the mobile terminal direction sensor, and storing the mapping relationship between the orientation data and the MAC address of the master device in the display device orientation information list; After the device orientation, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the master device;

   Step S22, after receiving the orientation confirmation command, the master device sends a proofreading instruction to a slave device, and receives the The slave device of the proofreading instruction feeds back the confirmation information to the master device;

   Step S23, after receiving the receiving confirmation information fed back from the device, the master device notifies the mobile terminal to perform orientation proofreading on the slave device;

   Step S24, using the mobile terminal direction sensor to perform orientation verification on the slave device, and recording the orientation data provided by the mobile terminal direction sensor, and storing the mapping relationship between the orientation data and the MAC address of the slave device in the display device orientation information list; After confirming the orientation of the slave device, the mobile terminal sends an orientation confirmation command to the master device to complete the orientation proofreading of the slave device; after receiving the orientation confirmation command of the slave device, the master device returns to step S22 to continue execution until the orientation of all the slave devices is completed. Proofreading.
4. The method according to claim 1 or 2, wherein said step S3 comprises the following substeps:

   Step S31: Monitoring whether there is a projection indication: the mobile terminal monitors whether there is a projection indication, and if yes, the projection operation is triggered, acquiring the orientation data of the current mobile terminal orientation sensor, and continuing to step S32; if not, repeating step S31 ;

   Step S32: determining whether the trigger is valid: detecting whether the orientation data of the current mobile terminal direction sensor matches the saved display device orientation information, and if yes, the mobile terminal points to a determined target display device, and the trigger is valid, and then proceeds to step S33. If not, the mobile terminal does not point to a certain target display device, this time the trigger is invalid, returning to step S31;

   Step S33, selecting a screen shooting mode;

   Step S34: packaging the projection screen data: packaging the projection screen data according to the selected projection mode;

   Step S35: transmitting the screencast data: the mobile terminal transmits the packaged screencast data to the master device, and then the master device forwards the packaged screencast data to the target display device;

   Step S36, displaying the projection screen data: after the target display device receives the packaged projection screen data, according to the location The selected projection mode displays the projection data.
5. The method according to claim 4, wherein in the step S32, the detecting whether the matching refers to whether the current direction sensor orientation data is located in an orientation data of a display device in the display device orientation information list Within the scope.
6. The method according to claim 2, wherein in the step S3, when the static projection mode is selected, the mobile terminal intercepts a screen displayed on the screen of the mobile terminal when the screen is triggered, and the intercepted screen is used as the projection screen data. The mobile terminal packages the intercepted picture, and the packet header information includes the MAC address of the target display device. When the dynamic projection mode is selected, the mobile terminal continuously encodes and packs the current screen data, and the packet header information includes the MAC address of the target display device. .
7. A system for implementing the method according to any one of claims 1 to 6, characterized in that the system comprises: a mobile terminal and a plurality of display devices, wherein one of the plurality of display devices is The master device, other display devices are slave devices, and the mobile terminal and all slave devices are connected to the master device.
8. A mobile terminal for controlling display content in a multi-display device environment, wherein one display device of the multi-display device is a master device, and other display devices are slave devices, and the mobile terminal and all slave devices are connected to the master device. The mobile terminal includes: a proofreading device orientation module and a transmitting module;

   The proofreading device orientation module is configured to perform orientation proofreading on each display device by using a mobile terminal direction sensor, and save orientation information of all display devices associated with the display device;

   The screen projection data sending module determines the target display device according to the orientation information provided by the mobile terminal direction sensor, and transmits the projection screen data to the target display device for display by the master device.
9. The mobile terminal according to claim 8, wherein the screen-casting data transmitting module comprises a monitoring module, a determining module, a selecting module, a packaging module, and a transmitting module, wherein

   The monitoring module is configured to monitor whether there is a projection indication, and if so, the projection operation is triggered, and the acquisition is performed. The orientation data of the mobile terminal direction sensor; if not, it is always monitored;

   The determining module is configured to detect whether the orientation data of the mobile terminal direction sensor matches the saved display device orientation information, and if yes, the mobile terminal points to a certain target display device, and the current trigger is valid; if not, the mobile terminal Did not point to a certain target display device, this trigger is invalid, notify the monitoring module to continue listening;

   The selection module is configured to select a screening mode;

   The packaging module is configured to package the projection screen data according to the selected screening mode;

   The transmitting module is configured to transmit the packaged screencast data to the master device to forward the packaged screencast data to the target display device by the master device.
10. The mobile terminal according to claim 9, wherein the detecting whether the matching refers to whether the current direction sensor orientation data is within an error range of the orientation data of a display device in the display device orientation information list.

Images