CN115129285A

CN115129285A - Collaborative display method and electronic equipment

Info

Publication number: CN115129285A
Application number: CN202110334007.7A
Authority: CN
Inventors: 李世明
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-03-29
Filing date: 2021-03-29
Publication date: 2022-09-30
Also published as: WO2022206566A1

Abstract

The electronic equipment adaptively adjusts a collaborative display scheme to achieve the effect of correct collaborative display without user intervention. The specific scheme is as follows: the first electronic device automatically determines position information of the second electronic device relative to the first electronic device, wherein the position information comprises an upper side, a lower side, a left side and a right side. And the first electronic equipment determines a collaborative display scheme according to the position information. And the first electronic equipment controls the first electronic equipment and the second electronic equipment to carry out cooperative display according to the cooperative display scheme.

Description

Collaborative display method and electronic equipment

Technical Field

The embodiment of the application relates to the field of electronic equipment, in particular to a collaborative display method and electronic equipment.

Background

The multi-screen collaborative technology can support collaborative display of a plurality of electronic devices, and is gradually widely used. As an example, an electronic device that controls the cooperative display is referred to as a master device, and an electronic device that performs the cooperative display under the control of the master device is referred to as an auxiliary device. The main device can set options corresponding to multi-screen coordination. The option may include a position relationship between the primary device and the secondary device, and the like. The main device can control other electronic devices (such as auxiliary devices) with display functions, which establish connection with the main device, according to the set options, so as to realize cooperative display. For example, the main device may determine the content that needs to be displayed by the main device and the auxiliary device in the cooperative display according to the position relationship with the auxiliary device. In this way, the main device can control the display screen to display according to the content required to be displayed by the main device. In addition, the main device can also indicate the auxiliary device, and the content which needs to be displayed by the auxiliary device is displayed on the display screen of the auxiliary device.

Because the position relationship between the electronic devices is set by the user through the main device, when the position relationship between the main device and the auxiliary device is inconsistent with the set position relationship, a problem of error in the cooperative display occurs. In order to correct the above problem, the user is required to reset the options corresponding to the multi-screen coordination set on the main device. Therefore, the realization of the cooperative display is complex and not intelligent enough.

Disclosure of Invention

The embodiment of the application provides a collaborative display method and electronic equipment, which can enable the electronic equipment (such as first electronic equipment) to automatically determine position information of other equipment (such as second electronic equipment) for collaborative display relative to the first electronic equipment, so that a collaborative display scheme is adaptively adjusted, and correct collaborative display is performed without user intervention.

In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

in a first aspect, a method for collaborative display is provided, the method comprising: the first electronic device automatically determines position information of the second electronic device relative to the first electronic device, wherein the position information comprises an upper side of the first electronic device, a lower side of the first electronic device, a left side of the first electronic device and a right side of the first electronic device. And the first electronic equipment determines a collaborative display scheme according to the position information. And the first electronic equipment controls the first electronic equipment and the second electronic equipment to carry out cooperative display according to the cooperative display scheme.

Based on the scheme, a collaborative display scheme of two electronic devices is provided. In this example, the first electronic device may act as a primary device and the second electronic device may act as a secondary device. For example, in the cooperative display process, the first electronic device may serve as a control center in the cooperative display process to control the screen of the first electronic device to display the corresponding content, and in addition, the first electronic device may also control the screen of the second electronic device to display the corresponding content, thereby implementing the cooperative display of the first electronic device and the second electronic device. In this embodiment, the first electronic device may determine the location of the second electronic device by itself, for example, the location of the second electronic device may be identified by a direction relative to the first electronic device. The first electronic device can determine the position information of the second electronic device without the intervention of a user, and accordingly the first electronic device and the second electronic device are controlled to carry out cooperative display. Illustratively, when the second electronic device starts cooperative display with the first electronic device, the first electronic device can determine, according to the position of the second electronic device, contents to be displayed on the two screens respectively in the cooperative display process, that is, determine a cooperative display scheme, on the premise that the user does not need to actively set the contents. Because the collaborative display scheme corresponds to the position relationship between the first electronic device and the second electronic device, the situation that collaborative display errors are caused due to the fact that the collaborative display is inconsistent with the position does not occur. It can be understood that the present solution may be applied to the above example, in a scene when the cooperative display starts, and may also be applied to a scene in which the relative positions of the first electronic device and the second electronic device change during the cooperative display. For similar reasons as in the foregoing description, it is therefore possible to ensure the accuracy of the cooperative display without requiring the user to set the related information again.

In one possible design, before the first electronic device automatically determines the location information of the second electronic device relative to the first electronic device, the method further includes: the first electronic equipment receives user operation, and the user operation is used for indicating the first electronic equipment to activate the cooperative display function. Alternatively, the operation of the first user is used to instruct the first electronic device to activate the extended display mode. Alternatively, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode. Based on the scheme, a scene example of the application of the scheme is pointed out. For example, the method can be applied to a scene that a user instructs the electronic device to enter the collaborative display mode. It is understood that, in other multi-screen display scenarios such as the extended mode and the mirror mode, in order to ensure the accuracy of the display content of each screen, the relative position relationship of each screen also needs to be determined. Therefore, the scheme of the embodiment of the invention can be applied to achieve the effect of accurate display and no need of the user to participate in setting.

In one possible design, the first electronic device has the capability of detecting the ambient magnetic field strength. The method for automatically determining the position information of the second electronic equipment relative to the first electronic equipment by the first electronic equipment comprises the following steps: the first electronic device determines the position information according to the change of the magnetic field around the first electronic device. The method comprises the steps that under the condition that the first electronic device determines that the magnetic field intensity in one direction exceeds a preset threshold value, the first electronic device determines the position information of the second electronic device relative to the first electronic device, wherein the direction comprises the upper side of the first electronic device, the lower side of the first electronic device, the left side of the first electronic device and the right side of the first electronic device. Based on the scheme, a specific example is given in which the first electronic device determines the location information of the second electronic device. For example, the first electronic device may determine the location of the second electronic device by monitoring changes in a magnetic field (e.g., changes in magnetic field strength) in the surrounding environment. It should be noted that, in this example, in order to avoid a false determination caused by a weak change of the magnetic field, the first electronic device may determine that the second electronic device is located in the corresponding direction when the magnetic field strength exceeds a preset threshold. Thereby further improving the accuracy of the determination of the position information.

In one possible design, the first electronic device is provided with a magnetic sensor for providing the first electronic device's ability to detect the ambient magnetic field strength. Based on the scheme, a specific means for detecting the magnetic field change by the electronic equipment is provided. For example, a magnetic sensor may be provided in the electronic device for monitoring the magnetic field strength.

In one possible design, the controlling, by the first electronic device, the first electronic device and the second electronic device to perform the cooperative display according to the cooperative display scheme includes: the first electronic device controls a first display screen of the first electronic device to display a first part of the content to be displayed according to the collaborative display scheme. And the first electronic equipment controls a second display screen of the second electronic equipment to display a second part of the content to be displayed according to a second cooperative display scheme, wherein the first part and the second part form the content to be displayed. Based on the scheme, a specific implementation of the collaborative display according to the collaborative display scheme is provided. In this example, the first electronic device may control the screen of the first electronic device and the screen of the second electronic device to display corresponding content according to the determined collaborative display scheme. It is to be understood that, since the collaborative display scheme corresponds to the positions of the first electronic device and the second electronic device, the position relationship of the first portion and the second portion in the content to be displayed may correspond to the positions of the first electronic device and the second electronic device. Thereby ensuring the accuracy of the collaborative display.

In one possible design, in a case where the second electronic device is located on the left side of the first electronic device, the first portion is a right portion of the content to be displayed, and the second portion is a left portion of the content to be displayed. In the case where the second electronic device is located on the right side of the first electronic device, the first portion is a left portion of the content to be displayed, and the second portion is a right portion of the content to be displayed. In a case where the second electronic device is located on an upper side of the first electronic device, the first portion is a lower portion of the content to be displayed, and the second portion is an upper portion of the content to be displayed. In a case where the second electronic device is located on the lower side of the first electronic device, the first portion is an upper portion of the content to be displayed, and the second portion is a lower portion of the content to be displayed. Based on the scheme, a corresponding relation between the position relation between the specific electronic equipment and the position relation between the collaborative display contents is provided. According to the description in this example, it can be seen that the positions of the contents displayed on different screens in the contents to be displayed correspond to the positional relationship between the electronic devices. Therefore, no matter the cooperative display is started or the position of the electronic equipment is changed in the process of the cooperative display, the cooperative display accuracy can be ensured because the content of the cooperative display is consistent with the position of the electronic equipment.

In one possible design, before the first electronic device determines the collaborative display scheme according to the location information, the method further includes: the first electronic device receives pose information of the second electronic device. The first electronic equipment determines a collaborative display scheme according to the relative position, and the collaborative display scheme comprises the following steps: and the first electronic equipment determines a collaborative display scheme according to the relative position and the posture information of the second electronic equipment. Based on the scheme, the first electronic device determines the cooperative display scheme and further realizes the cooperative display scheme. In this example, the first electronic device may further adjust the cooperatively displayed content in conjunction with the pose information of the second electronic device. Thereby further improving the accuracy of the collaborative display content.

In one possible design, the pose information includes: landscape screen attitude or portrait screen attitude. Based on this scheme, a specific example of the posture information is given. For example, based on the pose information, the first electronic device may determine that the second electronic device is a landscape or portrait screen pose. It will be appreciated that the optimal solution for displaying content is different in different poses. For example, the second electronic device may display the second content in a laterally stretched form in the landscape position, or in a vertically stretched form in the portrait position. The first electronic device can further adjust the display scheme of the second electronic device accordingly, and therefore a better collaborative display effect is obtained.

In one possible design, the first electronic device and the second electronic device are in a wirelessly connected state. Based on the scheme, an example of an implementation scenario of the scheme is given. In this example, the first electronic device may maintain a wireless connection state with the second electronic device, for example, the wireless connection state may be a collaborative display state based on the same local area network. Through the communication channel provided by the wireless connection state, the first electronic device may interact with the second electronic device, for example, control the second electronic device to perform cooperative display according to a cooperative display scheme, and for example, acquire the posture information of the second electronic device from the second electronic device.

In a possible design, before the first electronic device controls the first electronic device and the second electronic device to perform the cooperative display according to the cooperative display scheme, the position information of the first electronic device relative to the second electronic device is the first position information. And the first electronic equipment and the second electronic equipment perform first cooperative display according to the first position information and a first cooperative display scheme. The first location information corresponds to a first collaborative display scheme. When the first position information is changed into the second position information, the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, and the method comprises the following steps: the first electronic device automatically determines that the position information of the second electronic device relative to the first electronic device is second position information. The first electronic equipment determines a collaborative display scheme according to the position information, and the collaborative display scheme comprises the following steps: and the first electronic equipment determines a second collaborative display scheme according to the second position information. The first electronic device controls the first electronic device and the second electronic device to perform cooperative display according to the cooperative display scheme, and the method comprises the following steps: and the first electronic equipment controls the first electronic equipment and the second electronic equipment to be switched from the first collaborative display to the second collaborative display according to the second collaborative display scheme, wherein the second collaborative display is a display corresponding to the second collaborative display scheme. Based on the scheme, a concrete implementation scene example of the scheme is provided. In this example, the positional relationship of the first electronic device and the second electronic device may not be fixed. In the process of performing the collaborative display, the first electronic device may continuously/intermittently adjust the collaborative display scheme according to the location of the second electronic device, so that when the location of the second electronic device changes (e.g., changes from the first location information to the second location information), the first electronic device may adaptively adjust the collaborative display scheme, and control the screen of the first electronic device and the screen of the second electronic device to display the content corresponding to the current location information.

In a second aspect, a collaborative display apparatus is provided, which may be disposed in a first electronic device, and includes: the determining unit is used for automatically determining the position information of the second electronic equipment relative to the first electronic equipment, and the position information comprises an upper side, a lower side, a left side and a right side. And the determining unit is also used for determining the cooperative display scheme according to the position information. And the control unit is used for controlling the first electronic equipment and the second electronic equipment to carry out cooperative display according to the cooperative display scheme.

In one possible design, the apparatus further includes: the receiving unit is used for receiving user operation before the first electronic equipment automatically determines the position information of the second electronic equipment relative to the first electronic equipment, and the user operation is used for indicating the first electronic equipment to activate the cooperative display function. Alternatively, the operation of the first user is to instruct the first electronic device to activate the extended display mode. Alternatively, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode.

In one possible design, the first electronic device has the capability of detecting the ambient magnetic field strength. And the determining unit is specifically used for determining the position information according to the change of the peripheral magnetic field of the first electronic equipment. The determining unit determines the position information of the second electronic device at the first electronic device under the condition that the magnetic field intensity in one direction exceeds a preset threshold, wherein the direction comprises an upper direction, a lower direction, a left direction and a right direction.

In one possible design, the first electronic device is provided with a magnetic sensor for providing the first electronic device's ability to detect the ambient magnetic field strength.

In one possible design, the control unit is configured to control, by the first electronic device, the first display screen of the first electronic device to display the first portion of the content to be displayed according to the collaborative display scheme. And the control unit is further used for controlling a second display screen of the second electronic device to display a second part of the content to be displayed according to the second collaborative display scheme, and the first part and the second part form the content to be displayed.

In one possible design, in a case where the second electronic device is located on the left side of the first electronic device, the first portion is a right portion of the content to be displayed, and the second portion is a left portion of the content to be displayed. In the case where the second electronic device is located on the right side of the first electronic device, the first portion is a left portion of the content to be displayed, and the second portion is a right portion of the content to be displayed. In a case where the second electronic device is located on an upper side of the first electronic device, the first portion is a lower portion of the content to be displayed, and the second portion is an upper portion of the content to be displayed. In the case where the second electronic device is located on the lower side of the first electronic device, the first portion is an upper portion of the content to be displayed, and the second portion is a lower portion of the content to be displayed.

In a possible design, the receiving unit is further configured to receive posture information of the second electronic device before the first electronic device determines the collaborative display scheme according to the position information. And the determining unit is used for determining the collaborative display scheme according to the relative position and the posture information of the second electronic equipment.

In one possible design, the pose information includes: landscape screen attitude or portrait screen attitude.

In one possible design, the first electronic device and the second electronic device are in a wireless connection state.

In one possible design, before the control unit controls the first electronic device and the second electronic device to perform the cooperative display according to the cooperative display scheme, the position information of the first electronic device relative to the second electronic device is the first position information. And the control unit is used for controlling the first electronic equipment to perform first collaborative display according to the first collaborative display scheme according to the first position information. The first location information corresponds to a first collaborative display scheme. And the determining unit is used for automatically determining that the position information of the second electronic equipment relative to the first electronic equipment is the second position information under the condition that the first position information is changed into the second position information. And the determining unit is further used for determining a second collaborative display scheme according to the second position information. And the control unit is used for controlling the first electronic equipment and the second electronic equipment to be switched from the first collaborative display to the second collaborative display according to the second collaborative display scheme, wherein the second collaborative display is a display corresponding to the second collaborative display scheme.

In a third aspect, an electronic device is provided that includes one or more processors and one or more memories. The electronic device may be, for example, the first electronic device referred to in the first aspect or the second aspect. One or more memories are coupled to the one or more processors, the one or more memories storing computer instructions. The computer instructions, when executed by the one or more processors, cause the electronic device to perform the method of collaborative display as described in any one of the first aspect and any one of its possible designs.

In a fourth aspect, a system-on-chip is provided that includes an interface circuit and a processor. The chip system may be applied to the first electronic device referred to in the first aspect or the second aspect. The interface circuit and the processor are interconnected through a line; the interface circuit is used for receiving signals from the memory and sending signals to the processor, and the signals comprise computer instructions stored in the memory; when the processor executes the computer instructions, the system-on-chip performs the method of collaborative display as described in any one of the first aspect and any one of its possible designs.

In a fifth aspect, there is provided a computer readable storage medium comprising computer instructions which, when executed, perform the method of collaborative display according to any one of the first aspect and any one of its possible designs.

A sixth aspect provides a computer program product comprising instructions which, when run on a computer, enable the computer to perform the method according to any one of the first aspect and any one of its possible designs.

It should be understood that, in the technical solutions provided in the second aspect, the third aspect, the fourth aspect, the fifth aspect and the sixth aspect, technical features thereof may all correspond to the cooperative display method provided in the first aspect and possible designs thereof, and therefore beneficial effects that can be achieved are similar and will not be described herein again.

Drawings

FIG. 1 is a schematic illustration of a collaborative display;

FIG. 2 is a schematic view of a cooperative display arrangement;

FIG. 3 is a schematic diagram of the effect of a cooperative display;

FIG. 4 is a schematic diagram illustrating an effect of another cooperative display;

fig. 5 is a schematic composition diagram of an electronic device according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of position information determination provided in an embodiment of the present application;

fig. 7 is a schematic flowchart of a collaborative display method according to an embodiment of the present application;

fig. 8 is a schematic effect diagram of a cooperative display method according to an embodiment of the present application;

fig. 9 is a schematic flowchart of another collaborative display method according to an embodiment of the present application;

fig. 10 is a schematic view of a collaborative display scenario provided in an embodiment of the present application;

fig. 11 is a schematic effect diagram of another collaborative display method according to an embodiment of the present application;

fig. 12 is a schematic composition diagram of a cooperative display apparatus according to an embodiment of the present disclosure;

fig. 13 is a schematic flowchart of another shooting method provided in the embodiment of the present application;

fig. 14 is a schematic composition diagram of a chip system according to an embodiment of the present disclosure.

Detailed Description

The multi-screen cooperation technology plays an increasingly important role in mobile office, smart home and other scenes. Through multi-screen cooperation technology, the interactive interface of the user and the electronic equipment can be enlarged, and the interactive experience is enriched. Meanwhile, the multi-screen cooperation technology can also make the electronic equipment more intelligent by means of the sensing capability among the electronic equipment, so that the effects of simplifying user operation and improving productivity are achieved.

A scheme of cooperative display is illustrated below with reference to fig. 1-4.

Generally, when performing a collaborative display using a multi-screen collaborative technique, a user may set an option of the collaborative display on a main device. Illustratively, the main device is a notebook computer as shown in fig. 1, and the auxiliary device is a tablet computer. Referring to fig. 1, on a collaborative display setup interface of a notebook computer, interface elements as shown at 101 may be displayed. The interface element may be used to prompt a user for a collaborative display using an extended mode. The user can select the expansion mode through a control device such as a mouse and the like, so that the notebook computer enters a cooperative display state.

It should be noted that, in some embodiments, as shown in fig. 1, before the laptop displays the collaborative display setting interface, the laptop may be interconnected with other electronic devices capable of collaborative display, and through the interconnection communication channel, the laptop may be collaboratively displayed with other electronic devices (such as a tablet computer, etc.). For example, after the laptop and the tablet computer are successfully interconnected, an electronic device (such as device X) that has been connected to the laptop may be displayed on the cooperative display setup interface.

In the above example, the user inputs an operation (for example, an operation of selecting an extension mode) to the notebook computer through the control device. In other implementations of this example, for a notebook computer that supports touch operations, the user may also input the operations in the form of touch. It is understood that the user may also input operations by voice control or the like. The following description will be given taking an example in which a user inputs a control command to an electronic device through a mouse.

In this example, the host device (e.g., a laptop) may display other collaborative display setup interfaces (e.g., referred to as display parameter setup interfaces) after receiving an operation of the user selecting the extended mode. Illustratively, referring to fig. 2, the display parameter setting interface may include a "tablet screen position" option for setting the relative position of the tablet computer and the notebook computer, as well as display setting options for other auxiliary devices (such as the "tablet sidebar position" option shown in fig. 2).

As shown in fig. 2, the user may select a corresponding position relationship in the "tablet screen position" option according to the relative position of the tablet computer and the notebook computer when performing the cooperative display. The main device (such as a notebook computer) can determine the content required to be displayed by the main device and the content required to be displayed by the auxiliary device (such as a tablet computer) according to the position relation selected by the user. In this embodiment of the application, the position relationship between the main device and the auxiliary device may include that the auxiliary device is located on the left side of the main device, the auxiliary device is located on the right side of the main device, the auxiliary device is located on the upper side of the main device, the auxiliary device is located on the lower side of the main device, and the like.

Illustratively, a tablet computer is placed on the right side of a notebook computer. The user can select a corresponding position relation on the notebook computer by using the mouse to select a corresponding option on the interface (for example, select an option 201 shown in fig. 2). The notebook computer can receive the operation, and according to the operation, the notebook computer can determine that the tablet computer is positioned on the right side of the notebook computer in the cooperative display process. Therefore, the notebook computer can determine the content required to be displayed by the screen of the notebook computer and the content required to be displayed by the tablet computer in the collaborative display process.

In the embodiment of the present application, a user may activate a cooperative display function of an electronic device (e.g., a notebook and/or a tablet computer) through the operations shown in fig. 1 and fig. 2.

As an example, extended display of a picture by cooperative display is taken as an example. In connection with fig. 3. The display of the picture on the notebook computer is shown as (a) in fig. 3. By the cooperative display, a host device (such as a notebook computer) can display a part of an image of a picture on its screen. The primary device may control a secondary device (e.g., a tablet computer) to display another portion of the picture on its screen, thereby achieving the effect of expanding the display area of the displayed picture. With reference to the descriptions in fig. 1 and fig. 2, when the tablet computer is located on the right side of the notebook computer, the notebook computer can determine the position relationship of the tablet computer located on the right side of the notebook computer for performing the cooperative display by the arrangement shown in fig. 1 and fig. 2. For example, a notebook computer may display a left partial image of a picture on its screen. The notebook computer can also control the tablet computer to display a right partial image of the picture on its screen. Please refer to fig. 3 (b) for displaying the result. Thereby, the extended display of the picture through the cooperative display on the two electronic devices is realized.

However, the scheme in the above example also has some problems.

It can be understood that the position relationship of the main device and the auxiliary device in the real environment must be consistent with the selected position relationship in the collaborative display setting process, and the main device can know the position of the auxiliary device in the real environment, so as to determine the content that needs to be displayed by the main device and the auxiliary device in the collaborative display process. However, when the position relationship between the main device and the auxiliary device in the real environment is not consistent with the selected position relationship in the cooperative display setting process, or the position relationship between the main device and the auxiliary device in the cooperative display process changes, the main device cannot accurately know the position relationship between the main device and the auxiliary device, and thus the content to be displayed by the main device and the auxiliary device cannot be accurately determined.

For example, the primary device is a device a shown in fig. 4, and the secondary device is a device B shown in fig. 4. Refer to (a) in fig. 4. During the coordinated display, the device B may move in position. For example, device B may move from the right side of device a to the left side of device a. Thus, if the user does not reset the option of the cooperative display, the device a determines whether the device B is located on the right side of the device a according to the previous setting. The device A can continue to control the device A and the device B to perform cooperative display according to the information. Thus, the situation as in (b) of fig. 4 may occur. I.e. the device B on the left displays the right partial image of the picture, while the device a on the right actually displays the left partial image of the picture. This obviously does not satisfy the user's needs for viewing or editing pictures.

In order to solve the problems in the foregoing examples, the collaborative display scheme provided in the embodiments of the present application enables an electronic device to automatically determine the positions of other devices in a collaborative display process, and perform collaborative display according to the positions. For example, when the collaborative display scheme provided by the embodiment of the present application is used on the primary device, the primary device may automatically determine the position of the secondary device, and determine the content that needs to be displayed by the primary device and the secondary device according to the position. Therefore, the user does not need to actively set the option of the collaborative display, but the main device can automatically determine the position of the auxiliary device, so that the problem of errors of collaborative display contents caused by manual setting of the user or changes of the position relationship between the main device and the auxiliary device is avoided.

The following describes in detail a collaborative display scheme provided by an embodiment of the present application with reference to the drawings.

Please refer to fig. 5, which is a schematic diagram illustrating an electronic device 500 according to an embodiment of the present disclosure. In the embodiment of the present application, the electronic device 500 may be used for performing a collaborative display. For example, the electronic device 500 may be cooperatively displayed as a master device. For another example, the electronic device 500 may also be used as an auxiliary device to perform a cooperative display.

It should be noted that the scheme provided by the embodiment of the present application may be applied to electronic devices of users. The electronic device may be a portable mobile device which is provided with a display screen and can perform cooperative display, such as a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), an Augmented Reality (AR) \ Virtual Reality (VR) device, a media player, and the like, or may be a wearable electronic device which is provided with a display screen and can perform cooperative display, such as a smart watch, and the like. The embodiment of the present application does not particularly limit the specific form of the electronic device.

As shown in fig. 5, the electronic device may include a processor 510, an external memory interface 520, an internal memory 521, a Universal Serial Bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a speaker 570A, a receiver 570B, a microphone 570C, a headset interface 570D, a sensor module 580, a button 590, a motor 591, an indicator 592, a camera 593, a display 594, a Subscriber Identification Module (SIM) card interface 595, and the like.

It is to be understood that the illustrated structure of the present embodiment does not specifically limit the electronic device 500. In other embodiments, the electronic device 500 may include more or fewer components than illustrated, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 510 may include one or more processing units, such as: the processor 510 may include an Application Processor (AP), a modem processor, a graphics processor 510 (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), among others. Wherein the various processing units may be stand-alone devices or may be integrated within one or more of the processors 510.

The controller may be a neural center and a command center of the electronic device 500. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 510 for storing instructions and data. In some embodiments, the memory in processor 510 is a cache memory. The memory may hold instructions or data that have just been used or recycled by processor 510. If the processor 510 needs to use the instruction or data again, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 510, thereby increasing the efficiency of the system.

In the embodiment of the present application, the processor 510 may store an instruction or an instruction set corresponding to the collaborative display scheme provided in the embodiment of the present application. Illustratively, the processor 510 is provided in the master device. When the cooperative display scheme provided in the embodiment of the present application is implemented, the processor 510 may determine the position of the auxiliary device according to the change of the peripheral magnetic field strength. Processor 510 may also be configured to determine the content to be displayed by each of the primary device and the secondary device based on the location of the secondary device.

In some embodiments, processor 510 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver Transmitter (UART) interface, a mobile industry processor 510 interface (MIPI), a General Purpose Input Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The electronic device 500 may implement a capture function via the ISP, the camera 593, the video codec, the GPU, the display screen 594, and the application processor, etc.

The ISP is used to process the data fed back by the camera 593. For example, when taking a picture, the shutter is opened, light is transmitted to the photosensitive element of the camera 593 through the lens, the optical signal is converted into an electrical signal, and the photosensitive element of the camera 593 transmits the electrical signal to the ISP for processing and converting into an image visible to the naked eye. The ISP can also carry out algorithm optimization on noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 593.

The camera 593 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 500 may include 1 or N cameras 593, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the electronic device 500 selects at a frequency bin, the digital signal processor is used to perform a fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The electronic device 500 may support one or more video codecs. In this way, the electronic device 500 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor 510 that processes input information quickly by referencing a biological neural network structure, for example, by referencing a transfer pattern between neurons in the human brain, and may also learn itself continuously. Applications such as intelligent recognition of the electronic device 500 can be realized through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The charging management module 540 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 540 may receive charging input from a wired charger via the USB interface 530. In some wireless charging embodiments, the charging management module 540 may receive the wireless charging input through a wireless charging coil of the electronic device 500. The charging management module 540 may also supply power to the electronic device 500 through the power management module 541 while charging the battery 542.

The power management module 541 is used to connect the battery 542, the charging management module 540 and the processor 510. The power management module 541 receives input from the battery 542 and/or the charging management module 540, and provides power to the processor 510, the internal memory 521, the external memory, the display 594, the camera 593, the wireless communication module 560, and the like. The power management module 541 may also be used to monitor parameters such as the capacity of the battery 542, the number of cycles of the battery 542, and the state of health (leakage, impedance) of the battery 542. In some other embodiments, the power management module 541 may also be disposed in the processor 510. In other embodiments, the power management module 541 and the charging management module 540 may be disposed in the same device.

The wireless communication function of the electronic device 500 may be implemented by the antenna 1, the antenna 2, the mobile communication module 550, the wireless communication module 560, the modem processor, the baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 500 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 550 may provide a solution including 2G/3G/4G/5G wireless communication, etc. applied on the electronic device 500. The mobile communication module 550 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 550 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 550 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 550 may be disposed in the processor 510. In some embodiments, at least some of the functional modules of the mobile communication module 550 may be disposed in the same device as at least some of the modules of the processor 510.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 570A, the receiver 570B, etc.) or displays images or videos through the display screen 594. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be separate from the processor 510, and may be located in the same device as the mobile communication module 550 or other functional modules.

The wireless communication module 560 may provide a solution for wireless communication applied to the electronic device 500, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 560 may be one or more devices integrating at least one communication processing module. The wireless communication module 560 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 510. The wireless communication module 560 may also receive a signal to be transmitted from the processor 510, frequency-modulate and amplify the signal, and convert the signal into electromagnetic waves via the antenna 2 to radiate the electromagnetic waves.

In some embodiments, antenna 1 of the electronic device 500 is coupled to the mobile communication module 550 and antenna 2 is coupled to the wireless communication module 560 such that the electronic device 500 may communicate with networks and other devices via wireless communication techniques. The wireless communication technologies may include, but are not limited to, GSM, GPRS, CDMA, WCDMA, TD-SCDMA, LTE, 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, among others.

In some embodiments of the present application, the mobile communication module 550 and/or the wireless communication module 560 in the electronic device 500 may be used to establish an interconnection with other electronic devices under the control of the processor 510. For example, the electronic device 500 is taken as a main device. The electronic device 500 may join the same local area network as the secondary device through the mobile communication module 550 and/or the wireless communication module 560. The electronic device 500 may communicate with the secondary device via the local area network. For example, the electronic device 500 may send the display information corresponding to the auxiliary device through the local area network. And the auxiliary equipment is convenient to perform cooperative display according to the received display information.

The electronic device 500 implements display functions via the GPU, the display screen 594, and the application processor. The GPU is an image processing microprocessor 510 coupled to a display screen 594 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 510 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 594 is used for displaying images, video, and the like. In some embodiments, the electronic device 500 may include 1 or N display screens 594, N being a positive integer greater than 1.

In this embodiment, the main device and the auxiliary device may be provided with a display screen 594. In some scenarios, the display screen of the primary device may be displayed in cooperation with the display screen of the secondary device.

The external memory interface 520 may be used to connect an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 500. The external memory card communicates with the processor 510 through the external memory interface 520 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 521 may be used to store computer-executable program code, including instructions. The processor 510 executes various functional applications of the electronic device 500 and data processing by executing instructions stored in the internal memory 521. The internal memory 521 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, and the like) required by at least one function, and the like. The storage data area may store data (e.g., audio data, phone book, etc.) created during use of the electronic device 500, and the like. In addition, the internal memory 521 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a general flash memory, and the like.

The electronic device 500 may implement audio functions through the audio module 570, the speaker 570A, the receiver 570B, the microphone 570C, the headphone interface 570D, and the application processor, etc. Such as music playing, recording, etc.

The audio module 570 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 570 may also be used to encode and decode audio signals. In some embodiments, the audio module 570 may be disposed in the processor 510, or some functional modules of the audio module 570 may be disposed in the processor 510.

The speaker 570A, also called a "horn", is used to convert the audio electrical signals into sound signals. The electronic device 500 may listen to music through the speaker 570A or listen to a hands-free conversation.

Receiver 570B, also known as a "handset," is used to convert electrical audio signals into acoustic signals. When the electronic apparatus 500 receives a call or voice information, it is possible to receive voice by placing the receiver 570B close to the human ear.

The microphone 570C, also known as a "microphone," is used to convert sound signals into electrical signals. The earphone interface 570D is used to connect a wired earphone.

The sensor module 580 provided in the electronic device 500 provided in the embodiments of the present application may include one or more sensors of the same or different kinds. For example, the sensor module 580 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

Take the example of a magnetic sensor included in the sensor module 580. The magnetic sensor may comprise a hall sensor. The hall sensor may determine the change in the magnetic field around the electronic device 500 based on the hall effect. Among them, the hall effect is one of electromagnetic effects, which is found by hall of american physicist. According to the hall effect, when a current passes through a semiconductor perpendicular to an external magnetic field, carriers are transmitted to be offset, and an additional electric field is generated perpendicular to the direction of the current and the magnetic field, thereby generating a potential difference across the semiconductor. This potential difference is also referred to as hall potential difference.

In an embodiment of the present application, the electronic device 500 may determine the change in the ambient magnetic field from a hall sensor. It will be appreciated that one or more magnetic materials may be provided in the electronic device. Thus, when the slave device approaches the master device, the magnetic material on the slave device affects the magnetic field around the master device. When the electronic device 500 is used as a master device, the direction in which the slave device approaches the master device can be determined according to the change in the magnetic field, and thus the relative positions of the master device and the slave device can be determined.

In other embodiments of the present application, the electronic device 500 may detect the opening and closing of the flip holster using a magnetic sensor. In some embodiments, when the electronic device 500 is a flip phone, the electronic device 500 may detect the opening and closing of the flip according to the magnetic sensor. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

In other embodiments of the present application, in the case where the electronic device 500 is configured with a foldable screen, the electronic device 500 may detect an opening and closing state (e.g., an unfolded state and a closed state) of the foldable screen by using a magnetic sensor. The electronic device 500 may implement automatic control of functions such as automatic unlocking and volume adjustment according to the detected opening/closing state of the foldable screen.

Touch sensors, also known as "touch panels". The touch sensor may be disposed on the display screen 594, and the touch sensor and the display screen 594 form a touch screen, which is also called a "touch screen". The touch sensor is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. In some embodiments, visual output related to touch operations may be provided through the display screen 594. In other embodiments, the touch sensor may be disposed on a surface of the electronic device 500 at a different location than the display screen 594.

It should be noted that in some embodiments of the present application, the sensor module 580 in the electronic device 500 may be integrated together to implement the functions of the above-mentioned components, or may be partially integrated or separately provided to implement the sensing functions thereof. For example, in some embodiments, an Inertial Measurement Unit (IMU) may be disposed in electronic device 500. The IMU may be used to implement a 6-axis or 9-axis sensing function. Illustratively, a 9-axis test by the IMU is taken as an example. The IMU can be used to implement the sensing functions of components such as acceleration sensors, gyroscope sensors, hall sensors, and the like.

The keys 590 include a power-on key, a volume key, etc. The electronic device 500 may receive key 590 inputs, generating key signal inputs related to user settings and function control of the electronic device 500.

The motor 591 may generate a vibration indication. Indicator 592 can be an indicator light that can be used to indicate a charge status, a charge change, a message, a missed call, a notification, etc. The SIM card interface 595 is used to connect a SIM card.

The collaborative display scheme provided by the embodiment of the present application can be used in the electronic device 500 shown in fig. 5.

The scheme provided by the embodiment of the application is described in detail below with reference to the accompanying drawings.

Generally, an electronic device is provided with one or more magnetic materials therein in order to realize its functions. For example, the electronic device is a tablet computer. The usage mode of the tablet computer may include a holster mode. In the leather sheath mode, the tablet personal computer can execute corresponding strategies such as screen lighting/screen turning and the like according to the opening and closing state of the leather sheath. In order to enable the leather sheath to be tightly attached to the tablet personal computer, the magnetic material can be arranged in the tablet personal computer, so that when the leather sheath is closed, the magnetic material in the leather sheath and the magnetic material in the tablet personal computer can attract each other, and therefore the effect of tight closing is achieved. Of course, in other electronic devices, the magnetic material may be provided according to its function even if the electronic device does not support the holster mode.

When the magnetic material enters the magnetic field, it affects the magnetic field. For example, the magnetic field strength at the corresponding position of the magnetic material may change. The change in the magnetic field strength can be captured by the hall sensor, thereby allowing the electronic device in which the hall sensor is disposed to determine the orientation of the magnetic material.

Exemplarily, in connection with fig. 6. Take the example where the device 1 is provided with a sensor (e.g., a hall sensor) capable of detecting a change in magnetic field and the device 2 is provided with a magnetic material. The device 1 can acquire the distribution of the surrounding magnetic field through a sensor. When the device 2 approaches the device 1 in the negative X-axis direction, a change in the magnetic field strength of the device 1 in the positive X-axis direction is then caused. The device 1 can acquire this change in magnetic field strength by means of a sensor. From this, the device 1 knows that the device 2 is approaching the device 1 in the negative X-axis direction, that is, the device 2 is located on the right side of the device 1.

In the solution provided in the embodiment of the present application, the main device may determine the position of the auxiliary device close to the main device according to the technical means in the above description. Then, the main device can determine the contents to be displayed by the main device and the auxiliary device respectively in the cooperative display according to the position of the auxiliary device. And controlling the auxiliary equipment to display corresponding content on the screen of the main equipment, thereby realizing the cooperative display of the main equipment and the auxiliary equipment.

It can be understood that, since the main device can automatically identify the location of the auxiliary device according to its internal components, the user is not required to manually set the location option of the auxiliary device during the collaborative display (for example, the corresponding option of the collaborative display is set according to the scheme shown in fig. 1-2). Therefore, the problem of collaborative display errors caused by the fact that the position options set by the user are inconsistent with the positions of the main device and the auxiliary device in the actual scene can be solved, and meanwhile, the electronic device can provide more convenient collaborative display capability.

As an example, please refer to fig. 7, which is a flowchart illustrating a collaborative display method according to an embodiment of the present application. For example, the main device is the device 1, the auxiliary device is the device 2, the display screen of the main device is the screen 1, and the display screen of the auxiliary device is the screen 2. As shown in fig. 7, the scheme may include:

s701, the device 1 determines the position information of the device 2 relative to the device 1 according to the magnetic field change.

Wherein the position information of the device 2 relative to the device 1 may comprise one of an upper side of the device 1, a lower side of the device 1, a left side of the device 1, and a right side of the device 1.

In some embodiments, the device 1 may determine position information of the device 2 relative to the device 1 when the magnetic field strength in one direction reaches (or exceeds) a preset threshold (such as a first threshold).

For example, the device 1 may determine that the position information of the device 2 with respect to the device 1 at this time when it is determined that the magnetic field strength in the positive X-axis direction reaches the first threshold value: the right side of the device 1. That is, the device 1 may determine that the device 2 is located on the right side of the device 1. The device 1 may, upon determining that the magnetic field strength in the negative X-axis direction reaches a first threshold value, determine the position information of the device 2 relative to the device 1 at that time as: the left side of the device 1. The device 1 may determine that the position information of the device 2 with respect to the device 1 at this time when it is determined that the magnetic field strength in the positive direction of the Y axis reaches the first threshold value is: the upper side of the device 1. The device 1 may determine, when it is determined that the magnetic field strength in the negative Y-axis direction reaches the first threshold value, that the position information of the device 2 with respect to the device 1 at this time is: the underside of the device 1.

In other embodiments, the device 1 may determine the position information of the device 2 when the value of the change in the magnetic field strength in one direction reaches a corresponding further predetermined threshold value (e.g. a second threshold value).

For example, the device 1 may determine that the device 2 is located on the right side of the device 1 at this time when it is determined that the value of the magnetic-field-intensity change in the positive X-axis direction reaches the second threshold value. The device 1 may determine that the device 2 is located on the left side of the device 1 at this time when it is determined that the value of the magnetic-field-strength change in the negative X-axis direction reaches the second threshold value. The device 1 may determine that the device 2 is located on the upper side of the device 1 at this time when it is determined that the value of the magnetic-field-intensity change in the positive direction of the Y axis reaches the second threshold value. The device 1 may determine that the device 2 is located on the lower side of the device 1 at this time when it is determined that the value of the magnetic-field-intensity change in the negative Y-axis direction reaches the second threshold value.

S702, the device 1 determines a collaborative display scheme according to the position information of the device 2 relative to the device 1.

In the embodiment of the present application, the cooperative display scheme may also be simply referred to as a display scheme. The display scheme may include a display scheme 1 and a display scheme 2. Wherein, the display scheme 1 may be a scheme in which the screen 1 on the device 1 is displayed during the cooperative display. The display scenario 2 may be a scenario in which the screen 2 on the device 2 is displayed during the cooperative display.

In this example, the device 1 may determine the display scheme 1 from the position information of the device 2 with respect to the device 1.

Illustratively, the device 1 may determine the content to be displayed on the screen 1 of the device 1, i.e., determine the display scheme 1 on the screen 1, after determining the position information of the device 2 with respect to the device 1.

Take the example that the device 1 and the device 2 perform the cooperative display of pictures. Device 1 may determine that a left partial image of a picture needs to be displayed in display scheme 1 when device 2 is determined to be on the right side of device 1. Device 1 may determine that a right partial image of a picture needs to be displayed in display scheme 1 when device 2 is determined to be on the left side of device 1. Device 1 may determine that a lower partial image of a picture needs to be displayed in display scheme 1 when device 2 is determined to be on the upper side of device 1. The device 1 may determine that an upper partial image of the picture needs to be displayed in the display scheme 1 when the device 2 is determined to be on the lower side of the device 1.

Furthermore, the device 1 may also determine the display scheme 2 based on position information of the device 2 relative to the device 1.

It will be appreciated that in most collaborative display schemes, the device 2 acts as a secondary device, and the content that it needs to display is determined by the primary device and controlled by the device 2 for display. Therefore, in this embodiment, in conjunction with the foregoing description, the device 1 can also determine what the device 2 needs to display, according to the position information of the device 2 relative to the device 1.

Illustratively, the example of the cooperative display of pictures by the device 1 and the device 2 is continued. Device 1 may determine that a right partial image of the picture needs to be displayed in display scheme 2 when device 2 is determined to be on the right side of device 1. Device 1 may determine that a left partial image of the picture needs to be displayed in display scheme 2 when device 2 is determined to be on the left side of device 1. The device 1 may determine that an upper partial image of the picture needs to be displayed in the display scheme 2 when it is determined that the device 2 is on the upper side of the device 1. The device 1 may determine that the lower part image of the picture needs to be displayed in the display scheme 2 when it is determined that the device 2 is on the lower side of the device 1.

In this way, an effect is achieved that the device 1 adaptively adjusts the display scheme on the device 1 (e.g., display scheme 1) and the display scheme on the device 2 (e.g., display scheme 2) according to the position information of the device 2 with respect to the device 1.

It should be noted that, in different implementations of the embodiment of the present application, the device 1 may determine that the operations of the display scheme 1 and the display scheme 2 may have different orders. For example, in some implementations, device 1 may determine display scheme 2 after determining display scheme 1, thereby avoiding the focused consumption of computing power. In other implementations, the device 1 may determine the display scheme 1 after determining the display scheme 2, and thus the device 1 may control the device 2 to prepare for display according to the display scheme 2 while determining the display scheme 1, so as to achieve the effect of parallel processing and improve the synchronization of the display contents of the two screens in the cooperative display process. It will be appreciated that in other implementations, the device 1 may also determine the display scheme 2 at the same time as the display scheme 1, thereby also saving time consumption of data processing in the collaborative display process through parallel processing.

And S703, the device 1 controls the screen 1 of the device 1 and the screen 2 of the device 2 to perform cooperative display according to the cooperative display scheme.

For example, the device 1 may send the display scheme 2 to the device 2 after determining the display scheme 2 corresponding to the device 2, so that the device 2 performs display according to the display scheme 2.

In this example, after device 1 determines display scheme 2 for device 2, device 1 may send display scheme 2 for device 2 to device 2 through an interconnect channel with device 2. So that the device 2 performs the cooperative display according to the display scheme 2 transmitted by the device 1.

Furthermore, the device 1 may also control the screen 1 to perform a corresponding display according to the display scheme 1.

It is noted that, in the present example, since the device 1 can control the display of the screen 1 according to the display scheme 1 by internal control. The control of the display of the device 2 by the device 1 requires communication with the device 2 in order to control the device 2 to perform the corresponding cooperative display according to the display scheme 2.

In some embodiments of the present application, the device 1 may control the screen 1 to display after acquiring the display scheme 1; while the control device 2 performs the display according to the display scheme 2. In this way, the display of screen 1 may be displayed earlier than the display of screen 2, thereby allowing the user to quickly view a portion of the content of the collaborative display. For example, the scheme can be applied to scenes with more complex collaborative display contents.

In other embodiments, the device 1 may control the display of the screen 1 with reference to a time consuming control device 2 to perform the display after acquiring the display scenario 1 and the display scenario 2. Thereby achieving an effect of displaying the cooperative display content on the screen 1 and the screen 2 at the same time. Therefore, the user can completely view the content of the collaborative display, and the completeness of the content of the collaborative display can be better shown to the user. For example, the scheme can be applied to a scene with simpler collaborative display content.

In this way, the device 1 can automatically adjust the content to be displayed by each of the device 1 and the device 2 during the cooperative display by adaptively determining the position information of the device 2 with respect to the device 1. Therefore, the device 1 and the device 2 can automatically adjust the displayed content according to the position information of the device 2 relative to the device 1 on the premise of not needing user operation, and the problem of cooperative display error caused by the fact that the position option set by the user does not accord with the actual position is effectively avoided. For example, please refer to fig. 8 in conjunction with fig. 4. When the position of the tablet computer serving as the auxiliary device changes as shown in (a) of fig. 4, by using the scheme provided by the embodiment of the present application (the scheme shown in fig. 7), the main device may autonomously adjust the display contents of the two devices in the collaborative display process according to the position change of the auxiliary device, so that the correct collaborative display effect as shown in fig. 8 may be obtained without the problem as shown in (b) of fig. 4.

According to the above description of the scheme illustrated in fig. 7, it can be understood that the scheme provided in the embodiment of the present application can be applied to a plurality of scenarios to solve the corresponding problem.

For example, in some embodiments, the scheme may be applied in a scenario where two devices start to perform collaborative display. Therefore, the user can use the two devices for cooperative display without setting position-related options.

For example, the user may input the operation of selecting the extension mode as shown in fig. 1 without inputting the operation as shown in fig. 2. Correspondingly, after receiving the operation of the user selecting the expansion mode, the main device (e.g., the laptop shown in fig. 1) may detect the change of the surrounding magnetic field according to the scheme shown in fig. 7, determine the position information of the auxiliary device (e.g., the tablet) relative to the main device, and determine the display contents of the main device and the auxiliary device accordingly (e.g., determine the co-display scheme). And controlling the two electronic devices to perform corresponding collaborative display according to the determined display content. It is apparent that in this example, since the user does not need to input the location option to select both electronic devices, no situation occurs because the user-selected location does not correspond to the actual location. Meanwhile, the main device can accurately determine the position information of the auxiliary device, so that the cooperative display can be more accurately performed. In addition, by using the scheme shown in fig. 7, the operation of the user in the process of using the cooperative display function can be simplified, and the convenience is improved.

In other embodiments, the scheme can be applied to a scene that two devices are in a collaborative display process. Therefore, when the position information of the auxiliary equipment relative to the main equipment changes, the user does not need to set the position related options again, and the two equipment can be ensured to carry out correct cooperative display.

For example, during the process of the cooperative display of the main device (such as a notebook computer) and the auxiliary device (such as a tablet computer). The user moves the tablet computer from the left side to the right side of the laptop. Then, the notebook can determine the position information of the tablet computer relative to the notebook computer after moving according to the scheme shown in fig. 7, when the magnetic field is not changed any more, and according to the distribution of the magnetic field at the moment and the magnetic field at the moment when the magnetic field starts to change. The notebook computer can adjust the collaborative display scheme according to the determined position information of the tablet computer, so that self-adaptive adjustment of collaborative display is realized. It can be seen that in this example, since the locations of the primary device and the secondary device do not need to be reselected after the user moves the primary device and/or the secondary device, the situation that the user-selected location does not coincide with the actual location does not occur. Meanwhile, the main device can accurately determine the position information of the auxiliary device, so that the cooperative display can be more accurately performed. In addition, by using the scheme shown in fig. 7, the operation of the user in the process of using the cooperative display function can be simplified, and the convenience is improved.

In the above description, the example in which the master device controls the slave devices to perform the cooperative display is described. In other embodiments of the present application, the auxiliary device may also perform corresponding operations, so that the adaptive adjustment of the collaborative display is more flexible and accurate.

Exemplarily, in connection with fig. 7. When the device 1 performs S701, that is, determines the position of the device 2 according to the change of the magnetic field, the device 2 may also determine the change of the peripheral magnetic field during the movement of the device 2 according to a hall sensor provided on the device 2. For example, during the process that the device 2 approaches the device 1 along the negative X-axis direction, the device 2 can determine the change of the magnetic field in the negative X-axis direction through its hall sensor. Thus, device 2 can determine that device 1 is being approached from the right side of device 1. Device 2 may send the relative location information of device 2 and device 1 to facilitate device 1 in determining that device 2 is located on the right side of device 1. Alternatively, the device 1 may determine that the device 2 is located on the right side of the device 1 by comprehensively judging from the position information received from the device 2 and the position information of the device 2 determined by a hall sensor on the device 1.

In other embodiments, the device 2 may further send its own posture information (such as a landscape posture or a portrait posture) to the device 1, so that the device 1 adjusts the display scheme corresponding to the device 2 accordingly, and further, the device 1 may more accurately control the content cooperatively displayed on the device 2.

Exemplarily, with reference to fig. 9, a schematic flow chart of another collaborative display provided in the embodiment of the present application is shown. Here, the description is continued by taking the device 1 as a main device and the device 2 as an auxiliary device as an example. As shown in fig. 9, the method may include:

s901, the device 1 determines the position information of the device 2 relative to the device 1 according to the magnetic field change.

In conjunction with the description of fig. 7, the execution manner of S901 is similar to that of S701, and is not described here again.

S902, the device 1 determines the display scheme 3 on the screen 1 according to the position information of the device 2 relative to the device 1.

With reference to the description of fig. 7, reference may be made to the specific implementation process of the scheme of displaying scheme 1 on the screen 1, which is determined by the device 1 in S702, for the implementation manner of S902, and details are not repeated here.

It should be noted that, in some implementations of the present example, the device 1 may also perform comprehensive judgment in combination with the posture information 1 of the device 1 in the process of determining the display scheme 3. Wherein the posture information 1 of the device 1 may include a landscape screen posture or a portrait screen posture. The pose information 1 may be determined by the IMU provided on the device 1, or the device 1 may be determined by a gravity sensor provided on the device 1, or the device 1 may be determined by an aspect ratio of the content currently displayed on the screen 1. The embodiment of the present application does not limit the manner in which the device 1 acquires the posture information 1.

And S903, the device 2 determines the posture information 2 of the device 2.

In conjunction with the description in the foregoing S902, the posture information of the device 2 may include a landscape posture or a portrait posture, similar to the posture information 1 of the device 1. The acquisition method is similar.

S904, device 2 sends pose information 2 to device 1.

Illustratively, the device 2 may send the pose information 2 to the device 1 through an interconnected communication relationship established with the device 1.

And S905, the device 1 determines the display scheme 4 of the device 2 according to the position information and the posture information 2 of the device 2 relative to the device 1.

In this example, the device 2 may send the acquired pose information 2 to the device 1, so that the device 1 can more accurately determine the display content corresponding to the device 2 in the collaborative display process.

It will be appreciated that in the case of a rectangular screen of the device 2, if the device 2 is rotated during movement, problems may also arise if the display is made according to a display scheme prior to rotation.

Exemplarily, refer to fig. 10. After the device 2 has been moved to the left of the device 1, if the screen is rotated, for example from landscape to portrait. Then, if the display is performed in the portrait position according to the display scheme in the landscape case, the image that should have been displayed on the device 2 cannot be displayed completely by the display 2. In order to avoid this problem, in this embodiment of the application, the device 1 may determine, according to the posture information 2 acquired from the device 2, that the posture of the device 2 is a landscape screen or a portrait screen in the current collaborative display scene. Thereby enabling the device 1 to flexibly adjust the display scheme 4 corresponding to the device 2.

It should be noted that, in the specific implementation procedure of this example, the implementations of S901 to S905 do not have a fixed order. For example, in some embodiments, device 1 may obtain pose information for device 2 after determining display scheme 3. It should be understood that the device 1 may acquire pose information of the device 2 before determining the display scheme 3. Correspondingly, with reference to the foregoing description of fig. 7, the device 1 determines the display schemes 3 and 4 partially in sequence, and may execute the schemes sequentially or in parallel. The two can be mutually independent and not interfered.

And S906, the device 1 controls the screen 1 and the screen 2 to carry out cooperative display according to the display scheme 3 and the display scheme 4.

Illustratively, device 1 may send display scheme 4 to device 2; to control the device 2 to display on the screen 2 according to the display scheme 4. In addition, the device 1 may also control the display of the screen 1 according to the display scheme 3.

In the present example, after the device 1 determines the display scheme 4 that matches the position information of the device 2 with respect to the device 1 and the posture information of the device 2, the device 1 can control the device 2 to display according to the display scheme 4. For example, the device 1 may send the display scheme 4 to the device 2, so that the device 2 may perform a collaborative display according to the display scheme 4.

Illustratively, in the case where the scheme as shown in fig. 9 is used, the display effect thereof is as shown in fig. 11. Compare the effects shown in fig. 10 and fig. 11. As is apparent, in fig. 11, the device 1 can more reasonably control the device 2 to perform the cooperative display, after taking into account the posture information 2 of the device 2, without the situation that the image content cannot be completely displayed.

In the embodiments of fig. 7 and 9, the trigger mechanism for the device 1 to detect the change in the peripheral magnetic field may be that the device 1 performs detection of the peripheral magnetic field after receiving an instruction from the user to start the cooperative display, or may trigger detection of the magnetic field at a certain period.

In addition, in some embodiments of the present application, the device 1 determines the display scheme of the device 2, and may also be determined by referring to hardware information such as a device model of the device 2. Illustratively, the influence of different electronic devices on the magnetic field is different. That is, the variations in the magnetic field detected by the device 1 are different during the approach of different models of slave devices to the master device in different directions. Therefore, in some embodiments of the present application, the model of the device 1 may be configured to display the information in a coordinated manner, and the corresponding relationship between the magnetic field variation conditions corresponding to different devices. Different magnetic field variations may correspond to different threshold settings. For example, the scheme shown in fig. 7 is used as an example. The device 1 may be interconnected with the device 2 at the start of the cooperative display. In this way, the device 1 can know the device model of the device 2. According to the device model of the device 2, the device 1 can obtain the threshold value of the magnetic field change corresponding to the device 2 from the preset corresponding relation. Then, in the process of S701 executed by the device 1, the device 1 can more accurately determine whether the device 2 is close to the device 1 and the position information of the device 2 relative to the device 1 according to the change condition of the magnetic field and the threshold value of the change of the magnetic field corresponding to the device 2. It can be understood that, because the device 1 can flexibly select the corresponding threshold according to the device model of the device 2, the device 1 can more accurately determine the location information of the device 2, and further the device 1 can more accurately control the screen 1 and the screen 2 of the device 2 to cooperatively display.

It should be noted that, in the above description of the solutions provided in the embodiments of the present application, the main device automatically determines the position information of the auxiliary device according to the change condition of the magnetic field, so as to achieve the effect of accurate and convenient cooperative display. In other embodiments, the method provided in the embodiments of the present application may also be applied to some other multi-screen display scenes besides the collaborative display. For example, in a display mode such as a mirror image display mode or an extended display mode, the master device also needs to determine a corresponding display scheme according to a position relationship between multiple screens, so as to control each screen to display corresponding content. By adopting the collaborative display scheme provided by the embodiment of the application, the main device can automatically determine the position information of other devices participating in multi-screen display, and an accurate display scheme is provided according to the position information. Therefore, the effect of accurately displaying under the condition of not needing the setting of a user can be achieved.

The scheme provided by the embodiment of the application is mainly introduced from the perspective of electronic equipment. To implement the above functions, it includes hardware structures and/or software modules for performing the respective functions. Those of skill in the art would readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the functional modules of the devices involved in the method may be divided according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

Please refer to fig. 12, which is a schematic diagram illustrating a composition of a cooperative display apparatus 1200 according to an embodiment of the present disclosure. The apparatus may be disposed in the first electronic device, and is configured to implement the collaborative display scheme in the foregoing embodiment. The cooperative display apparatus 1200 shown in fig. 12 may be disposed in the electronic device 500, or may be another description of the electronic device 500, with reference to fig. 5. The functions of the respective units included therein may be implemented by hardware components in the electronic device 500.

Illustratively, the apparatus includes: a determining unit 1201 for automatically determining position information of the second electronic device relative to the first electronic device, the position information including an upper side of the first electronic device, a lower side of the first electronic device, a left side of the first electronic device, and a right side of the first electronic device.

The determining unit 1201 is further configured to determine a collaborative display scheme according to the position information.

A control unit 1202, configured to control the first electronic device and the second electronic device to perform cooperative display according to the cooperative display scheme.

In one possible design, the apparatus further includes: a receiving unit 1203, configured to receive an operation of a user before the first electronic device automatically determines the location information of the second electronic device relative to the first electronic device, where the operation of the user is used to instruct the first electronic device to activate the cooperative display function. Alternatively, the operation of the first user is used to instruct the first electronic device to activate the extended display mode. Alternatively, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode.

In one possible design, the first electronic device has the capability of detecting the ambient magnetic field strength. The determining unit 1201 is specifically configured to determine the position information according to a change of a magnetic field around the first electronic device. The determining unit 1201 determines the position information of the second electronic device relative to the first electronic device when the magnetic field strength in a direction exceeds a preset threshold, wherein the direction includes an upper side of the first electronic device, a lower side of the first electronic device, a left side of the first electronic device, and a right side of the first electronic device.

In one possible design, the control unit 1202 is configured to control, by the first electronic device, the first display screen of the first electronic device to display the first portion of the content to be displayed according to the collaborative display scheme. The control unit 1202 is further configured to control a second display screen of the second electronic device to display a second portion of the content to be displayed according to a second collaborative display scheme, where the first portion and the second portion constitute the content to be displayed.

In one possible design, in a case where the second electronic device is located on the left side of the first electronic device, the first portion is a right portion of the content to be displayed, and the second portion is a left portion of the content to be displayed. In the case where the second electronic device is located on the right side of the first electronic device, the first portion is a left portion of the content to be displayed, and the second portion is a right portion of the content to be displayed. In a case where the second electronic device is located on an upper side of the first electronic device, the first portion is a lower portion of the content to be displayed, and the second portion is an upper portion of the content to be displayed. In a case where the second electronic device is located on the lower side of the first electronic device, the first portion is an upper portion of the content to be displayed, and the second portion is a lower portion of the content to be displayed.

In one possible design, the receiving unit 1203 is further configured to receive the posture information of the second electronic device before the first electronic device determines the collaborative display scheme according to the position information. A determining unit 1201, configured to determine a collaborative display scheme according to the relative position and the posture information of the second electronic device.

In one possible design, the first electronic device and the second electronic device are in a wirelessly connected state.

In one possible design, before the control unit 1202 controls the first electronic device and the second electronic device to perform the cooperative display according to the cooperative display scheme, the position information of the first electronic device with respect to the second electronic device is the first position information. The control unit 1202 is configured to control the first electronic device to perform the first collaborative display according to the first collaborative display scheme according to the first location information. The first location information corresponds to a first collaborative display scheme. When the first location information changes to second location information, the determining unit 1201 is configured to automatically determine that the location information of the second electronic device relative to the first electronic device is the second location information. The determining unit 1201 is further configured to determine a second collaborative display scheme according to the second position information. The control unit 1202 is configured to control, according to a second collaborative display scheme, the first electronic device and the second electronic device to switch from the first collaborative display to a second collaborative display, where the second collaborative display is a display corresponding to the second collaborative display scheme.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again.

Fig. 13 shows a schematic block diagram of an electronic device 1300. Illustratively, the electronic device 1300 may be the first electronic device in the foregoing embodiments. As shown in fig. 13, the electronic device 1300 may include: a processor 1301 and a memory 1302. The memory 1302 is used to store computer-executable instructions. For example, in some embodiments, the processor 1301, when executing the instructions stored in the memory 1302, may cause the electronic device 1300 to perform any one of the collaborative display methods described in the above embodiments.

Fig. 14 shows a schematic diagram of a chip system 1400. The chip system 1400 may include: a processor 1401 and a communication interface 1402 to enable associated devices to implement the functionality involved in the embodiments described above. In one possible design, the system-on-chip further includes a memory for storing necessary program instructions and data for the terminal. The chip system may be constituted by a chip, or may include a chip and other discrete devices. It should be noted that, in some implementations of the present application, the communication interface 1402 may also be referred to as an interface circuit.

The functions or actions or operations or steps, etc., in the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the present application are all or partially generated upon loading and execution of computer program instructions on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or can comprise one or more data storage devices, such as a server, a data center, etc., that can be integrated with the medium. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Although the present application has been described in conjunction with specific features and embodiments thereof, it will be evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and figures are merely exemplary of the present application as defined in the appended claims and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include such modifications and variations.

Claims

1. A collaborative display method, the method comprising:

the method comprises the steps that a first electronic device automatically determines position information of a second electronic device relative to the first electronic device, wherein the position information comprises the upper side of the first electronic device, the lower side of the first electronic device, the left side of the first electronic device and the right side of the first electronic device;

the first electronic equipment determines a collaborative display scheme according to the position information;

and the first electronic equipment controls the first electronic equipment and the second electronic equipment to carry out cooperative display according to the cooperative display scheme.

2. The method of claim 1, wherein prior to the first electronic device automatically determining location information of a second electronic device relative to the first electronic device, the method further comprises:

the first electronic equipment receives user operation, wherein the user operation is used for indicating the first electronic equipment to activate a collaborative display function; or the operation of the first user is used for indicating the first electronic equipment to activate an expansion display mode; or the operation of the first user is used for indicating the first electronic equipment to activate a mirror image display mode.

3. The method of claim 1 or 2, wherein the first electronic device has the capability of detecting ambient magnetic field strength;

the method for automatically determining the position information of the second electronic equipment relative to the first electronic equipment by the first electronic equipment comprises the following steps:

the first electronic equipment determines the position information according to the change of the peripheral magnetic field of the first electronic equipment;

wherein, in a case that the first electronic device determines that a magnetic field strength in a direction exceeds a preset threshold, the first electronic device determines position information of the second electronic device relative to the first electronic device, the direction including an upper side of the first electronic device, a lower side of the first electronic device, a left side of the first electronic device, and a right side of the first electronic device.

4. The method of claim 3, wherein the first electronic device is provided with a magnetic sensor for providing the first electronic device's ability to detect ambient magnetic field strength.

5. The method according to any one of claims 1 to 4,

the first electronic device controls the first electronic device and the second electronic device to perform cooperative display according to the cooperative display scheme, including:

the first electronic equipment controls a first display screen of the first electronic equipment to display a first part of the content to be displayed according to the cooperative display scheme; and the first electronic equipment controls a second display screen of the second electronic equipment to display a second part of the content to be displayed according to the second cooperative display scheme, wherein the first part and the second part form the content to be displayed.

6. The method of claim 5,

in the case that the second electronic device is located on the left side of the first electronic device, the first portion is a right portion of the content to be displayed, and the second portion is a left portion of the content to be displayed;

under the condition that the second electronic device is positioned on the right side of the first electronic device, the first part is left part content in the content to be displayed, and the second part is right part content in the content to be displayed;

in the case that the second electronic device is located on the upper side of the first electronic device, the first part is a lower part of the content to be displayed, and the second part is an upper part of the content to be displayed;

in a case where the second electronic device is located on a lower side of the first electronic device, the first portion is an upper portion of the content to be displayed, and the second portion is a lower portion of the content to be displayed.

7. The method according to any of claims 1-6, wherein before the first electronic device determines a collaborative display scheme based on the location information, the method further comprises:

the first electronic device receives attitude information of the second electronic device, wherein the attitude information comprises: a landscape screen attitude or a portrait screen attitude;

the first electronic device determines a collaborative display scheme according to the relative position, and the collaborative display scheme includes:

and the first electronic equipment determines the collaborative display scheme according to the relative position and the posture information of the second electronic equipment.

8. The method according to any one of claims 1 to 7,

before the first electronic device controls the first electronic device and the second electronic device to perform cooperative display according to the cooperative display scheme, the position information of the first electronic device relative to the second electronic device is first position information; the first electronic equipment and the second electronic equipment perform first collaborative display according to a first collaborative display scheme according to the first position information; the first location information corresponds to the first collaborative display scheme;

when the first position information changes into second position information, the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, and the method comprises the following steps:

the first electronic equipment automatically determines that the position information of the second electronic equipment relative to the first electronic equipment is the second position information;

the first electronic device determines a collaborative display scheme according to the position information, and the collaborative display scheme includes:

the first electronic equipment determines the second collaborative display scheme according to the second position information;

and the first electronic device controls the first electronic device and the second electronic device to switch from the first collaborative display to a second collaborative display according to the second collaborative display scheme, wherein the second collaborative display is a display corresponding to the second collaborative display scheme.

9. An electronic device, characterized in that the electronic device comprises one or more processors and one or more memories; the one or more memories coupled with the one or more processors, the one or more memories storing computer instructions;

the computer instructions, when executed by the one or more processors, cause the electronic device to perform the collaborative display method of any of claims 1-8.

10. A computer-readable storage medium comprising computer instructions which, when executed, perform the collaborative display method according to any one of claims 1-8.