WO2022206566A1

WO2022206566A1 - Collaborative display method and electronic device

Info

Publication number: WO2022206566A1
Application number: PCT/CN2022/082856
Authority: WO
Inventors: 李世明
Original assignee: 华为技术有限公司
Priority date: 2021-03-29
Filing date: 2022-03-24
Publication date: 2022-10-06
Also published as: CN115129285A

Abstract

The present application relates to the field of electronic devices, and provides a collaborative display method and an electronic device. The electronic device adaptively adjusts a collaborative display solution to achieve an effect of performing correct collaborative display without user intervention. The specific solution comprises: a first electronic device automatically determines position information of a second electronic device with respect to the first electronic device, the position information comprising an upper side, a lower side, a left side, and a right side; the first electronic device determines a collaborative display solution according to the position information; and the first electronic device controls, according to the collaborative display solution, the first electronic device and the second electronic device to perform collaborative display.

Description

A collaborative display method and electronic device

This application claims the priority of the Chinese patent application with the application number 202110334007.7 and the application title "A Collaborative Display Method and Electronic Device" filed with the State Intellectual Property Office on March 29, 2021, the entire contents of which are incorporated herein by reference Applying.

technical field

The embodiments of the present application relate to the field of electronic devices, and in particular, to a collaborative display method and electronic device.

Background technique

Multi-screen collaboration technology can support the collaborative display of multiple electronic devices and is gradually being widely used. As an example, an electronic device that controls collaborative display is called a master device, and an electronic device that performs collaborative display under the control of the master device is called a slave device. The main device can set options corresponding to multi-screen collaboration. The option may include the positional relationship between the primary device and the secondary device, and the like. The master device can control other electronic devices (such as auxiliary devices) with a display function that establish a connection relationship with the master device according to the set options, so as to realize collaborative display. For example, the main device may determine the content that the main device and the auxiliary device need to display in the collaborative display according to the positional relationship with the auxiliary device. In this way, the main device can control its display screen to display according to what the main device needs to display. In addition, the main device can also instruct the auxiliary device, and display on the display screen of the auxiliary device according to the content that the auxiliary device needs to display.

Since the positional relationship between the electronic devices is set by the user through the main device, when the positional relationship between the main device and the auxiliary device is inconsistent with the set positional relationship, a collaborative display error occurs. In order to correct the above problem, the user needs to reset the options corresponding to the multi-screen collaboration on the main device. Therefore, the realization of collaborative display is complicated and not intelligent enough.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a collaborative display method and electronic device, which can enable an electronic device (eg, a first electronic device) to automatically determine the relative value of another device (eg, a second electronic device) for collaborative display relative to the first electronic device. position information, thereby adaptively adjusting the collaborative display scheme so that the correct collaborative display occurs without user intervention.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

A first aspect provides a collaborative display method, the method comprising: a first electronic device automatically determining position information of a second electronic device relative to the first electronic device, the position information includes an upper side of the first electronic device, the first electronic device The underside of the device, the left side of the first electronic device, and the right side of the first electronic device. The first electronic device determines a collaborative display scheme according to the location information. The first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme.

Based on this solution, a collaborative display solution of two electronic devices is provided. In this example, the first electronic device may serve as a master device, and the second electronic device may serve as a slave device. For example, in the collaborative display process, the first electronic device can act as a control center in the collaborative display process, controlling the screen of the first electronic device to display corresponding content, and the first electronic device can also control the screen of the second electronic device to display corresponding content content, thereby realizing the collaborative display of the first electronic device and the second electronic device. In this solution, the first electronic device can determine the position of the second electronic device by itself, for example, the position of the second electronic device can be identified by a direction relative to the first electronic device. The first electronic device can determine the location information of the second electronic device without user intervention, and control the first electronic device and the second electronic device to perform collaborative display accordingly. Exemplarily, when the second electronic device starts the collaborative display with the first electronic device, the first electronic device can automatically determine the two screens in the collaborative display process according to the position of the second electronic device without the user's active setting. The content to be displayed separately is to determine the collaborative display scheme. Since the collaborative display scheme corresponds to the positional relationship between the first electronic device and the second electronic device, there will be no collaborative display error caused by inconsistent collaborative display and location. It can be understood that this solution can be applied not only to the above-mentioned example, when the collaborative display is started, but also to the scenario where the relative positions of the first electronic device and the second electronic device change during the collaborative display. Based on similar reasons in the foregoing description, the accuracy of the collaborative display can therefore be ensured, and at the same time, the user does not need to set the relevant information again.

In a possible design, before the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, the method further includes: the first electronic device receives a user's operation, and the user's operation is used to indicate The first electronic device activates the collaborative display function. Or, the operation of the first user is used to instruct the first electronic device to activate the extended display mode. Or, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode. Based on this scheme, an example of the application of this scheme is pointed out. For example, it can be applied in the scenario where the user instructs the electronic device to enter the collaborative display mode. It can be understood that in other multi-screen display scenarios such as the extended mode and the mirroring mode, in order to ensure the accuracy of the displayed content of each screen, the relative positional relationship of each screen also needs to be clarified. Therefore, the solution exemplified in the present invention can also be applied to achieve the effect of accurate display without requiring the user to participate in setting.

In one possible design, the first electronic device has the ability to detect the strength of the surrounding magnetic field. The first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, including: the first electronic device determines the position information according to the change of the surrounding magnetic field of the first electronic device. Wherein, when the first electronic device determines that the magnetic field strength in one direction exceeds a preset threshold, the first electronic device determines the position information of the second electronic device relative to the first electronic device, and the direction includes the upper side of the first electronic device. , the lower side of the first electronic device, the left side of the first electronic device, and the right side of the first electronic device. Based on this solution, a specific example is given in which the first electronic device determines the location information of the second electronic device. For example, the first electronic device may determine the location of the second electronic device by monitoring changes in the magnetic field (such as changes in the strength of the magnetic field) in the surrounding environment. It should be noted that, in this example, in order to avoid misjudgment caused by weak changes in the magnetic field, the first electronic device may determine that the second electronic device is located in a corresponding direction when the magnetic field strength exceeds a preset threshold. Thereby, the accuracy of determining the location information is further improved.

In a possible design, the first electronic device is provided with a magnetic sensor, and the magnetic sensor is used to provide the first electronic device with the ability to detect the strength of the surrounding magnetic field. Based on this scheme, a specific method for electronic equipment to detect magnetic field changes is presented. For example, the electronic device may be provided with a magnetic sensor for monitoring the strength of the magnetic field.

In a possible design, the first electronic device controlling the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme includes: the first electronic device controlling the first display of the first electronic device according to the collaborative display scheme The screen displays the first part of the content to be displayed. The first electronic device controls the second display screen of the second electronic device to display the second part of the content to be displayed according to the second collaborative display scheme, and the first part and the second part constitute the content to be displayed. Based on this scheme, a specific implementation of collaborative display according to the collaborative display scheme is given. In this example, the first electronic device may control the screen of the first electronic device and the screen of the second electronic device to display corresponding content according to the determined collaborative display scheme. It can be understood that, since the collaborative display scheme corresponds to the positions of the first electronic device and the second electronic device, the positional relationship between the first part and the second part in the content to be displayed may be the same as that of the first electronic device and the second electronic device. The position of the second electronic device corresponds to. This ensures the accuracy of the collaborative display.

In a possible design, when the second electronic device is located on the left side of the first electronic device, the first part is the right part of the content to be displayed, and the second part is the left part of the content to be displayed . When the second electronic device is located on the right side of the first electronic device, the first part is the left part of the content to be displayed, and the second part is the right part of the content to be displayed. When the second electronic device is located on the upper side of the first electronic device, the first part is the lower part of the content to be displayed, and the second part is the upper part of the content to be displayed. When the second electronic device is located on the lower side of the first electronic device, the first part is the upper part of the content to be displayed, and the second part is the lower part of the content to be displayed. Based on this solution, a specific correspondence between the positional relationship between electronic devices and the positional relationship between collaboratively displayed contents is given. According to the description in this example, it can be seen that the positions of the content displayed on different screens in the content to be displayed correspond to the positional relationship between electronic devices. Therefore, no matter when the collaborative display is started or during the collaborative display, the position of the electronic device changes, since the content of the collaborative display is consistent with the position of the electronic device, the accuracy of the collaborative display can be guaranteed.

In a possible design, before the first electronic device determines the collaborative display scheme according to the position information, the method further includes: the first electronic device receives the attitude information of the second electronic device. The first electronic device determines the collaborative display scheme according to the relative position, including: the first electronic device determines the collaborative display scheme according to the relative position and the attitude information of the second electronic device. Based on this solution, another implementation of the first electronic device determining a collaborative display solution is provided. In this example, the first electronic device may further adjust the collaboratively displayed content in combination with the gesture information of the second electronic device. In this way, the accuracy of the collaboratively displayed content is further improved.

In a possible design, the posture information includes: landscape orientation or portrait orientation. Based on this scheme, a specific example of pose information is given. For example, according to the posture information, the first electronic device may determine whether the second electronic device is in a landscape orientation or a portrait orientation. It is understandable that under different postures, the best solution for displaying content is different. For example, the second electronic device can display the second content in a horizontally stretched form in a landscape orientation, and for example, the second electronic device can display the second content in a vertical orientation in a portrait orientation. The first electronic device can further adjust the display scheme of the second electronic device accordingly, thereby obtaining a better collaborative display effect.

In a possible design, the first electronic device and the second electronic device are in a wireless connection state. Based on this scheme, an example of an implementation scenario of this scheme is given. In this example, the first electronic device may maintain a wireless connection state with the second electronic device, for example, the wireless connection state may be a collaborative display state based on the same local area network. Through the communication channel provided by the wireless connection state, the first electronic device can interact with the second electronic device. posture information.

In a possible design, before the first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, the position information of the first electronic device relative to the second electronic device is the first position information. The first electronic device and the second electronic device perform a first collaborative display according to the first collaborative display scheme according to the first position information. The first location information corresponds to the first collaborative display scheme. When the first position information is changed to the second position information, the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, including: the first electronic device automatically determines that the second electronic device is relative to the first electronic device. The location information of an electronic device is the second location information. The first electronic device determining the collaborative display scheme according to the position information includes: the first electronic device determining the second collaborative display scheme according to the second position information. The first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, including: the first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the second collaborative display scheme. The display is switched to the second collaborative display, and the second collaborative display is a display corresponding to the second collaborative display scheme. Based on this solution, a specific implementation scenario example of this solution is provided. In this example, the positional relationship of the first electronic device and the second electronic device may not be fixed. During the collaborative display process, the first electronic device may continuously/intermittently adjust the collaborative display scheme according to the position of the second electronic device, so that when the position of the second electronic device changes (for example, from the first position information change to the second position information), the first electronic device can adaptively adjust the collaborative display scheme, and control the screen of the first electronic device and the screen of the second electronic device to display content corresponding to the current position information.

In a second aspect, a collaborative display device is provided, the device can be set in a first electronic device, the device includes: a determining unit for automatically determining position information of the second electronic device relative to the first electronic device, the position information includes Top side, bottom side, left side, right side. The determining unit is further configured to determine the collaborative display scheme according to the location information. The control unit is configured to control the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme.

In a possible design, the apparatus further includes: a receiving unit, configured to receive a user's operation before the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, and the user's operation is used for The first electronic device is instructed to activate the collaborative display function. Or, the operation of the first user is used to instruct the first electronic device to activate the extended display mode. Or, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode.

In one possible design, the first electronic device has the ability to detect the strength of the surrounding magnetic field. The determining unit is specifically configured to determine the position information according to the change of the surrounding magnetic field of the first electronic device. Wherein, when the magnetic field strength in one direction exceeds a preset threshold, the determining unit determines the position information of the second electronic device in the first electronic device, and the directions include up, down, left, and right.

In a possible design, the first electronic device is provided with a magnetic sensor, and the magnetic sensor is used to provide the first electronic device with the ability to detect the strength of the surrounding magnetic field.

In a possible design, the control unit is used for the first electronic device to control the first display screen of the first electronic device to display the first part of the content to be displayed according to the collaborative display scheme. The control unit is further configured to control the second display screen of the second electronic device to display the second part of the content to be displayed according to the second collaborative display scheme, and the first part and the second part constitute the content to be displayed.

In a possible design, when the second electronic device is located on the left side of the first electronic device, the first part is the right part of the content to be displayed, and the second part is the left part of the content to be displayed . When the second electronic device is located on the right side of the first electronic device, the first part is the left part of the content to be displayed, and the second part is the right part of the content to be displayed. When the second electronic device is located on the upper side of the first electronic device, the first part is the lower part of the content to be displayed, and the second part is the upper part of the content to be displayed. When the second electronic device is located on the lower side of the first electronic device, the first part is the upper part of the content to be displayed, and the second part is the lower part of the content to be displayed.

In a possible design, the receiving unit is further configured to receive the attitude information of the second electronic device before the first electronic device determines the collaborative display scheme according to the position information. The determining unit is configured to determine the collaborative display scheme according to the relative position and the attitude information of the second electronic device.

In a possible design, the posture information includes: landscape orientation or portrait orientation.

In a possible design, the first electronic device and the second electronic device are in a wireless connection state.

In a possible design, before the control unit controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, the location information of the first electronic device relative to the second electronic device is the first location information. The control unit is configured to control the first electronic device to perform the first collaborative display according to the first collaborative display scheme according to the first position information. The first location information corresponds to the first collaborative display scheme. In the case where the first position information is changed to the second position information, the determining unit is configured to automatically determine the position information of the second electronic device relative to the first electronic device as the second position information. The determining unit is further configured to determine a second collaborative display scheme according to the second position information. The control unit is configured to control the first electronic device and the second electronic device to switch from the first collaborative display to the second collaborative display according to the second collaborative display scheme, and the second collaborative display is the display corresponding to the second collaborative display scheme.

In a third aspect, an electronic device is provided, the electronic device includes one or more processors and one or more memories. Exemplarily, the electronic device may be the first electronic device involved in the first aspect or the second aspect. One or more memories are coupled to the one or more processors, and the one or more memories store computer instructions. The computer instructions, when executed by one or more processors, cause an electronic device to perform the method of co-displaying as described in any one of the first aspect and any of its possible designs.

In a fourth aspect, a chip system is provided, the chip system includes an interface circuit and a processor. Exemplarily, the chip system may be applied to the first electronic device involved in the first aspect or the second aspect. The interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the chip system performs the first aspect and The synergy of any of its possible designs shows the method.

In a fifth aspect, a computer-readable storage medium is provided, the computer-readable storage medium includes computer instructions, and when the computer instructions are executed, the collaborative display described in any one of the first aspect and any possible designs thereof is performed. this method.

A sixth aspect provides a computer program product, the computer program product includes instructions, when the computer program product runs on a computer, the computer can execute any one of the first aspect and any possible designs thereof according to the instructions The described synergy shows the method.

It should be understood that the technical features of the technical solutions provided in the second aspect, the third aspect, the fourth aspect, the fifth aspect and the sixth aspect can all correspond to the collaborative display provided in the first aspect and its possible designs. Therefore, the beneficial effects that can be achieved are similar, which will not be repeated here.

Description of drawings

1 is a schematic diagram of a collaborative display;

Fig. 2 is a kind of schematic diagram of the setting of collaborative display;

3 is a schematic diagram of the effect of a collaborative display;

Fig. 4 is another kind of synergistic display effect schematic diagram;

FIG. 5 is a schematic diagram of the composition of an electronic device provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of location information determination according to an embodiment of the present application;

FIG. 7 is a schematic flowchart of a collaborative display method provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of the effect of a collaborative display method provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of another collaborative display method provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a collaborative display scenario provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of the effect of another collaborative display method provided by an embodiment of the present application;

12 is a schematic diagram of the composition of a collaborative display device provided by an embodiment of the present application;

13 is a schematic flowchart of another shooting method provided by an embodiment of the present application;

FIG. 14 is a schematic diagram of the composition of a chip system according to an embodiment of the present application.

Detailed ways

Multi-screen collaboration technology plays an increasingly important role in scenarios such as mobile office and smart home. Through multi-screen collaboration technology, the interactive interface between users and electronic devices can be expanded and the interactive experience can be enriched. At the same time, the multi-screen collaboration technology can also use the perception ability between electronic devices to make electronic devices more intelligent, thereby simplifying user operations and improving productivity.

In the following, an example of a collaborative display solution will be described with reference to FIG. 1 to FIG. 4 .

Generally speaking, when using the multi-screen collaborative technology for collaborative display, the user can set options for collaborative display on the main device. Exemplarily, take the main device as a notebook computer as shown in FIG. 1 and the auxiliary device as a tablet computer as an example. Referring to FIG. 1 , on the collaborative display setting interface of the notebook computer, interface elements shown as 101 may be displayed. This interface element can be used to prompt the user to use extended mode for collaborative display. The user can control devices such as a mouse, and select the extended mode, so that the notebook computer enters a collaborative display state.

It should be noted that, in some embodiments, as shown in FIG. 1 , before the notebook computer displays the collaborative display setting interface, the notebook computer can be interconnected with other electronic devices capable of collaborative display. It can be displayed collaboratively with other electronic devices (such as tablet computers, etc.). For example, after the laptop computer and the tablet computer are successfully interconnected, on the collaborative display setting interface, an electronic device (such as device X) that has been connected to the laptop computer may be displayed.

In addition, in the above example, the user inputs an operation (such as an operation of selecting an extended mode) to the notebook computer through the control device as an example for description. In other implementations of this example, for a notebook computer that supports touch operations, the user may also input the above operations in the form of touch. It can be understood that the user can also input operations through voice control or the like. The following description is given by taking the user inputting a control command to the electronic device through a mouse as an example.

In this example, the host device (such as a notebook computer) may display other collaborative display setting interfaces (such as a display parameter setting interface) after receiving the operation of selecting the extended mode by the user. Exemplarily, referring to FIG. 2 , the display parameter setting interface may include a “tablet screen position” option for setting the relative position of the tablet computer and the notebook computer, and display setting options of other auxiliary devices (such as the “tablet screen position” shown in FIG. 2 . Sidebar Position" option).

As shown in FIG. 2 , the user can select the corresponding positional relationship in the “tablet screen position” option according to the relative positions of the tablet computer and the notebook computer during collaborative display. The main device (such as a laptop computer) can determine the content to be displayed by the main device and the content to be displayed by the auxiliary device (such as a tablet computer) according to the position relationship selected by the user. In this embodiment of the present application, the positional relationship between the primary device and the secondary device may include that the secondary device is located on the left side of the primary device, the secondary device is located on the right side of the primary device, the secondary device is located on the upper side of the primary device, and the secondary device is located on the lower side of the primary device Wait.

Illustratively, take the tablet computer placed on the right side of the notebook computer as an example. The user can select an option on the interface corresponding to the corresponding position relationship on the laptop computer through the mouse (for example, select option 201 as shown in FIG. 2 ). The notebook computer can receive the operation, and according to the operation, the notebook computer can determine that the tablet computer is located on the right side of the notebook computer during the collaborative display process. Thus, the notebook computer can determine the content that needs to be displayed on the screen of the notebook computer and the content that the tablet computer needs to display during the collaborative display process.

In this embodiment of the present application, the user can activate the collaborative display function of the electronic device (such as a notebook and/or a tablet computer) through the above operations as shown in FIG. 1 and FIG. 2 .

As an example, take the extended display of pictures through collaborative display as an example. In conjunction with Figure 3. The display of the picture on the laptop is shown in (a) in Figure 3. With co-display, a master device, such as a laptop, can display a portion of a picture on its screen. The main device can control the auxiliary device (such as a tablet computer) to display another part of the image on its screen, thereby achieving the effect of expanding the display area of the displayed image. Combined with the descriptions in Figures 1 and 2, when the tablet computer is located on the right side of the notebook computer, through the settings shown in Figures 1 and 2, the notebook computer can determine that the tablet computer is located on the right side of the notebook computer for collaborative display. . For example, a laptop can display the left-hand portion of a picture on the upper line of its screen. The laptop can also control the tablet to display the right portion of the image on its screen. Please refer to (b) in Figure 3 for the display results. In this way, the extended display of pictures through collaborative display is realized on the two electronic devices.

However, there are some problems with the solution in the above example.

It can be understood that the positional relationship between the main device and the auxiliary device in the real environment must be consistent with the positional relationship selected in the collaborative display setting process, so that the main device can know the position of the auxiliary device in the real environment, and then determine the collaborative display process. , the content that the primary and secondary devices need to display. However, when the positional relationship between the main device and the auxiliary device in the real environment is inconsistent with the positional relationship selected in the collaborative display setting process, or when the positional relationship between the main device and the auxiliary device changes during the collaborative display process, the main device cannot accurately Therefore, it is impossible to accurately determine the content that the main device and the auxiliary device need to display.

For example, it is assumed that the main device is the device A shown in FIG. 4 and the auxiliary device is the device B shown in FIG. 4 . Refer to (a) in FIG. 4 . During the collaborative display process, device B may move in position. For example, device B can move from the right side of device A to the left side of device A. In this way, if the user does not reset the option of collaborative display, device A determines whether device B is located on the right side of device A according to the preset settings. Device A can continue to control device A and device B to perform collaborative display accordingly. In this way, the situation as shown in (b) in FIG. 4 may occur. That is, the device B on the left side displays the image of the right part of the picture, while the device A on the right actually displays the image of the left part of the picture. This obviously cannot meet the needs of users for viewing or editing pictures.

In order to solve the problems in the above examples, the collaborative display solution provided by the embodiments of the present application enables the electronic device to automatically determine the positions of other devices in the collaborative display process, and perform collaborative display accordingly. For example, when using the collaborative display solution provided by the embodiments of the present application on the main device, the main device can automatically determine the position of the auxiliary device, and accordingly determine the content to be displayed by the main device and the auxiliary device. In this way, since the user does not need to actively set the option of collaborative display, but the main device can automatically determine the position of the auxiliary device, there is no need for the user to set manually, or the positional relationship between the main device and the auxiliary device changes. Cooperative display content error problem.

The collaborative display solution provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Please refer to FIG. 5 , which is a schematic diagram of the composition of an electronic device 500 according to an embodiment of the present application. In this embodiment of the present application, the electronic device 500 may be used for collaborative display. For example, the electronic device 500 may act as a master device for collaborative display. For another example, the electronic device 500 can also be used as an auxiliary device for collaborative display.

It should be noted that, the solutions provided by the embodiments of the present application can be applied to the user's electronic device. The electronic device can be a mobile phone, a tablet computer, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, a media player, etc. For a portable mobile device that performs collaborative display, the electronic device may also be a wearable electronic device that is equipped with a display screen, such as a smart watch, and can perform collaborative display. The specific form of the electronic device is not particularly limited in the embodiments of the present application.

As shown in FIG. 5 , the electronic device may include a processor 510 , an external memory interface 520 , an internal memory 521 , a universal serial bus (USB) interface 530 , a charge management module 540 , a power management module 541 , and a battery 542 , Antenna 1, Antenna 2, Mobile Communication Module 550, Wireless Communication Module 560, Audio Module 570, Speaker 570A, Receiver 570B, Microphone 570C, Headphone Interface 570D, Sensor Module 580, Key 590, Motor 591, Indicator 592, Camera 593 , display screen 594, and subscriber identification module (subscriber identification module, SIM) card interface 595 and so on.

It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 500 . In other embodiments, the electronic device 500 may include more or fewer components than shown, or some components may be combined, or some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 510 may include one or more processing units, for example, the processor 510 may include an application processor (application processor, AP), a modem processor, a graphics processor 510 (graphics processing unit, GPU), image signal processing image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural-network processing unit (NPU) )Wait. The different processing units may be independent devices, or may be integrated into one or more processors 510 .

The controller may be the nerve center and command center of the electronic device 500 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 510 for storing instructions and data. In some embodiments, the memory in processor 510 is cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 510 . If the processor 510 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the waiting time of the processor 510 is reduced, thereby increasing the efficiency of the system.

In this embodiment of the present application, the processor 510 may store an instruction or an instruction set corresponding to the collaborative display solution provided by the embodiment of the present application. Illustratively, it is taken as an example that the processor 510 is provided in the main device. When implementing the collaborative display solution provided by the embodiment of the present application, the processor 510 may determine the position of the auxiliary device according to the change of the surrounding magnetic field intensity. The processor 510 may also be configured to determine the content to be displayed by the main device and the auxiliary device according to the position of the auxiliary device.

In some embodiments, the processor 510 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver transmitter (UART) interface, a mobile industry processor 510 interface (MIPI) interface. ), General Purpose Input Output (GPIO) interface, Subscriber Identity Module (SIM) interface, and/or Universal Serial Bus (USB) interface, etc.

The electronic device 500 may implement a shooting function through an ISP, a camera 593, a video codec, a GPU, a display screen 594, an application processor, and the like.

The ISP is used to process the data fed back by the camera 593 . For example, when taking a photo, the shutter is opened, the light is transmitted to the photosensitive element of the camera 593 through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera 593 transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 593 .

Camera 593 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 500 may include 1 or N cameras 593 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 500 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 500 may support one or more video codecs. In this way, the electronic device 500 can play or record videos in various encoding formats, for example, moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor 510 , which can rapidly process input information and continuously self-learn by learning from the biological neural network structure, such as the transmission mode between neurons in the human brain. Applications such as intelligent cognition of the electronic device 500 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The charging management module 540 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 540 may receive charging input from the wired charger through the USB interface 530 . In some wireless charging embodiments, the charging management module 540 may receive wireless charging input through a wireless charging coil of the electronic device 500 . While the charging management module 540 charges the battery 542 , the electronic device 500 can also be powered by the power management module 541 .

The power management module 541 is used to connect the battery 542 , the charging management module 540 and the processor 510 . The power management module 541 receives input from the battery 542 and/or the charging management module 540, and supplies power to the processor 510, the internal memory 521, the external memory, the display screen 594, the camera 593, and the wireless communication module 560. The power management module 541 can also be used to monitor parameters such as the capacity of the battery 542, the number of cycles of the battery 542, and the health status (electric leakage, impedance) of the battery 542. In some other embodiments, the power management module 541 may also be provided in the processor 510 . In other embodiments, the power management module 541 and the charging management module 540 may also be provided in the same device.

The wireless communication function of the electronic device 500 may be implemented by the antenna 1, the antenna 2, the mobile communication module 550, the wireless communication module 560, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 500 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 550 may provide a wireless communication solution including 2G/3G/4G/5G, etc. applied on the electronic device 500 . The mobile communication module 550 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 550 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 550 can also amplify the signal modulated by the modulation and demodulation processor, and then convert it into electromagnetic waves for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 550 may be provided in the processor 510 . In some embodiments, at least part of the functional modules of the mobile communication module 550 may be provided in the same device as at least part of the modules of the processor 510 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 570A, receiver 570B, etc.), or displays images or videos through display screen 594 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modulation and demodulation processor may be independent of the processor 510, and may be provided in the same device as the mobile communication module 550 or other functional modules.

The wireless communication module 560 can provide applications on the electronic device 500 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 560 may be one or more devices integrating at least one communication processing module. The wireless communication module 560 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 510 . The wireless communication module 560 can also receive the signal to be sent from the processor 510 , perform frequency modulation on it, amplify it, and then convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 500 is coupled with the mobile communication module 550, and the antenna 2 is coupled with the wireless communication module 560, so that the electronic device 500 can communicate with the network and other devices through wireless communication technology. The wireless communication technologies may include, but are not limited to, GSM, GPRS, CDMA, WCDMA, TD-SCDMA, LTE, 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and/or IR technologies.

In some embodiments of the present application, the mobile communication module 550 and/or the wireless communication module 560 in the electronic device 500 may be used to establish interconnection with other electronic devices under the control of the processor 510 . For example, take the electronic device 500 as the main device as an example. The electronic device 500 may join the same local area network as the auxiliary device through the mobile communication module 550 and/or the wireless communication module 560 . The electronic device 500 can communicate with the auxiliary device through the local area network. For example, the electronic device 500 may send the display information corresponding to the auxiliary device to the auxiliary device through the local area network. In order to facilitate the auxiliary device to perform collaborative display according to the received display information.

The electronic device 500 implements a display function through a GPU, a display screen 594, an application processor, and the like. The GPU is the microprocessor 510 for image processing, and is connected to the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 510 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 594 is used to display images, videos, and the like. In some embodiments, the electronic device 500 may include 1 or N display screens 594 , where N is a positive integer greater than 1.

In this embodiment of the present application, both the main device and the auxiliary device can be provided with a display screen 594 . In some scenarios, the display screen of the primary device may perform collaborative display with the display screen of the secondary device.

The external memory interface 520 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 500. The external memory card communicates with the processor 510 through the external memory interface 520 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 521 may be used to store computer executable program code, which includes instructions. The processor 510 executes various functional applications and data processing of the electronic device 500 by executing the instructions stored in the internal memory 521 . The internal memory 521 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 500 and the like. In addition, the internal memory 521 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, general-purpose flash memory, and the like.

The electronic device 500 may implement audio functions through an audio module 570, a speaker 570A, a receiver 570B, a microphone 570C, an earphone interface 570D, and an application processor. Such as music playback, recording, etc.

The audio module 570 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 570 may also be used to encode and decode audio signals. In some embodiments, the audio module 570 may be provided in the processor 510 , or some functional modules of the audio module 570 may be provided in the processor 510 .

Speaker 570A, also referred to as "speaker", is used to convert audio electrical signals into sound signals. The electronic device 500 can listen to music through the speaker 570A, or listen to a hands-free call.

The receiver 570B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 500 answers a call or a voice message, the voice can be answered by placing the receiver 570B close to the human ear.

The microphone 570C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 570D is used to connect wired headphones.

The sensor module 580 provided in the electronic device 500 provided in this embodiment of the present application may include one or more sensors of the same or different types. Exemplarily, the sensor module 580 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

Take the sensor module 580 including a magnetic sensor as an example. The magnetic sensor may comprise a Hall sensor. The Hall sensor can determine the change of the magnetic field around the electronic device 500 according to the Hall effect. Among them, the Hall effect is a kind of electromagnetic effect, which was discovered by American physicist Hall. According to the Hall effect, when a current passes through a semiconductor perpendicular to the external magnetic field, the carriers are sent offset, and an additional electric field is generated perpendicular to the direction of the current and the magnetic field, resulting in a potential difference across the semiconductor. This potential difference is also called the Hall potential difference.

In this embodiment of the present application, the electronic device 500 may determine the change of the surrounding magnetic field according to the Hall sensor. It can be understood that one or more magnetic materials are provided in electronic devices. Therefore, when the auxiliary device is close to the main device, the magnetic material on the auxiliary device will have an effect on the magnetic field around the main device. When the electronic device 500 is used as the main device, it can determine the direction in which the auxiliary device approaches the main device according to the change of the magnetic field, thereby determining the relative positions of the main device and the auxiliary device.

In other embodiments of the present application, the electronic device 500 can detect the opening and closing of the flip holster by using a magnetic sensor. In some embodiments, when the electronic device 500 is a flip machine, the electronic device 500 can detect the opening and closing of the flip according to the magnetic sensor. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

In other embodiments of the present application, when the electronic device 500 is configured with a foldable screen, the electronic device 500 can use a magnetic sensor to detect the opening and closing states (eg, the unfolded state, or the closed state) of the foldable screen. The electronic device 500 can realize automatic control of functions such as automatic unlocking and volume adjustment according to the detected opening and closing state of the folding screen.

Touch sensor, also known as "touch panel". The touch sensor may be disposed on the display screen 594, and the touch sensor and the display screen 594 form a touch screen, also called a "touch screen". A touch sensor is used to detect touch operations on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. In some embodiments, visual output related to touch operations may be provided via display screen 594 . In other embodiments, the touch sensor may also be disposed on the surface of the electronic device 500 , which is different from the location where the display screen 594 is located.

It should be noted that, in some embodiments of the present application, the sensor module 580 in the electronic device 500 may be integrated together to implement the functions of the above components, or may be partially integrated or separately provided with the above components to implement its sensing function of. For example, in some embodiments, an inertial measurement unit (Inertial Measurement Unit, IMU) may be provided in the electronic device 500 . The IMU can be used to implement 6-axis or 9-axis sensing functions. Illustratively, take 9-axis detection through the IMU as an example. The IMU can be used to implement the sensing functions of components such as acceleration sensors, gyroscope sensors, and Hall sensors.

The keys 590 include a power-on key, a volume key, and the like. The electronic device 500 may receive key 590 inputs and generate key signal inputs related to user settings and function control of the electronic device 500 .

Motor 591 can generate vibrating cues. The indicator 592 can be an indicator light, which can be used to indicate the charging status, the change of power, and can also be used to indicate messages, missed calls, notifications, and the like. The SIM card interface 595 is used to connect a SIM card.

The collaborative display solution provided by the embodiments of the present application can be used in the electronic device 500 shown in FIG. 5 above.

The solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Generally speaking, in order to realize the functions of electronic devices, one or more magnetic materials are disposed therein. For example, take the electronic device as a tablet computer as an example. The mode of use of the tablet may include a holster mode. In the holster mode, the tablet can execute corresponding strategies according to the opening and closing status of the holster, such as screen-on/off-screen, etc. In order to make the holster fit closely with the tablet computer, a magnetic material can be arranged in the tablet computer, so that when the holster is closed, the magnetic material in the holster can attract each other with the magnetic material in the tablet computer, thereby achieving a tight fit. closed effect. Of course, in other electronic devices, even if the electronic device does not support the holster mode, the magnetic material will be set according to its function implementation.

When a magnetic material enters a magnetic field, it has an effect on the magnetic field. For example, the strength of the magnetic field at the corresponding location of the magnetic material will change. The change of the magnetic field strength can be acquired by the Hall sensor, thereby enabling the electronic device provided with the Hall sensor to determine the orientation of the magnetic material.

Exemplarily, in conjunction with FIG. 6 . Take the device 1 provided with a sensor (such as a Hall sensor) capable of detecting changes in the magnetic field, and the device 2 provided with a magnetic material as an example. The device 1 can obtain the distribution of the surrounding magnetic field through the sensor. When the device 2 approaches the device 1 in the negative direction of the X-axis, it will cause the change of the magnetic field strength of the device 1 in the positive direction of the X-axis. The device 1 can acquire the change of the magnetic field strength through the sensor. Accordingly, device 1 can know that device 2 is approaching device 1 along the negative X-axis direction, that is, device 2 is located on the right side of device 1 .

In the solution provided by the embodiment of the present application, the master device may determine the position of the auxiliary device close to the master device according to the technical means in the above description. Then, the main device can determine the content to be displayed by the main device and the auxiliary device respectively in the collaborative display according to the position of the auxiliary device. Then, the corresponding content is displayed on the screen of the main device and the auxiliary device is controlled to display the corresponding content on the screen, thereby realizing the coordinated display of the main device and the auxiliary device.

It can be understood that since the main device can automatically identify the position of the auxiliary device according to its internal components, it is not necessary for the user to manually set the position options of the auxiliary device in the collaborative display process (for example, according to the scheme shown in Fig. 1-Fig. Options). Therefore, the problem of collaborative display errors caused by the inconsistency between the location options set by the user and the positions of the main device and the auxiliary device in the actual scene can be avoided, and at the same time, the electronic device can provide a more convenient collaborative display capability.

As an example, please refer to FIG. 7 , which is a schematic flowchart of a collaborative display method provided by an embodiment of the present application. The main device is device 1, the auxiliary device is device 2, the display screen of the main device is screen 1, and the display screen of the auxiliary device is screen 2 as an example. As shown in Figure 7, the solution can include:

S701 , the device 1 determines the position information of the device 2 relative to the device 1 according to the change of the magnetic field.

The position information of the device 2 relative to the device 1 may include one of the upper side of the device 1 , the lower side of the device 1 , the left side of the device 1 , and the right side of the device 1 .

In some embodiments, the device 1 may determine the position information of the device 2 relative to the device 1 when the magnetic field strength in one direction reaches (or exceeds) a preset threshold (eg, a first threshold).

For example, when it is determined that the magnetic field strength in the positive direction of the X-axis reaches the first threshold, the device 1 may determine that the position information of the device 2 relative to the device 1 at this time is: the right side of the device 1 . That is, device 1 can determine that device 2 is located to the right of device 1 . When it is determined that the magnetic field strength in the negative direction of the X-axis reaches the first threshold, the device 1 may determine that the position information of the device 2 relative to the device 1 at this time is: the left side of the device 1 . When it is determined that the magnetic field strength in the positive direction of the Y-axis reaches the first threshold, the device 1 may determine that the position information of the device 2 relative to the device 1 at this time is: the upper side of the device 1 . When it is determined that the magnetic field strength in the negative direction of the Y-axis reaches the first threshold, the device 1 may determine that the position information of the device 2 relative to the device 1 at this time is: the lower side of the device 1 .

In other embodiments, the device 1 may determine the location information of the device 2 when the change value of the magnetic field strength in one direction reaches another corresponding preset threshold (eg, a second threshold).

For example, the device 1 may determine that the device 2 is located on the right side of the device 1 when it is determined that the change value of the magnetic field strength in the positive direction of the X-axis reaches the second threshold. The device 1 may determine that the device 2 is located on the left side of the device 1 when it is determined that the change value of the magnetic field strength in the negative direction of the X-axis reaches the second threshold. The device 1 may determine that the device 2 is located on the upper side of the device 1 when it is determined that the change value of the magnetic field strength in the positive direction of the Y-axis reaches the second threshold. The device 1 may determine that the device 2 is located on the lower side of the device 1 at this time when it is determined that the change value of the magnetic field strength in the negative direction of the Y-axis reaches the second threshold.

S702 , the device 1 determines a collaborative display scheme according to the position information of the device 2 relative to the device 1 .

In this embodiment of the present application, the collaborative display scheme may also be simply referred to as a display scheme. The display scheme may include display scheme 1 and display scheme 2. The display scheme 1 may be a scheme displayed by the screen 1 on the device 1 during the collaborative display process. The display scheme 2 may be a scheme displayed by the screen 2 on the device 2 during the collaborative display process.

In this example, the device 1 can determine the display scheme 1 according to the position information of the device 2 relative to the device 1 .

Exemplarily, after determining the position information of the device 2 relative to the device 1, the device 1 may determine the content to be displayed on the screen 1 of the device 1, that is, determine the display scheme 1 on the screen 1.

Take the collaborative display of pictures by device 1 and device 2 as an example. When it is determined that the device 2 is on the right side of the device 1, the device 1 may determine that the left part of the image of the picture needs to be displayed in the display solution 1. When it is determined that the device 2 is on the left side of the device 1, the device 1 may determine that the right part of the image of the picture needs to be displayed in the display solution 1. When it is determined that the device 2 is on the upper side of the device 1, the device 1 may determine that the lower part of the image of the picture needs to be displayed in the display scheme 1. When it is determined that the device 2 is on the lower side of the device 1, the device 1 may determine that the upper part of the image of the picture needs to be displayed in the display scheme 1.

In addition, the device 1 can also determine the display scheme 2 according to the position information of the device 2 relative to the device 1 .

It can be understood that, in most collaborative display solutions, the device 2 is used as a secondary device, and the content to be displayed is determined by the main device and controls the device 2 to display. Therefore, in this embodiment, in combination with the foregoing description, the device 1 can also determine the content to be displayed by the device 2 according to the position information of the device 2 relative to the device 1 .

Exemplarily, continue to take the device 1 and the device 2 performing the collaborative display of pictures as an example. When it is determined that the device 2 is on the right side of the device 1, the device 1 may determine that the right part of the image of the picture needs to be displayed in the display scheme 2. When it is determined that the device 2 is on the left side of the device 1, the device 1 may determine that the left part of the image of the picture needs to be displayed in the display scheme 2. When it is determined that the device 2 is on the upper side of the device 1, the device 1 may determine that the upper part of the image of the picture needs to be displayed in the display scheme 2. When it is determined that the device 2 is on the lower side of the device 1, the device 1 may determine that the lower part of the image of the picture needs to be displayed in the display scheme 2.

In this way, device 1 can adaptively adjust the display scheme on device 1 (such as display scheme 1) and the display scheme on device 2 (such as display scheme 2) according to the position information of device 2 relative to device 1.

It should be noted that, in different implementations of the embodiments of the present application, the operations performed by the device 1 to determine the display scheme 1 and the display scheme 2 may have different sequences. For example, in some implementations, device 1 may determine display solution 2 after determining display solution 1, thereby avoiding centralized consumption of computing power. In other implementations, the device 1 can determine the display scheme 1 after determining the display scheme 2, so that the device 1 can control the device 2 to prepare for display according to the display scheme 2 while determining the display scheme 1, so as to achieve the effect of parallel processing , to improve the synchronization of the content displayed on the two screens during the collaborative display process. It can be understood that, in other implementations, the device 1 can also determine the display scheme 2 while determining the display scheme 1, so that the time-consuming of data processing in the collaborative display process can also be saved through parallel processing.

S703 , the device 1 controls the screen 1 of the device 1 and the screen 2 of the device 2 to perform collaborative display according to the collaborative display scheme.

Exemplarily, after determining the display scheme 2 corresponding to the device 2, the device 1 may send the display scheme 2 to the device 2, so that the device 2 can display according to the display scheme 2.

In this example, after the device 1 determines the display scheme 2 of the device 2, the device 1 can send the display scheme 2 of the device 2 to the device 2 through the interconnection channel with the device 2. So that the device 2 can perform collaborative display according to the display scheme 2 sent by the device 1.

In addition, the device 1 can also control the screen 1 to perform corresponding display according to the display scheme 1 .

It should be noted that, in this example, since the device 1 can be controlled internally, the display of the screen 1 can be controlled according to the display scheme 1 . However, the device 1 needs to communicate with the device 2 to control the display of the device 2 so that the device 2 can be controlled to perform the corresponding collaborative display according to the display scheme 2 .

In some embodiments of the present application, after acquiring the display scheme 1, the device 1 may control the screen 1 to display; at the same time, the device 2 may be controlled to display according to the display scheme 2. In this way, the display of the screen 1 may be slightly earlier than the display of the screen 2, thereby allowing the user to quickly view a part of the content displayed in conjunction. For example, this solution can be applied to a scenario where the collaborative display content is relatively complex.

In other embodiments, after acquiring the display scheme 1 and the display scheme 2, the device 1 may control the display of the screen 1 with reference to the timing of controlling the device 2 to perform display which takes a long time. In this way, the effect of displaying the collaboratively displayed content on the screen 1 and the screen 2 at the same time is achieved. In this way, the user can view the collaboratively displayed content completely, and the integrity of the collaboratively displayed content can be better displayed to the user. For example, this solution can be applied to a scenario where the collaborative display content is relatively simple.

In this way, by adaptively determining the position information of the device 2 relative to the device 1, the device 1 can automatically adjust the content to be displayed by the device 1 and the device 2 during the collaborative display process. This enables device 1 and device 2 to automatically adjust the displayed content according to the location information of device 2 relative to device 1 without user operation, effectively avoiding the situation where the location option set by the user does not match the actual location. A problem with the synergy display error occurred. Exemplarily, in conjunction with FIG. 4 , please refer to FIG. 8 . When the position of the tablet computer serving as the auxiliary device changes as shown in (a) in FIG. 4 , using the solution provided by the embodiment of the present application (the solution shown in FIG. 7 ), the main device can autonomously adjust according to the auxiliary device. The position of , adjusts the display content of the two devices during the collaborative display process, so the correct collaborative display effect as shown in Figure 8 can be obtained without the problem shown in (b) of Figure 4.

According to the above description of the solution in FIG. 7 , it can be understood that the solutions provided by the embodiments of the present application can be applied to multiple scenarios to solve corresponding problems.

Exemplarily, in some embodiments, the solution may be applied to a scenario where two devices start to perform collaborative display. As a result, the user does not need to set location-related options, and can use two devices for collaborative display.

For example, the user may not need to input the operation shown in FIG. 2 after inputting the operation of selecting the expansion mode as shown in FIG. 1 . Correspondingly, the main device (the notebook computer shown in Fig. 1) can detect the change of the surrounding magnetic field according to the scheme shown in Fig. 7 after receiving the operation of the expansion mode selected by the user, and determine the auxiliary device (such as the tablet computer). ) relative to the position information of the master device, and determine the display content of the master device and the slave device accordingly (eg, determine a co-display scheme). Further, according to the determined display content, the two electronic devices are controlled to perform corresponding collaborative display. Obviously, in this example, since the user does not need to input the location options for selecting two electronic devices, there will be no situation where the location selected by the user does not match the actual location. At the same time, since the master device can accurately determine the position information of the slave device, collaborative display can be performed more accurately. In addition, using the solution shown in FIG. 7 can also simplify the user's operation in the process of using the collaborative display function, and improve the convenience.

In other embodiments, the solution can be applied to a scenario where two devices are in the process of co-displaying. Therefore, when the location information of the auxiliary device relative to the main device changes, the user does not need to re-set the location-related options, and it is also possible to ensure that the two devices perform correct collaborative display.

For example, in the process of collaborative display between the main device (such as a laptop computer) and the auxiliary device (such as a tablet computer). The user moves the tablet from the left side of the laptop to the right side. Then, the notebook can determine the position information of the tablet computer relative to the notebook computer after moving according to the scheme shown in Figure 7, when the magnetic field no longer changes, according to the current magnetic field and the distribution of the magnetic field when the magnetic field begins to change. The notebook computer can adjust the collaborative display scheme according to the determined position information of the tablet computer, thereby realizing the adaptive adjustment of the collaborative display. It can be seen that, in this example, after the user moves the primary device and/or the secondary device, there is no need to re-select the positions of the primary device and the secondary device, so there is no possibility that the user-selected location is different from the actual location. A match occurs. At the same time, since the master device can accurately determine the position information of the slave device, collaborative display can be performed more accurately. In addition, using the solution shown in FIG. 7 can also simplify the user's operations in the process of using the collaborative display function, and improve convenience.

It should be noted that, in the above description, the master device controls the slave devices to perform collaborative display as an example for description. In other embodiments of the present application, the auxiliary device may also perform corresponding operations, so that the adaptive adjustment of the collaborative display is more flexible and accurate.

Exemplarily, in conjunction with FIG. 7 . When the device 1 executes S701, that is, when the position of the device 2 is determined according to the change of the magnetic field, the device 2 can also judge the change of the surrounding magnetic field during the movement of the device 2 according to the Hall sensor set on the device 2. For example, when the device 2 approaches the device 1 in the negative direction of the X-axis, the device 2 can determine that the magnetic field in the negative direction of the X-axis changes through its Hall sensor. Thus, device 2 can determine that device 1 is approaching from the right side of device 1 . The device 2 can send the relative position information of the device 2 and the device 1 to the device 1, so that the device 1 can determine that the device 2 is located on the right side of the device 1. Alternatively, the device 1 can comprehensively determine that the device 2 is located on the right side of the device 1 according to the position information received from the device 2 and the position information of the device 2 determined by the Hall sensor on the device 1 .

In other embodiments, device 2 may also send its own posture information (such as landscape orientation or portrait orientation) to device 1, so that device 1 can adjust the display scheme corresponding to device 2 accordingly, so that device 1 can Control the content of the collaborative display on device 2 more accurately.

Exemplarily, with reference to FIG. 9 , another schematic flowchart of collaborative display provided by an embodiment of the present application. Among them, continue to take the device 1 as the main device and the device 2 as the auxiliary device as an example for description. As shown in Figure 9, the method may include:

S901 , the device 1 determines the position information of the device 2 relative to the device 1 according to the change of the magnetic field.

With reference to the description of FIG. 7 , the execution manner of S901 is similar to that of S701, and details are not repeated here.

S902 , the device 1 determines the display scheme 3 on the screen 1 according to the position information of the device 2 relative to the device 1 .

With reference to the description of FIG. 7 , for the execution manner of S902, reference may be made to the specific execution process of the solution of the display solution 1 on the screen 1 determined by the device 1 in S702, which will not be repeated here.

It should be noted that, in some implementations of this example, in the process of determining the display scheme 3, the device 1 may also make a comprehensive judgment in combination with the attitude information 1 of the device 1 . The posture information 1 of the device 1 may include a landscape orientation or a portrait orientation. The gesture information 1 can be determined by the IMU set on the device 1, or determined by the gravity sensor set on the device 1, or determined by the device 1 through the aspect ratio of the content currently displayed on the screen 1 . This embodiment of the present application does not limit the manner in which the device 1 acquires the gesture information 1 .

S903 , the device 2 determines the posture information 2 of the device 2 .

With reference to the above description in S902, similar to the posture information 1 of the device 1, the posture information of the device 2 may include a landscape orientation or a portrait orientation. Its acquisition method is similar.

S904 , the device 2 sends the gesture information 2 to the device 1 .

Exemplarily, the device 2 may send the gesture information 2 to the device 1 through the interconnected communication relationship established with the device 1 .

S905 , the device 1 determines the display scheme 4 of the device 2 according to the position information and the attitude information 2 of the device 2 relative to the device 1 .

In this example, the device 2 can send the acquired gesture information 2 to the device 1, so that the device 1 can more accurately determine the display content corresponding to the device 2 in the collaborative display process.

It can be understood that when the screen of the device 2 is rectangular, if the device 2 is rotated during the movement, some problems will also occur if the display is displayed according to the display scheme before the rotation.

Illustratively, refer to FIG. 10 . After the device 2 is moved to the left side of the device 1, if the screen is rotated, for example, it is rotated from a landscape screen to a portrait screen. Then, if it is displayed in the portrait orientation according to the display scheme in the landscape situation, the image that should be displayed on the device 2 cannot be completely displayed by the device 2 . In order to avoid this problem, in this embodiment of the present application, the device 1 may determine, according to the posture information 2 obtained from the device 2, whether the posture of the device 2 is a landscape screen or a portrait screen in the current collaborative display scenario. Further, the device 1 can flexibly adjust the display scheme 4 corresponding to the device 2 accordingly.

It should be noted that, in the specific execution process of this example, there is no fixed sequence in the execution of the foregoing S901-S905. Exemplarily, in some embodiments, the device 1 may acquire the gesture information of the device 2 after determining the display scheme 3 . It should be understood that the device 1 may acquire the posture information of the device 2 before determining the display scheme 3 . Correspondingly, with reference to the foregoing description of FIG. 7 , the timings at which the device 1 determines the display scheme 3 and the display scheme 4 are also partially sequential, and may be executed sequentially or in parallel. The two can not interfere with each other and are independent of each other.

S906, the device 1 controls the screen 1 and the screen 2 to perform collaborative display according to the display scheme 3 and the display scheme 4.

Exemplarily, the device 1 may send the display scheme 4 to the device 2; so as to control the device 2 to display on the screen 2 according to the display scheme 4. In addition, the device 1 can also control the screen 1 to display according to the display scheme 3 .

In this example, after device 1 determines a display scheme 4 that matches the position information of device 2 relative to device 1 and the posture information of device 2 , device 1 can control device 2 to display according to the display scheme 4 . For example, device 1 can send the display scheme 4 to device 2, so that device 2 can perform collaborative display according to the display scheme 4.

Exemplarily, when the solution shown in FIG. 9 is used, the display effect is as shown in FIG. 11 . Compare the effects shown in Figure 10 and Figure 11 . Obviously, in FIG. 11 , after considering the posture information 2 of the device 2, the device 1 can more reasonably control the device 2 to perform collaborative display, without the situation that the image content cannot be fully displayed.

It should be noted that, in the above solutions in FIGS. 7 and 9 , the trigger mechanism for device 1 to detect changes in its surrounding magnetic field may be that after device 1 receives the user’s instruction to start the collaborative display operation, it implements the detection of its surrounding magnetic field. , or the magnetic field detection can be triggered according to a certain period.

In addition, in some embodiments of the present application, the device 1 determines the display scheme of the device 2, which may also be determined with reference to hardware information such as the device model of the device 2. Exemplarily, different electronic devices have different effects on the magnetic field. That is to say, when the auxiliary devices of different models approach the main device in different directions, the changes of the magnetic field detected by the device 1 are different. Therefore, in some embodiments of the present application, the device 1 may be configured with the model of the device with which it may be displayed collaboratively, and the corresponding relationship with the magnetic field changes corresponding to different devices. Different magnetic field changes can correspond to different threshold settings. For example, take the solution shown in FIG. 7 as an example. Device 1 can be interconnected with device 2 when the collaborative display is started. In this way, device 1 can know the device model of device 2. According to the device model of the device 2, the device 1 can obtain the threshold value of the magnetic field change corresponding to the device 2 from the pre-configured corresponding relationship. Then, in the process of performing S701 by the device 1, the device 1 can more accurately determine whether the device 2 is close to the device 1 and whether the device 2 is relative to the device 2 according to the change of the magnetic field and the threshold value of the magnetic field change corresponding to the device 2 determined above. 1 location information. It can be understood that, since device 1 can flexibly select the corresponding threshold according to the device model of device 2, device 1 can more accurately determine the location information of device 2, thereby enabling device 1 to more accurately control screen 1 and device 2. screen 2 for collaborative display.

It should be noted that, the above descriptions of the solutions provided by the embodiments of the present application are described by taking the main device automatically determining the position information of the auxiliary device according to the magnetic field change, thereby achieving the effect of accurate and convenient collaborative display as an example. In other embodiments, the methods provided by the embodiments of the present application may also be applied to other multi-screen display scenarios other than collaborative display. For example, in a display mode such as a mirror display mode or an extended display mode, the master device also needs to determine a corresponding display scheme according to the positional relationship between multiple screens, and then control each screen to display corresponding content. Using the collaborative display solution provided by the embodiments of the present application, the master device can automatically determine the location information of other devices participating in multi-screen display, and provide an accurate display solution accordingly. In this way, the effect of accurate display can also be achieved without user setting.

The solutions provided by the embodiments of the present application have been introduced above mainly from the perspective of electronic devices. In order to realize the above-mentioned functions, it includes corresponding hardware structures and/or software modules for executing each function. Those skilled in the art should easily realize that the present application can be implemented in hardware or a combination of hardware and computer software with the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In this embodiment of the present application, the devices involved may be divided into functional modules according to the above method examples. For example, each functional module may be divided into each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. It should be noted that, the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and there may be other division manners in actual implementation.

Please refer to FIG. 12 , which is a schematic diagram of the composition of a collaborative display device 1200 according to an embodiment of the present application. The apparatus can be set in the first electronic device, and is used to realize the collaborative display solution in the above-mentioned embodiment. With reference to FIG. 5 , the collaborative display apparatus 1200 described in FIG. 12 may be provided in the electronic device 500 , or may be another description of the electronic device 500 . The functions of each unit included therein may be implemented by hardware components in the electronic device 500 .

Exemplarily, the apparatus includes: a determining unit 1201, configured to automatically determine position information of the second electronic device relative to the first electronic device, where the position information includes the upper side of the first electronic device, the lower side of the first electronic device, and the first electronic device. The left side of one electronic device, the right side of the first electronic device.

The determining unit 1201 is further configured to determine a collaborative display scheme according to the location information.

The control unit 1202 is configured to control the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme.

In a possible design, the apparatus further includes: a receiving unit 1203, configured to receive the user's operation before the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device. for instructing the first electronic device to activate the collaborative display function. Or, the operation of the first user is used to instruct the first electronic device to activate the extended display mode. Or, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode.

In one possible design, the first electronic device has the ability to detect the strength of the surrounding magnetic field. The determining unit 1201 is specifically configured to determine the position information according to the change of the surrounding magnetic field of the first electronic device. Wherein, when the magnetic field strength in one direction exceeds a preset threshold, the determining unit 1201 determines the position information of the second electronic device relative to the first electronic device, the direction includes the upper side of the first electronic device, the The lower side, the left side of the first electronic device, the right side of the first electronic device.

In a possible design, the control unit 1202 is used for the first electronic device to control the first display screen of the first electronic device to display the first part of the content to be displayed according to the collaborative display scheme. The control unit 1202 is further configured to control the second display screen of the second electronic device to display the second part of the content to be displayed according to the second collaborative display scheme, and the first part and the second part constitute the content to be displayed.

In a possible design, the receiving unit 1203 is further configured to receive the attitude information of the second electronic device before the first electronic device determines the collaborative display scheme according to the position information. The determining unit 1201 is configured to determine the collaborative display scheme according to the relative position and the attitude information of the second electronic device.

In a possible design, before the control unit 1202 controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, the location information of the first electronic device relative to the second electronic device is the first location information . The control unit 1202 is configured to control the first electronic device to perform the first collaborative display according to the first collaborative display scheme according to the first position information. The first location information corresponds to the first collaborative display scheme. In the case where the first position information is changed to the second position information, the determining unit 1201 is configured to automatically determine that the position information of the second electronic device relative to the first electronic device is the second position information. The determining unit 1201 is further configured to determine a second collaborative display scheme according to the second position information. The control unit 1202 is configured to control the first electronic device and the second electronic device to switch from the first collaborative display to the second collaborative display according to the second collaborative display scheme, and the second collaborative display is the display corresponding to the second collaborative display scheme.

It should be noted that, all relevant contents of the steps involved in the above method embodiments can be cited in the functional description of the corresponding functional module, which will not be repeated here.

FIG. 13 shows a schematic diagram of the composition of an electronic device 1300 . Exemplarily, the electronic device 1300 may be the first electronic device in the foregoing embodiment. As shown in FIG. 13 , the electronic device 1300 may include: a processor 1301 and a memory 1302 . The memory 1302 is used to store computer-implemented instructions. Exemplarily, in some embodiments, when the processor 1301 executes the instructions stored in the memory 1302, the electronic device 1300 can be made to execute the collaborative display method shown in any one of the foregoing embodiments.

FIG. 14 shows a schematic diagram of the composition of a chip system 1400 . The chip system 1400 may include: a processor 1401 and a communication interface 1402, which are used to support related devices to implement the functions involved in the above embodiments. In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the terminal. The chip system may be composed of chips, or may include chips and other discrete devices. It should be noted that, in some implementation manners of the present application, the communication interface 1402 may also be referred to as an interface circuit.

The functions or actions or operations or steps in the above embodiments may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line, DSL) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or data storage devices including one or more servers, data centers, etc. that can be integrated with the medium. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made therein without departing from the spirit and scope of the application. Accordingly, this specification and drawings are merely exemplary illustrations of the application as defined by the appended claims, and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

A collaborative display method, characterized in that the method comprises:

The first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, the position information includes the upper side of the first electronic device, the lower side of the first electronic device, and the first electronic device. The left side of an electronic device, the right side of the first electronic device;

The first electronic device determines a collaborative display scheme according to the location information;

The first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme.
The method according to claim 1, wherein before the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, the method further comprises:

The first electronic device receives a user's operation, where the user's operation is used to instruct the first electronic device to activate the collaborative display function; or, the first user's operation is used to instruct the first electronic device to activate the extension display mode; or, the operation of the first user is used to instruct the first electronic device to activate the mirror display mode.
The method according to claim 1 or 2, wherein the first electronic device has the ability to detect the strength of the surrounding magnetic field;

The first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, including:

The first electronic device determines the location information according to changes in the surrounding magnetic field of the first electronic device;

Wherein, when the first electronic device determines that the magnetic field strength in one direction exceeds a preset threshold, the first electronic device determines the position information of the second electronic device relative to the first electronic device, so The direction includes the upper side of the first electronic device, the lower side of the first electronic device, the left side of the first electronic device, and the right side of the first electronic device.
The method according to claim 3, wherein the first electronic device is provided with a magnetic sensor, and the magnetic sensor is used to provide the first electronic device with the ability to detect the strength of the surrounding magnetic field.
The method according to any one of claims 1-4, wherein,

According to the collaborative display scheme, the first electronic device controlling the first electronic device and the second electronic device to perform collaborative display includes:

The first electronic device controls the first display screen of the first electronic device to display the first part of the content to be displayed according to the collaborative display scheme; the first electronic device controls the display according to the second collaborative display scheme The second display screen of the second electronic device displays a second part of the content to be displayed, and the first part and the second part constitute the content to be displayed.
The method of claim 5, wherein:

When the second electronic device is located on the left side of the first electronic device, the first part is the right part of the content to be displayed, and the second part is part of the content to be displayed the left part of ;

When the second electronic device is located on the right side of the first electronic device, the first part is the left part of the content to be displayed, and the second part is part of the content to be displayed the right part of ;

When the second electronic device is located on the upper side of the first electronic device, the first part is a lower part of the content to be displayed, and the second part is a part of the content to be displayed the upper part of the

When the second electronic device is located on the lower side of the first electronic device, the first part is an upper part of the content to be displayed, and the second part is a part of the content to be displayed the lower part of the .
The method according to any one of claims 1-6, wherein before the first electronic device determines a collaborative display scheme according to the location information, the method further comprises:

The first electronic device receives the posture information of the second electronic device, and the posture information includes: a horizontal screen posture or a vertical screen posture;

The first electronic device determines a collaborative display scheme according to the relative position, including:

The first electronic device determines the collaborative display scheme according to the relative position and the attitude information of the second electronic device.
The method according to any one of claims 1-7, wherein,

Before the first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, the position information of the first electronic device relative to the second electronic device is the first location information; the first electronic device and the second electronic device perform a first collaborative display according to the first collaborative display scheme according to the first location information; the first location information and the first Collaborative display scheme correspondence;

When the first position information is changed to the second position information, the first electronic device automatically determines the position information of the second electronic device relative to the first electronic device, including:

The first electronic device automatically determines that the position information of the second electronic device relative to the first electronic device is the second position information;

The first electronic device determines a collaborative display scheme according to the location information, including:

The first electronic device determines the second collaborative display scheme according to the second position information;

The first electronic device controls the first electronic device and the second electronic device to perform collaborative display according to the collaborative display scheme, including:

The first electronic device controls the first electronic device and the second electronic device to switch from the first collaborative display to the second collaborative display according to the second collaborative display scheme, and the second collaborative display is A display corresponding to the second collaborative display scheme.
An electronic device, characterized in that the electronic device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, the one or more memories a memory storing computer instructions;

When the one or more processors execute the computer instructions, the electronic device is caused to perform the collaborative display method according to any one of claims 1-8.
A computer-readable storage medium, characterized in that, the computer-readable storage medium comprises computer instructions, and when the computer instructions are executed, the collaborative display method according to any one of claims 1-8 is executed.