WO2023030196A1

WO2023030196A1 - Device interaction method, electronic device and system

Info

Publication number: WO2023030196A1
Application number: PCT/CN2022/115212
Authority: WO
Inventors: 陈绍君; 胡建荣; 彭峰; 熊张亮
Original assignee: 华为技术有限公司
Priority date: 2021-08-31
Filing date: 2022-08-26
Publication date: 2023-03-09
Also published as: CN115729392A

Abstract

The present application discloses a device interaction method, an electronic device and a system. The present application provides a unified ingress for the interaction between a core device and multiple associated devices. When multiple devices initiate a connection, a hyperterminal system provides a user with one or more scenario-based service options. Different scenario-based service options are ingresses of corresponding services in different application scenarios. In addition, different atomic service/function combinations are recommended to users, as well as matching devices and the like. By implementing the method provided in the present application, users may quickly select scenario-based services and/or atomic services according to personal needs. Scenario-based services and/or atomic services available under a device combination are targetedly provided by means of a user-selected device combination. Detailed and rich usage scenarios are listed for the users, and the users are guided to fully explore more application scenarios by using the functions of a hyperterminal, thereby achieving the effect of recommending scenario-based services according to device combinations.

Description

A device interaction method, electronic device and system

This application claims the priority of the Chinese patent application with the application number 202111016484.5 and the application name "A Device Interaction Method, Electronic Device and System" submitted to the China Patent Office on August 31, 2021, and the request is filed in March 2022 The priority of a Chinese patent application with application number 202210270366.5 and application title "A Device Interaction Method, Electronic Device and System" submitted to the China Patent Office on the 18th, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the technical field of terminals, and in particular to a device interaction method, electronic device and system.

Background technique

As consumption upgrades, users own more and more smart terminal devices. In some usage scenarios, users often need two or more terminal devices to link or work together to provide certain services and realize certain functions, which can also be called cross-device services. For example, mobile phones and smart screens cooperate to realize video projection; mobile phones share files or pictures with other devices.

Generally speaking, the same device manufacturer or the same application manufacturer may set different entry points and different trigger methods for different cross-device services. Different device manufacturers or different application manufacturers may also set different entry points and different trigger methods for the same cross-device service. For example, if a user needs to share a picture on electronic device A to electronic device B, the user can select the picture to be shared on electronic device A, click the sharing control, and then electronic device A starts to search for surrounding devices and display a list of devices. Select electronic device B in the device list, and electronic device A can send the selected picture to electronic device B. For another example, the user needs to cast the screen content of electronic device A to electronic device B. The user needs to turn on the wireless function of electronic device A and electronic device B, and then pair and establish a connection. After that, the user opens and casts on electronic device A. For screen-related applications, the screen projection control is operated in the application interface, and then electronic device A sends screen projection data to electronic device B, and electronic device B displays the screen image of electronic device A.

It can be seen from the above that when multiple devices are enabled to work together, the user operation is cumbersome, the entrances are scattered, the human-computer interaction performance is poor, and the user experience is poor.

Contents of the invention

This application provides a device interaction method, electronic device and system. The device interaction method provides a unified entrance for the interaction between the core device and multiple associated devices. When multiple devices initiate a connection, the hyper terminal system provides the user with One or more scenario-based service options, different scenario-based service options are the entrances of corresponding services in different application scenarios. In addition, it can also recommend different atomic service/function combinations to users, as well as recommend collocation equipment, etc.

The above and other objects are achieved by the features in the independent claims. Further implementations are presented in the dependent claims, the description and the drawings.

In a first aspect, an embodiment of the present application provides a device interaction method, the method includes: the first device displays a first interface, the first interface includes a first control and a second control, and the first control is for indicating the first The device option of the device, the second control indicates the device option of the second device, and the second device is a device discovered by the first device. The first device detects a first user operation, and the first user operation is used for the user to associate the first device with the second device. The first device displays a second interface, the second interface displays service options of a first scenario-based service, and the first scenario-based service is a scenario supported by the first device and the second device associated with the first user operation services. The first device detects a second user operation for selecting the first scenario-based service. The first device instructs the second user to operate the running of the first scenario-based service selected.

Implementing the method in the first aspect can improve the performance of human-computer interaction. Users can quickly select scene-based services and/or atomized services according to their personal needs. Through the user-selected device combination, the available scene-based services under the device combination are provided in a targeted manner. Service and/or atomic service lists detailed and rich usage scenarios for users, guides users to fully explore more application scenarios by using the functions of HyperTerminal, and achieves the effect of recommending scenario-based services according to device combinations.

With reference to the first aspect, in some embodiments, in the first interface, the second controls are distributed on a circle centered on the first control, and the first control is separated from the second control. Referring to the example shown in FIG. 5 , the first device is a core device, such as a mobile phone, and the second device is an associated device, which may be a smart watch, a smart TV, and the like.

With reference to the first aspect, in some embodiments, the positional relationship between the first control and the second control in the first interface may indicate one or more of the following relationships between the first device and the second device: connection relationship, or orientation relationship, or distance relationship, or signal strength relationship. Wherein, the first control is separated from the second control, indicating that the first device is not connected to the second device, and the first control is connected with the second control, indicating that the first device is connected to the second device. And/or, the second control is located on the left, or right, or above, or below the first control, respectively indicating that the second device is located on the left, or right, or front, or rear of the first device. And/or, the farther the distance between the first control and the second control is, the farther the distance between the first device and the second device is indicated, or the weaker the signal between the first device and the second device is, the second The closer the distance between a control and the second control is, the closer the distance between the first device and the second device is, or the stronger the signal between the first device and the second device is.

The device connection status icon in the first interface can visually and concisely indicate each device, and display the relationship between the core device and the associated device. The user is easy to operate, the interface is easy to understand, the user interface design is more humanized, and the user experience is good.

With reference to the first aspect, in some embodiments, the first user operation includes an operation in which the second control is selected, or the first control is selected, or the first control and the second control are simultaneously selected.

With reference to the first aspect, in some embodiments, the first user operation is an operation in which the user selects the second control, drags the second control to a designated area in a direction close to the first control, and then releases the operation of the second control. Or the first user operation is that the user selects the first control and the second control at the same time, and drags the first control and the second control to be close to each other until the distance between the first control and the second control is less than a certain distance, and then releases the first control and the second control. The operation of the second control. The specified area may be a circular area with the first control as the center and a radius equal to the first radius distance. Or the first user operation is for the user to select the first control, and drag the first control to move towards the direction close to the second control, until the distance between the first control and the second control is less than a certain distance, release the operation of the first control .

The second interface also shows that the first control and the second control are adsorbed together, indicating that the first device and the second device have established a connection and can work together to provide system-level services. Referring to the embodiment shown in Fig. 6 and Fig. 7 . The drag-and-drop operation is more in line with the user's operating habits, the user's learning cost is low, and the operation is easy to use. The two device options being attracted together may refer to that the edges of the corresponding icons of the two devices are tangent, or the two device options being adsorbed together may also be that the two device icons are completely or partially overlapped. In this embodiment, no special limitation is imposed on the form of adsorption.

In some embodiments, the second control in the second interface may have a different form from the second control in the first interface. For example, in FIG. 7 , the smart watch icon 702 that has been connected to a mobile phone to form a hyperterminal can display a distinctive logo, so as to be easily distinguished from the associated device icon in an unconnected state. For example, the smart watch icon 702 is displayed in a dark filled form, indicating that the smart watch is connected to the mobile phone, and the size of the smart watch icon 702 is smaller than the smart watch icon 506 when not connected.

With reference to the first aspect, in some embodiments, the first user operation is an operation in which the user clicks the second control, or simultaneously clicks the first control and the second control.

With reference to the first aspect, in some embodiments, a floating control is displayed on the second interface, the floating control includes a first area, and the first area displays service options of the first scenario-based service. The first user operation may be an operation in which the user presses and holds the second control. The second user operation may be an operation of releasing the second control after the user drags the second control to the first area, or the second user operation is the user dragging the second control through the first area toward a direction close to the first control After moving to the specified area, the operation of the second control is released. The specified area may be a circular area with the first control as the center and a radius equal to the first radius distance. Embodiments described with reference to FIGS. 11, 12, 13A, 13B, 13C, and 13D. In this embodiment, a second user operation includes two functions of associating the second device with the first device and selecting the first scenario-based service, which improves user operation efficiency.

With reference to the first aspect, in some embodiments, the selected device option can change its position along with the track of the user's finger dragging on the screen. This embodiment does not limit the finger movement trajectory when the user drags the device icon, and the user's dragging trajectory may be any curve.

With reference to the first aspect, in some embodiments, the floating control includes multiple circular areas, or multiple rectangular areas, or multiple fan-shaped areas, or multiple fan-shaped ring areas, or multiple circular ring-shaped areas, or multiple polygonal area, etc.

With reference to the first aspect, in some embodiments, the second interface includes the first control. The second interface may also not include the first control.

Scenario-based services refer to functional services based on usage scenarios, which are a collection of multiple functions or services that can be provided for a specific scenario. A scene service can be supported by one or more atom services. The service options of the scene-based service displayed on the second interface may be the scene-based service provided by the current hyper terminal system, that is, the combination of the first device and the second device, on the first device or on the second device The atomic capabilities corresponding to these scenario-based services have been installed and deployed.

In some embodiments, a certain scenario-based service may be based on a certain same application, and different devices provide different atomic capabilities under the application service. The scenario-based service is actually provided by the same service provider. For example, in the example shown in Figure 8, both the mobile phone and the smart watch are installed with sports and health applications. Both jump to the outdoor running service in the sports and health application. The smart watch can provide positioning services and statistical user sports data services. The mobile phone can provide services such as displaying the user's movement track. For the same scenario-based service, even if the same client is used, different devices may display different interfaces and provide different services.

With reference to the first aspect, in some embodiments, the first scenario-based service is a scenario-based service queried from a first database, and the first database records one or more scenario-based services supported by one or more device combinations. service, the device combination includes a device combination formed by a first device and its associated second device. The scenario-based service can be determined by one or more of the following parameters: device type of device combination, device features, product positioning, device usage, environment or scenario of the device, status of the device, recently run applications, etc. Refer to Table 1 in the subsequent examples. The first database can be stored locally or on a cloud server.

With reference to the first aspect, in some embodiments, the first scenario-based service is a scenario-based service queried from a second database, and the second database records one or more Scenario-based services, the atomic service combination includes an atomic service combination composed of atomic capabilities of the first device and its associated second device. The atomic capabilities may include one or more of the following: audio output capabilities, audio input capabilities, display capabilities, camera capabilities, touch input capabilities, and keyboard and mouse input capabilities. The second database can be stored locally or on a cloud server.

For example, the audio output capability may include whether the device supports mono or multi-channel, supported sound effects, supported frequency response range, noise reduction capability, audio analysis capability, etc. when the device is playing audio; the audio input capability may include receiving Acoustic range, noise reduction capability, etc.; display capability may include the device’s screen size, display resolution parameters, refresh rate, color expressiveness, etc.; camera capability may include the device’s camera type, shooting pixels, night scene shooting capability, and image adjustment capability, etc.; touch input capability, keyboard and mouse input capability refer to whether the device can support touch input or keyboard and mouse input, etc.

In combination with the first aspect, in some embodiments, the first scenario-based service can calculate the probability of being selected according to one or more of the following parameters: the frequency of the user using the first scenario-based service, the first scenario-based service set by the user The order of the first device and/or the second device is the environment or scene, the state of the first device and/or the second device is in, and the application recently run by the first device and/or the second device. The service option of the first scenario-based service with a higher probability of being selected by the user is recommended and displayed with higher priority.

For example, different device types have different device characteristics. For example, smart TVs have larger screens, which provide users with a better visual viewing experience and are more suitable for watching videos; smart speakers have strong audio output capabilities and better speaker effects. It is more suitable for playing audio; the earphone is small in size, easy to carry, does not need to be placed outside, does not disturb people around, and can also have a noise reduction effect, shielding external environmental noise, suitable for listening to audio in public places, such as talking in public places, listening to music, etc.; Smart watches and smart bracelets can measure the user's exercise data or heart rate and other health information more accurately than mobile phones; laptops have powerful processing capabilities and are more suitable for office work. The device type and device features will not be described in detail here, and the above examples do not limit other embodiments.

In some embodiments, when listing the scenario-based service options, the option with a high usage rate may be displayed in front according to the statistics of usage frequency of each scenario-based service option by the user within a certain period of time. The certain time can be all the time in history, or a certain period of time, such as the last three months. The higher the historical usage rate of the scenario-based service option, the higher the display.

In some other embodiments, the first device or the second device may report the detected environment of the current device, and sort the scene-based service options according to the environment. If it is detected that the user's environment is outdoors, it will give priority to recommending outdoor related service options, such as outdoor running. If it is detected that the user's environment is indoors, it will give priority to recommending indoor related service options, such as indoor running. This embodiment does not limit the method of detecting the user's environment. For example, the environment can be judged according to the current positioning information of the device; or the camera can be used to collect photos of the current environment, or ultrasonic waves, infrared rays, etc. size to determine the current user's environment.

In some other embodiments, the first device or the second device may also determine the environment or scene where the device is located according to the application recently opened or used by the user, or the currently running application, and then recommend scene-based service options. For example, when a mobile phone and a smart TV form a super terminal, if it is detected that the application currently running on the mobile phone is an office presentation application, then the multi-screen collaborative service option can be recommended first; or, if it is detected that the application currently running on the mobile phone is a video application, then the Recommend home theater service options; or, when it is detected that the webpage browsed by the user is a game-related webpage, preferentially recommend game entertainment-related service options, etc.

In some other embodiments, scenario-based services may also be recommended according to the running state of the device, such as the on-screen or off-screen state. For example, when a mobile phone is connected to a smart TV to form a super terminal, if it is detected that the smart TV is in the on-screen state, then the scene-based service is recommended as screen mirroring; if the smart TV is detected to be off-screen, the scene-based service is recommended as Casting service.

With reference to the first aspect, in some embodiments, in this embodiment, the arrangement order of the recommended scene-based services is also different if the layout of the floating control is different. Developers can design the area layout of the floating control according to the user experience.

Optionally, referring to the embodiments shown in FIG. 13C and FIG. 13D , the shape of the floating control is a circular ring or a fan ring, where multiple circular ring areas or fan ring areas respectively indicate multiple first scene-based services. The first scenario-based service indicated by the circular or fan-shaped region close to the center of the circle is a more preferentially recommended scenario-based service.

Optionally, referring to the embodiments shown in FIG. 11, FIG. 12, FIG. 13A, and FIG. 13B, the shape of the floating control is a circle or a rectangle, and the circle or rectangle is divided into four areas, and the orientation of the first quadrant is the area A, the orientation of the second quadrant is area B, the orientation of the third quadrant is area C, and the orientation of the fourth quadrant is area D. The recommended order of scenario-based service options is scenario-based service option A, scenario-based service option B, scenario-based service option C. Scenario-based service option D, the areas respectively placed are area B, area A, area C, area D, or area B, area C, area A, area D, or area A, area D, area B, Area C, or area A, area B, area D, area C, or area B, area A, area D, area C, or area A, area B, area C, area D.

With reference to the first aspect, in some embodiments, device options of a third device are also displayed on the second interface, and the third device is a device recommended for evaluating the device use requirements of the first device and its associated second device. This can help users expand more and richer usage scenarios, and better experience the HyperTerminal ecosystem.

With reference to the first aspect, in some embodiments, if the third device is a device discovered by the first device, and the device option of the third device is clicked, the first device displays the service options of the second scenario-based service. Among them, the service options of the second scenario-based service may be that some scenario-based service options are added on the basis of the service options of the first scenario-based service, or the service options of the second scenario-based service are re-updated scenario-based services, It is different from the service options of the first scenario-based service. This makes it easy for users to operate.

In combination with the first aspect, in some embodiments, if the third device is not a device discovered by the first device, click on the device option of the third device, and the first device can jump to display the fifth interface, such as the fifth interface can be On the purchase interface of the third device, the fifth interface includes multiple item options, the multiple item options come from multiple supply sources, and the multiple item options include item options indicating the third device. The fifth interface may include an introduction to the functions of the third device, an introduction to usage methods, an introduction to matching usage scenarios, etc., so as to provide users with more convenient services.

With reference to the first aspect, in some embodiments, the third device is a device recommended based on one or more parameters: device type, device feature, product positioning, The purpose of the device, the environment or scene where the device is located, the state of the device, the application that has been running recently, or the service of the first scene, or the use scene that can be predicted and expanded, etc.

After adding a recommended device, the system can also recommend new devices according to the newly formed hyperterminal. Users can choose to add more devices to form a hyper terminal, making the user use scenarios more abundant. After adding more associated devices, the HyperTerminal can provide more and richer scene-based services, improving the HyperTerminal ecosystem.

With reference to the first aspect, in some embodiments, the first scenario-based service is supported by a first atomic service composition, and the first atomic service composition includes at least one first service provided by the first device, and/or the second device At least one second service is provided. Atomic service refers to the smallest capability unit that can run independently. It is a concept of abstract encapsulation of a single function/capability. It can be a hardware service or a software service.

With reference to the first aspect, in some embodiments, the first device may display the first atomized service combination on the seventh interface, refer to the embodiment shown in FIG. 10 . The first atomic service combination may be selected by the user, or the first atomic service combination is an atomic service combination determined according to one or more of the following parameters: the device type and device characteristics of the first device and/or the second device, Either the user's commonly used atomic service combination, or the user's default atomic service combination, or the most suitable atomic service combination obtained from the analysis for the first scenario service, or the usage frequency obtained from the server under the first scenario service The highest atomic service portfolio.

With reference to the first aspect, in some embodiments, the first device runs the first scenario-based service, and/or, the second device associated with the first device runs the first scenario-based service.

With reference to the first aspect, in some embodiments, the third interface displayed by the first device running the first scenario-based service is different from the fourth interface displayed by the second device running the first scenario-based service.

In combination with the first aspect, in some embodiments, if the first scenario-based service is a projection service, the second device is a large-screen device, the fourth interface can be a video or image screen, and the third interface can be a control video or image screen Display the control interface of the function. Reference is made to the embodiment shown in FIG. 18 . In this way, the second device can be used in conjunction with the first device, making full use of the characteristics of the device, and providing more convenience to the user. Certainly, the user may also select the third interface to display the chat interface, and the user uses the first device to chat while using the second device to watch the video. Optionally, the screen content displayed on the third interface and the fourth interface may be the same. Referring to the embodiment shown in FIG. 19 , both the mobile phone and the TV display the same video screen.

In combination with the first aspect, in some embodiments, if the first scenario-based service is a motion monitoring service, the second device is a wearable motion device, the third interface can be a picture of a motion track, and the fourth interface can be a detected user The exercise data includes: exercise speed, exercise distance, exercise time, heart rate. Reference is made to the embodiment shown in FIG. 8 . In this way, the second device can be used in conjunction with the first device, making full use of the characteristics of the device, and providing more convenience to the user. For example, after the user associates the mobile phone with the smart watch, the user puts the mobile phone at home and wears the smart watch for outdoor running. The smart watch has positioning functions and speed measurement functions. The smart watch can continuously upload the positioning information to the cloud server. Send it to the mobile phone, and the user can see his movement track on the mobile phone after returning home.

When the user drags the second control to attract it to the first control from different directions and areas, the effects and services triggered may also be different.

With reference to the first aspect, in some embodiments, in the screen projection scenario, the first device determines that the screen interface of the first device is projected on the second screen according to the relative position of the second control and the first control when the drag operation is released. The position on the device screen.

Embodiments described with reference to FIGS. 15 , 16 , and 17 . If the relative position is that the second control is on the left side of the first control, then the screen interface of the first device is projected on the right side of the screen of the second device. If the relative position is that the second control is on the right side of the first control, then the screen interface of the first device is projected on the left side of the screen of the second device. If the relative position is that the second control is in the middle of the first control, then the full screen of the screen interface of the first device is projected on the screen of the second device.

In another example, for audio devices, such as speakers and earphones, different sound channels can be associated with different orientations. Assuming that the user has two or more speakers, the user drags the icon of the first speaker to fit it from the left area of the phone icon and sticks it together, then the first speaker can provide the audio output of the left channel; the user drags The second speaker icon is pasted from the right area of the mobile phone icon and absorbed together, then the second speaker can provide the audio output of the right channel. The arrangement of other multi-channels is the same.

In some other embodiments, the dragging direction is exchanged between the first control and the second control, and the services or interactive functions executed by them may be different. If the data transmission direction is different, the device as the adsorbing party or as the adsorbed party indicates that the device is the sender or receiver of the data. For example, the data transmission direction is stipulated that the adsorbing device sends data streams to the adsorbed device. For example, when sharing a file, if you press and hold the second control and drag it to attach to the first control, the service triggered is to send the file of the second device to the first device; if you press and hold the first control and drag it to attach On the second control, the triggered service is to send the file of the first device to the second device.

In some other embodiments, the scenario-based service triggered by the device option as the absorbing party or the adsorbed party may also be different. For example, if you press and hold the second control and drag it to attach to the first control, the triggered service is to send the file of the second device to the first device, if you press and hold the first control and drag it to attach to the second control , the triggered service is to cast the screen of the first device to the screen of the second device.

With reference to the first aspect, in some embodiments, the method may further include: the first device detects a third user operation acting on the second interface, the third user operation is used to associate the fourth device with the first device, the second The third user operation includes selecting a third control operation indicating a fourth device. In response to the third user operation, the first device displays a sixth interface, the sixth interface displays the first control, the second control, and the third control are adsorbed together, and the sixth interface also displays a third scene-based service option, The third scenario-based service option is determined according to the first device, the second device, and the fourth device, and the third scenario-based service option is different from the first scenario-based service option. Reference is made to the embodiment shown in FIG. 9 .

With reference to the first aspect, in some embodiments, after the first device runs the first scenario-based service selected by the second user operation, the first device detects a fourth user operation, where the fourth user operation includes combining the first control with the second The two controls operate separately. In response to the fourth user operation, the first device instructs the first scenario-based service selected by the second user operation to stop running.

With reference to the first aspect, in some embodiments, in response to the fourth user operation, the first device stops running the first scenario-based service selected by the second user operation, and/or the second device stops running the service selected by the second user operation. The first scenario-based service.

In combination with the first aspect, in some embodiments, the fourth user operation can be for the user to select the second control, and drag the second control to move away from the first control until it moves out of the specified area, and release the second control operate. The second control returns to the position before it is not connected, and the adsorption state is released. At the same time, the second device is also disconnected from the first device, and the atomization service provided by the first device or the second device is also terminated. Or, when the user clicks the second control in the connected state, or the user presses and holds the second control and the first control, and drags them in opposite directions to separate them, it can also trigger the disconnection of the first device and the second device Connection means that the second control is separated from the first control on the interface, and the second control returns to the position of the unconnected state.

In some embodiments, a one-key disconnection control can also be set on the interface of the first device, and clicking the one-key disconnection control can cause all connected associated devices to disconnect from the first device and restore to the first device There is no status of any associated device connection, and the services provided by each associated device will also be terminated.

With reference to the first aspect, in some embodiments, the first device includes any one of the following: a mobile phone, a tablet computer, a portable/non-portable computer, a personal computer, a smart TV, and the like. The second device includes any of the following: mobile phone, tablet computer, portable mobile computer, desktop personal computer, smart speaker, smart watch, smart bracelet, smart TV, earphones, smart glasses, car machine, smart cockpit, game console, running Machines, spinning bikes, weight scales, body fat scales, water heaters, lamps, air conditioners, blood glucose meters, blood oxygen saturation meters, heart rate meters, AR/VR equipment, cloud hosts/cloud servers, smart wearable devices, smart home devices, printers wait.

With reference to the first aspect, in some embodiments, the above-mentioned first scenario-based service may be a printing service, and the above-mentioned second device is a printer. When the first device and the second device cooperate to run the above-mentioned first scene-based service, the first device can display the printing floating control. The above-mentioned printing suspension control can be suspended and displayed on the user interface of the first device.

That is to say, during the period when the first device is running the first scene service, the first device may always display the above-mentioned printing floating control in a floating manner on the user interface. Wherein, the content displayed on the user interface of the first device is changed, but the printing suspension control can still be suspended and displayed on the user interface. In this way, it is convenient for the user to quickly print the content displayed on the current user interface through the print floating control.

With reference to the first aspect, in some embodiments, the first device detects a fifth user operation on the printing suspension control. The first device may send the first print file to the second device for printing based on the fifth user operation, where the first print file is content displayed on the first device when the first device detects the fifth user operation.

In some embodiments, when the content displayed on the user interface of the first device changes from the first print file to the second print file, the first device may send the second print file to the second print file in response to the operation of the print suspension control. The second device prints.

With reference to the first aspect, in some embodiments, the first device detects a sixth user operation on the printing suspension control. The first device may delete the printing suspension control based on the sixth user operation, and instruct the first scenario-based service to stop running.

It can be seen that the user can directly trigger the first device and the second device to stop running the first scene service through the above-mentioned printing suspension control, so that there is no need to return to the above-mentioned first interface to trigger the first device through the above-mentioned first control and second control . The second device stops running the first scenario-based service.

The above-mentioned user operations for the first device to cooperate with the printer to provide printing services and user operations for stopping the cooperation between the first device and the printer are relatively simple, which can improve user experience.

In a second aspect, the embodiment of the present application provides a device interaction method. Wherein, the first device displays the first interface, the first interface includes a first control and a second control, the first control indicates the device option of the first device, the second control indicates the device option of the second device, and the second device is Devices discovered by the first device. The first device detects a first user operation, and the first user operation is used to associate the first device with the second device. The first device sends a first binding request to the second device according to the first user operation, and receives a first consent message from the second device. The first device displays the second interface according to the first consent message, and the second interface displays service options of the first scenario-based service, and the first scenario-based service is a scenario-based service supported by the first device and the second device in cooperation.

Implementing the method in the second aspect can improve the performance of human-computer interaction. Users can quickly select scene-based services and/or atomized services according to individual needs, and provide specific scene-based services available under the device combination through the user-selected device combination. Service and/or atomic service lists detailed and rich usage scenarios for users, guides users to fully explore more application scenarios by using the functions of HyperTerminal, and achieves the effect of recommending scenario-based services according to device combinations.

With reference to the second aspect, in some embodiments, the first device may display a connection code setting box according to the first user operation, and the connection code setting box is used to set a connection code for binding between the first device and the second device. The first device receives the first connection code input in the connection code setting box, and sends a first binding request to the second device. Wherein, the first consent message includes the first connection code.

In some embodiments, after the second device receives the first binding request, the second device may ask the user whether to allow the second device to cooperate with the first device. If it is detected that the user agrees to the coordinated operation of the second device and the first device, the second device may display a connection code input box to allow the user to input the connection code. When receiving the connection code input by the user in the connection code input box, the second device may send the above-mentioned first consent message. After receiving the first agreement message, the first device may detect whether the connection code contained in the first agreement message is the connection code set in the above connection code setting box. If so, the first device and the second device can complete device binding and establish collaboration, thereby providing users with scenario-based services.

It can be seen from the foregoing embodiments that before the first device and the second device run the first scene service, device binding may be performed first. During the above device binding process, the first device can set a connection code, and the second device can input the connection code for device binding. The above device binding process may ask whether the user to which the second device belongs agrees that the second device cooperates with the first device. The foregoing embodiments can reduce the probability that the user to which the second device belongs is used by another user without consent or knowledge when the user to which the first device belongs is different from the user to which the second device belongs. Moreover, the above verification process of the connection code can ensure the accuracy of the electronic device that is bound with the first device, and improve user experience.

With reference to the second aspect, in some embodiments, in the first interface, the second controls are distributed on a circle whose center is the first control and whose radius is the first length, and the first control and the second control are separated.

The above-mentioned first user operation may be an operation for the user to select the second control, drag the second control to a designated area in a direction close to the first control, and then release the operation of the second control. Or, the first user operation can be that the user selects the first control and the second control at the same time, and drags the first control and the second control to be close to each other until the distance between the first control and the second control is less than a certain distance, and then releases the first control. control and the operation of the second control. The specified area may be a circular area with the first control as the center and a radius equal to the first radius distance. Or, the first user operation can be for the user to select the first control, and drag the first control to move towards the direction close to the second control, until the distance between the first control and the second control is less than a certain distance, then release the first control operation.

In some embodiments, the first control and the second control are also displayed in the second interface. In the second interface, the first control and the second control can be attracted together, indicating that the first device and the second device have established a connection and can work together to provide system-level services. The two device options being attracted together may refer to that the edges of the corresponding icons of the two devices are tangent, or the two device options being adsorbed together may also be that the two device icons are completely or partially overlapped.

The above-mentioned first operation conforms to the user's operating habits, the user's learning cost is low, and the operation is easy to use.

With reference to the second aspect, in some embodiments, the first interface further includes a fourth control, the fourth control is a device option indicating a fifth device, the fifth device is a device discovered by the first device, and the account associated with the fifth device is associated with the The accounts associated with the first device are the same; the fourth control is distributed on a circle with the first control as the center and a radius of the second length, the first control and the fourth control are separated, and the first length is different from the second length.

With reference to the second aspect, in some embodiments, the account associated with the first device is different from the account associated with the second device, or the second device has no associated account.

It can be seen from the above embodiments that the device options indicating cross-account devices (or non-account devices) and device options indicating same-account devices can be displayed on circles with different radii centered on the first control. For example, device options indicating devices of the same account may be displayed on a circle. Device options indicating cross-account devices and/or non-account devices may be displayed on another circle. In this way, users can quickly distinguish which devices are their own and which devices belong to other users.

The aforementioned device with the same account may refer to a device that is associated with the same account as the first device. The above-mentioned cross-account device may refer to a device with a different account associated with the above-mentioned first device. The aforementioned device without an account may refer to a device without an associated account.

In some embodiments, in the case that the above-mentioned second device is a cross-account device or a non-account device, the first device may perform device binding with the second device in response to the above-mentioned first operation. Wherein, if the first device and the second device perform device binding for the first time, the first device and the second device can perform device binding through the above connection code. Exemplarily, the first device may display the above connection code setting box, receive the connection code set by the user, and send a binding request to the second device. After receiving the binding request, the second device may ask the user whether to agree to bind the second device with the first device. If the operation of agreeing to bind the second device with the first device is detected, the second device may display a connection code input box, requiring the user to belong to the connection code that the second device binds with the first device. When receiving the connection code input in the connection code input box, the second device may send a consent message to the first device. The consent message also includes the connection code received in the connection code input box. User operations when the first device is bound to the second device for a non-first time (such as the second, third time, etc.) may be simpler than the above-mentioned user operations when the device is bound for the first time. Exemplarily, in the case that the first device and the second device have performed device binding, the first device may send a binding request to the second device in response to the above-mentioned first user operation. When receiving the above-mentioned binding request, the second device may ask the user whether to agree to bind the second device with the first device. If an operation of agreeing to bind the second device with the first device is detected, the second device may send an approval message to the first device.

In some embodiments, when the above-mentioned first user operation of associating the first device and the second device is received for the first time, the first device and the second device may perform device binding. After the first device and the second device have been bound, the first device can directly establish a connection with the second device when receiving the above-mentioned user operation of associating the first device with the second device, and work together to provide scene-based services .

With reference to the second aspect, in some embodiments, the first interface further includes a fifth control, the fifth control is a device option indicating a sixth device, the sixth device is a device discovered by the first device, and the fifth control and the fourth control in a separate state. The first device detects a seventh user operation, and the seventh user operation is used to associate the fifth device with the sixth device. According to the seventh user operation, the first device instructs the fifth device to request binding with the sixth device, and receives a second consent message for instructing the sixth device to agree to bind with the fifth device. The first device displays the eighth interface according to the second consent message, and in the eighth interface, the fourth control and the fifth control are adsorbed together.

The attachment of the fourth control and the fifth control above can indicate that the first device and the second device have established a connection, and can work together to provide system-level services.

It can be seen from the above embodiments that the core device in the hyperterminal can also respond to user operations to associate other same-account devices in the hyperterminal with cross-account devices, so that the above-mentioned same-account devices and cross-account devices can work together to provide scene-based services . Wherein, before the above-mentioned same-account device and cross-account device cooperate to work, device binding can be performed first.

With reference to the second aspect, in some embodiments, the first device detects a user operation acting on the second control, and removes the second control on the first interface.

With reference to the second aspect, in some embodiments, a recycle bin control is further displayed on the first interface. The first device detects a user operation acting on the recycle bin control, and may display a recycle bin list. One or more device options can be included in the Recycle Bin list. The device indicated by the device option in the recycle bin list may be the device indicated by the device option removed from the first interface. Exemplarily, when the user removes the above-mentioned second control on the first interface, the user can view the recycle bin list through the above-mentioned recycle bin control. A device option indicative of a second device may be included in the recycle bin list. Wherein, in response to operating the device option indicating the second device in the recycle bin list, the first device may display the second control on the first interface again.

With reference to the second aspect, in some embodiments, the second control in the second interface may have a different expression form from the second control in the first interface.

With reference to the second aspect, in some embodiments, a floating control is displayed on the second interface, the floating control includes a first area, and the first area displays service options of the first scenario-based service. The first user operation may be an operation in which the user presses and holds the second control. The second user operation may be an operation of releasing the second control after the user drags the second control to the first area, or the second user operation is the user dragging the second control through the first area toward a direction close to the first control After moving to the specified area, the operation of the second control is released. The specified area may be a circular area with the first control as the center and a radius equal to the first radius distance. Embodiments described with reference to FIGS. 11, 12, 13A, 13B, 13C, and 13D. In this embodiment, a second user operation includes two functions of associating the second device with the first device and selecting the first scenario-based service, which improves user operation efficiency.

With reference to the second aspect, in some embodiments, the floating control includes multiple circular areas, or multiple rectangular areas, or multiple fan-shaped areas, or multiple fan-shaped ring areas, or multiple circular ring-shaped areas, or multiple polygonal area, etc.

With reference to the second aspect, in some embodiments, the second interface includes the first control. The second interface may also not include the first control.

With reference to the second aspect, in some embodiments, after receiving the first consent message from the second device, the first device may cooperate with the second device to run the first scenario-based service. Wherein, the first scenario-based service may be the scenario-based service with the highest probability of being selected by the user among the K1 scenario-based services. The above K1 scenario-based service may be a scenario-based service that can be provided by the cooperation of the first device and the second device. The above K1 is a positive integer. Among K1 scenario-based services, the probability of a scenario-based service being selected by the user can be calculated based on one or more of the following parameters: the frequency of users using this scenario-based service, the user-set rate of this scenario-based service Sorting, environment or scene of the first device and/or the second device, state of the first device and/or the second device, applications recently run by the first device and/or the second device. The aforementioned K1 scenario-based services also include a fourth scenario-based service. The above-mentioned second interface may display service options of the fourth scenario-based service. The user can use the service option of the first scenario-based service to adjust the scenario-based service provided by the first device and the second device from the first scenario-based service to the fourth scenario-based service.

It can be known from the above embodiments that the first device can cooperate with the second device to directly provide the scenario-based service with the highest probability of being selected by the user according to the user operation associated with the first device and the second device. If the scenario-based service desired by the user is not the scenario-based service with the highest probability of being selected by the user as determined by the first device and the second device, the user can select the scenario-based service he wants through the service options of other scenario-based services.

In a third aspect, the embodiment of the present application provides an electronic device, and the electronic device may include: a communication device, a display device, a memory, a processor coupled to the memory, multiple application programs, and one or more programs. The communication device is used for communication, the display device is used for displaying an interface, and computer-executable instructions are stored in the memory. When the processor executes the instructions, the electronic device can realize any function of the first device in the first aspect.

In a fourth aspect, the embodiment of the present application provides an electronic device, and the electronic device may include: a communication device, a display device, a memory, a processor coupled to the memory, multiple application programs, and one or more programs. The communication device is used for communication, the display device is used for displaying the interface, and computer-executable instructions are stored in the memory. When the processor executes the instructions, the electronic device can realize any function of the first device in the second aspect.

In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which a computer program is stored, and the computer program includes executable instructions, and when executed by a processor, the executable instruction causes the processor to perform the following steps: In one aspect, the methods correspond to the operations provided.

In a sixth aspect, an embodiment of the present application provides a computer storage medium, in which a computer program is stored, the computer program includes executable instructions, and when the executable instructions are executed by a processor, the processor executes the following steps: The operations corresponding to the methods provided by the two aspects.

In a seventh aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is run on an electronic device, causes the electronic device to execute any possible implementation manner in the first aspect.

In an eighth aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is run on an electronic device, causes the electronic device to execute any possible implementation manner in the second aspect.

In the ninth aspect, the embodiment of the present application provides a chip system, which can be applied to electronic equipment, and the chip includes one or more processors, and the processors are used to call computer instructions so that the electronic equipment implements the electronic equipment as described in the first aspect. any possible implementation.

In the tenth aspect, the embodiment of the present application provides a chip system, which can be applied to electronic devices, and the chip includes one or more processors, and the processors are used to call computer instructions so that the electronic device implements the electronic device as described in the second aspect. any possible implementation.

In an eleventh aspect, an embodiment of the present application provides a communication system, where the communication system includes a first device and a second device, where the first device implements any possible implementation manner in the first aspect.

Implementing the above aspects provided by this application, users can quickly select scenario-based services and/or atomization services according to personal needs, and provide scenario-based services and/or atomization services available under the device combination through the user-selected device combination The service lists detailed and rich usage scenarios for users, guides users to fully explore more application scenarios by using the functions of the HyperTerminal, and realizes the effect of recommending scenario-based services according to device combinations.

Description of drawings

FIG. 1 is a schematic diagram of a communication system provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a hardware structure of an electronic device provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of the software architecture of the electronic device provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a user interface provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 11 is a schematic diagram of the user interface provided by the embodiment of the present application;

Fig. 12 is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 13A is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 13B is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 13C is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 13D is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 14 is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 15 is a schematic diagram of a user interaction interface provided by an embodiment of the present application;

FIG. 16 is a schematic diagram of a user interaction interface provided by an embodiment of the present application;

FIG. 17 is a schematic diagram of a user interaction interface provided by an embodiment of the present application;

Figure 18 is a schematic diagram of the user interface provided by the embodiment of the present application;

Fig. 19 is a schematic diagram of the user interface provided by the embodiment of the present application;

FIG. 20 is a flow chart of a device interaction method provided in an embodiment of the present application;

FIG. 21 is a schematic diagram of a functional module provided by an embodiment of the present application;

Figures 22A to 22J are schematic diagrams of some user interfaces provided by the embodiments of the present application;

23A to 23D are schematic diagrams of some user interfaces provided by the embodiment of the present application;

Figures 24A to 24D are schematic diagrams of some user interfaces provided by the embodiments of the present application;

Figures 25A to 25H are schematic diagrams of some user interfaces provided by the embodiments of the present application;

26A to 26C are schematic diagrams of some user interfaces provided by the embodiment of the present application;

Figures 27A to 27E are schematic diagrams of some user interfaces provided by the embodiments of the present application;

28A to 28K are schematic diagrams of some user interfaces provided by the embodiments of the present application;

Figure 29A and Figure 29B are schematic diagrams of some user interfaces provided by the embodiment of the present application;

30A to 30C are schematic diagrams of some user interfaces provided by the embodiment of the present application;

31A to 31C are schematic diagrams of some user interfaces provided by the embodiment of the present application;

Figure 32A and Figure 32B are schematic diagrams of some user interfaces provided by the embodiment of the present application;

33A to 33C are schematic diagrams of some user interfaces provided by the embodiment of the present application;

34A to 34O are schematic diagrams of some user interfaces provided by the embodiments of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. Among them, in the description of the embodiments of this application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in the text is only a description of associated objects The association relationship of indicates that there may be three kinds of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently.

Hereinafter, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

The term "user interface (UI)" in the following embodiments of the present application is a medium interface for interaction and information exchange between an application program (application, APP) or an operating system (operating system, OS) and a user. Realize the conversion between the internal form of information and the form acceptable to users. The user interface is the source code written in a specific computer language such as java and extensible markup language (XML). The source code of the interface is parsed and rendered on the electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is the graphical user interface (graphic user interface, GUI), which refers to the user interface related to computer operation displayed in a graphical way. It may be text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, and other visible interface elements displayed on the display screen of the electronic device.

As consumption upgrades, users own more and more smart terminal devices. In some usage scenarios, users often need two or more terminal devices to link or work together to provide certain services and realize certain functions, which can also be called cross-device services. For example, mobile phones and smart screens cooperate to realize video projection; mobile phones share files or pictures with other devices. In practical applications, it will be found that when multiple devices are turned on to work together, the user operation is cumbersome, the entry distribution is scattered, the human-computer interaction performance is not good, and the user experience is poor.

In some implementations, when multiple devices work together, only one default service is provided, but one service cannot meet the requirements of diverse scenarios. For example, when a mobile phone and a smart screen are coordinated, multi-screen collaborative services can be provided. However, the cooperation of mobile phones and smart screens can also provide more services, such as services in application scenarios such as home theater, motion-sensing games, and enjoying karaoke. However, there is no entry for service options corresponding to various scenarios on the mobile phone. If the user wants to switch to a service in another application scenario, he may need to exit the current interface, open a new application, and enable the corresponding service, which is cumbersome for the user.

The present application provides a device interaction method, which can be applied in a communication system composed of multiple devices. In some embodiments, the communication system can also be called a hyper terminal or a hyper terminal system. The purpose of this method is to explore a new way of human-computer interaction, provide a more intuitive and convenient device interaction method, meet the user's needs for coordinating multiple devices, and provide users with access to service options corresponding to various scenarios.

In the following embodiments of the present application, a plurality of terminal devices may form a "hyper terminal" (or called a hyper terminal system). "Super terminal" refers to the ability to integrate the capabilities of multiple terminals through distributed technology, store them in a virtual hardware resource pool, and provide external services based on the unified management, scheduling, and integration capabilities of each terminal according to business needs. Fast connection, capability mutual assistance, and resource sharing are realized between different terminals.

The hyper terminal may include a core device/central device, and a cooperative device/associated device. The core device is the control center of the super terminal system, carrying functions such as device management, service management, and service scheduling. The core device is connected with one or more associated devices to form a HyperTerminal. The associated device may be an electronic device located on the same communication network as the core device (such as a mobile phone), for example, an electronic device located on the same Wi-Fi network. The associated device can also be an electronic device whose login account is the same as the core device login account (such as a Huawei account), or an electronic device whose login account and the core device login account belong to the same group (such as the same family account). The associated device can also be an electronic device that establishes a trusted relationship (or pairing) with the core device in other ways, such as an electronic device that has been paired with Bluetooth, an electronic device that has connected to a hotspot shared by a mobile phone, or a mobile phone that has been connected to the electronic device. The hotspot of the device, or the mobile phone has established a Wi-Fi P2P connection with the electronic device, etc. The embodiment of the present application does not limit the specific manner of associating the foregoing associated device with the core device.

Different associated devices are connected to the core device and work together to achieve different functions. For example, the core device mobile phone and associated device laptop form a super terminal, which can realize the function of projecting the screen of the mobile phone interface to the laptop; the core device mobile phone and related device smart speakers form a super terminal, which can realize the function of mobile phone remote control smart speaker ,etc.

It should be noted that, in the following embodiments of the present application, "hyperterminal" is used to collectively refer to a core device and a communication system composed of one or more associated devices that have an associated relationship with the core device. That is, "hyperterminal" is used to describe a collection of electronic devices having the above-mentioned association relationship. It can be understood that the "super terminal" is only an exemplary vocabulary, and it can also be replaced by other words, such as "smart collaboration", "multi-device collaboration", etc., which does not constitute a specific description of the communication system in the embodiment of the present application. limited. This application also does not impose any limitation on the device types of the core device and associated devices.

The electronic device in this application can specifically be a mobile phone, a tablet computer, a notebook computer, a personal computer (PC), a smart TV (also called a smart screen, a large screen, etc.), or a smart watch, a smart bracelet, etc. The wearable device is not limited in this application. For example, with the mobile phone as the core device/central device, the mobile phone can cooperate with one or more other associated devices (such as laptops, PCs, tablets, smart watches, etc.) to provide consumers with multi-functional, personalized, more interesting and more convenient service.

The device interaction method provides a unified entrance for the interaction between the core device and multiple associated devices, that is, the hyper terminal interface, and the user can trigger any two or more electronic devices to form a hyper terminal in the hyper terminal interface. And it can support that when multiple devices initiate collaboration, the HyperTerminal system can judge the usage scenario according to the device type, device characteristics, product positioning, common functions, device status, current running applications, and current device environment of each device. And provide users with one or more scenario-based service options based on usage scenarios. Different scenario-based service options are the entrances of corresponding services in different application scenarios, which solves the problem that users cannot be provided with rich usage scenarios in the case of different device combinations. In addition to supporting electronic devices to intelligently recommend scene-based service options corresponding to various application scenarios for users, this application can also recommend different atomic service/function combinations or interactive functions according to different application scenarios or different device types, features, product positioning, etc. option. In addition, in order to expand more scenarios with better user experience, it can also recommend matching equipment for users.

Implementing the method provided by this application can enhance the performance of human-computer interaction, and users can quickly select scene-based services and/or atomized services according to individual needs, without cumbersome decentralized settings, thus solving the problem of setting interactive functions between electronic devices The distribution of entrances is scattered, the settings are cumbersome, and it is impossible to list rich usage scenarios for users. Through the device combination selected by the user, the available scenario services and/or atomic services under the device combination are provided in a targeted manner, and detailed and rich usage scenarios are listed for the user, guiding the user to use the functions of the hyper terminal to fully explore more application scenarios, realizing the effect of recommending scenario-based services based on device combinations. Moreover, a more intuitive and concise user interface is presented to the user, the intuitiveness, readability and operability of the user interface are improved, user operations are simplified, and the electronic device can quickly respond to open different application services, thereby improving the user experience.

The communication system 10 provided by the embodiment of the present application is introduced below.

FIG. 1 shows a communication system 10 provided by an embodiment of the present application.

As shown in FIG. 1, the communication system 10 includes a plurality of terminal devices, and these terminal devices may include the illustrated mobile phone 101, smart watch 102, notebook computer 103, tablet computer 104, smart bracelet 105, earphone 106, smart TV ( It can also be called a smart screen, a large screen, etc.) 107, a smart speaker 108, etc. In addition to the above-mentioned devices shown in FIG. 1, the communication system 10 can also include other types of electronic devices, such as a personal computer (personal computer, PC), desktop computer, laptop computer, handheld computer, augmented reality (augmented reality, AR) equipment, virtual reality (virtual reality, VR) equipment, artificial intelligence (artificial intelligence, AI) equipment, car machine, game machine , treadmill, cloud host/cloud server, other smart wearable devices, etc., can also include Internet of Things (IOT) devices or smart home devices such as smart water heaters, smart lamps, smart air conditioners, smart scales, etc., this The application does not place any restrictions on the type of equipment. In this embodiment, a terminal device may also be referred to simply as a terminal, and a terminal device is generally an intelligent electronic device that can provide a user interface, interact with a user, and provide a service function for the user.

In the communication system 10, the mobile phone 101 serves as a central device/core device, and can establish communication connections with other cooperative devices/associated devices to form a "super terminal". "Super terminal" refers to the ability to integrate the capabilities of multiple terminals through distributed technology, store them in a virtual hardware resource pool, and provide external services based on the unified management, scheduling, and integration capabilities of each terminal according to business needs. Fast connection, capability mutual assistance, and resource sharing are realized between different terminals. For example, the mobile phone 101 and the notebook computer 103 form a super terminal, which can realize the function of projecting the interface of the mobile phone 101 to the notebook computer 103; ,etc.

It should be noted that, in the following embodiments of the present application, "hyperterminal" is used to collectively refer to the system composed of the mobile phone 101 as the core device and the associated devices having the above-mentioned association relationship with the mobile phone 101 . That is, "hyperterminal" is used to describe a collection of electronic devices having the above-mentioned association relationship. It can be understood that the "super terminal" is only an exemplary vocabulary, and it can also be replaced by other words, such as "smart collaboration", "multi-device collaboration", etc., which does not constitute a specific description of the communication system in the embodiment of the present application. limited.

The communication connection between each terminal device in the communication system 10 may be a wired connection or a wireless connection, which is not limited in this embodiment. Data or instructions can be transmitted between terminals through the established communication connection.

The communication link may be a short-range communication link. For example, a wired connection, such as a universal serial bus (uniersalserialbus, USB) connection, a high definition multimedia interface (high definition multimedia interface, HDMI) connection, a display port (display port, DP) connection, and the like. Or wireless connection, such as Bluetooth (bluetooth, BT) connection, wireless fidelity (wi-fidelity, Wi-Fi) connection, hotspot connection, near field communication (near field communication, NFC), ZigBee, etc., to realize the communication between each terminal Communicate without an account or with a different account. The wireless connection is not bound by wires, and the user has a higher degree of freedom of movement. The embodiment of the present application does not limit the type of the communication connection. A Bluetooth (bluetooth, BT) module and/or a wireless local area network (wireless local area networks, WLAN) module may be configured in the terminal device. Among them, the Bluetooth module can provide one or more Bluetooth communication solutions including classic Bluetooth (Bluetooth 2.1) or Bluetooth low energy (Bluetooth low energy, BLE), and the WLAN module can provide wireless fidelity point-to-point connections (wireless fidelity peer-to-peer, Wi-Fi P2P), wireless fidelity local area networks (wireless fidelity local area networks, Wi-Fi LAN) or wireless fidelity software access point (wireless fidelity software access point, Wi-Fi softAP) One or more WLAN communication solutions. In some embodiments, Wi-Fi P2P refers to allowing devices in a wireless network to connect to each other in a point-to-point manner without going through a wireless router.

In the system, it may also be called wireless fidelity direct (Wi-Fi direct). Devices that establish a Wi-Fi P2P connection can exchange data directly through Wi-Fi (must be in the same frequency band) without connecting to a network or hotspot, and realize peer-to-peer communication, such as transferring files, pictures, videos and other data. Compared with Bluetooth, Wi-Fi P2P has the advantages of faster search speed and transmission speed, and longer transmission distance.

The communication link can also be a long-distance communication link. For example, each terminal logs in the same account to realize connection and communication through the Internet. Multiple electronic devices in the communication system 10 may also log in to different accounts, but are connected through binding. For example, the mobile phone 101 and the smart watch 102 can log in to different accounts, and the mobile phone 101 binds the smart watch 102 with itself in the device management application, and then connects through the device management application.

In addition, multiple terminal devices in the communication system 10 may also be connected and communicate in combination with any of the above methods, which is not limited in this embodiment of the present application. For example, the communication connection between the mobile phone 101 and the notebook computer 103 may be a combination of multiple connections. For example, the mobile phone 101 or the notebook computer 103 establishes a connection with a router through Wi-Fi to access the network, or communicates with a base station through a cellular signal. A connection is established to access the network, and the mobile phone 101 and the notebook computer 103 can communicate through the network. For example, the mobile phone 101 sends the information to the cloud server through the network, and the cloud server then sends the information to the notebook computer 103 through the network.

In some embodiments, when two terminal devices are mutually trusted devices, for example, the two terminal devices have been matched or connected before, and when they are to be connected again, the two terminal devices will automatically establish a communication connection, and then For data interaction, the user does not need to manually connect or match again, saving time and effort.

Each terminal device in the communication system 10 can carry

system,

operating system, HarmonyOS or other types of operating systems, the operating systems of each terminal device in the communication system 10 may be the same or different, and this application does not limit this. In some embodiments, each connected terminal in the communication system 10 is equipped with HarmonyOS, so the communication system 10 may be called a HarmonyOS super virtual device (super virtual device), or a HarmonyOS super terminal.

In some embodiments, a hyper terminal application is installed on the mobile phone 101 or other terminal devices, which is convenient for users to manage each terminal device.

For example, a smart watch can detect the user's motion data, such as the number of steps taken, running time, swimming time, etc., and synchronize the user's motion data to the treadmill and/or mobile phone. Similarly, treadmills and/or mobile phones can also synchronize detected user data to smart watches. Multiple terminals can be used together to achieve more accurate detection of the user's motion data.

In this embodiment, the communication system 10 may be based on the currently connected device type and/or device features, product positioning, commonly used functions and other information, or according to the detected environment where the user is currently located, or the operating status of the device, etc. , to analyze the usage scenario of the current user.

In this embodiment, the communication system 10 can be based on the type of device in the system, device features, product positioning, device usage, collaborative services that the device can provide, the current environment of the device, the operating status of the device, the application recently used by the user, the current One or more scenario-based service options are obtained after comprehensive consideration of running applications, location information input by users, etc., and displayed on the user interface, so that users can quickly start related services or functions.

In some embodiments, when listing the scenario-based service options, the scenario service providing module may calculate the user's latest use frequency of different collaborative services, or detect the environment where the current device is located, or the user's recently run application, etc. For the probability of each scenario-based service option being selected, the scenario-based service option that is estimated to have a higher probability of being selected is displayed first in the scenario-based service option area.

In some embodiments, the communication system 10 can also analyze the current scene based on the types of devices that currently make up the system, and list and recommend other devices suitable for the current scene to the user, so as to help the user obtain a better scene experience; Expanded usage scenarios, and then list and recommend other devices to the user. If the user adopts the device recommended by the system, more and richer usage scenarios can be further expanded.

In some embodiments, the communication system 10 can also be based on the current device combination, the queried atomic services that can be provided by each device, and then based on the scene service actually selected by the user, or based on the user's common service combination, or the user-set The default service combination, or the service combination that is most suitable for the scenario based on life cognition, or the service combination that is most frequently used in the scenario is obtained from the server, etc., and the atomic service combination in the scenario is automatically set, saving the user's cumbersome Choose an action. The communication system 10 can also support the user to personalize the atomic service combination, and the user can manually modify the atomic service combination according to personal preferences or actual conditions, and adjust a certain atomic service to be provided by other devices.

Please understand the description of specific embodiments in conjunction with the following embodiments.

It should be noted that the communication system 10 shown in FIG. 1 is only used to assist in describing the technical solution provided by the embodiment of the present application, and does not limit other embodiments of the present application. In an actual business scenario, the communication system 10 may include more or fewer terminal devices, and this application does not make any restrictions on the type, quantity, and connection mode of the terminal devices in the communication system 10 .

The exemplary electronic device 100 provided by the embodiment of the present application is introduced below.

FIG. 2 is a schematic diagram of a hardware structure of an electronic device 100 provided in an embodiment of the present application. The exemplary electronic device 100 provided in the embodiment of the present application may be, but not limited to, a mobile phone, a notebook computer, a tablet computer (portable android device, PAD), a smart bracelet, a smart watch, a headset, a personal computer (personal computer, PC), a smart phone TVs, smart speakers, etc., can also be desktop computers, laptop computers, handheld computers, augmented reality (augmented reality, AR) equipment, virtual reality (virtual reality, VR) equipment, artificial intelligence (artificial intelligence, AI) equipment , car machines, game consoles, treadmills, cloud hosts/cloud servers, other smart wearable devices, etc., or Internet of Things (IOT) devices or smart home devices such as smart water heaters, smart lamps, smart air conditioners, smart weight Other types of electronic equipment such as scales, this application does not impose any restrictions on the type of equipment.

It can be understood that, the structure shown in this embodiment does not constitute a specific limitation on the first device. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the illustration, some hardware structures may be added or reduced, some components may be combined, some components may be split, or different components layout. The illustrated components can be realized in hardware, software or a combination of software and hardware.

2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, and an antenna 1. Antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

The processor 110 is generally used to control the overall operation of the electronic device 100 and may include one or more processing units. For example: the processor 110 may include a central processing unit (central processing unit, CPU), an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor ( image signal processor (ISP), video processing unit (video processing unit, VPU), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processing device (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of electronic devices can be realized through NPU, such as: image recognition, face recognition, speech recognition, text understanding, etc.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, serial peripheral interface (serial peripheral interface, SPI) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device.

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface. Both I2S interface and PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing audio.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device. The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device, and can also be used to transmit data between the electronic device and peripheral devices. This interface can also be used to connect other electronic devices, such as mobile phones, PCs, smart TVs, etc. The USB interface may be USB3.0, which is compatible with high-speed display port (DP) signal transmission, and can transmit video and audio high-speed data.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device. While the charging management module 140 is charging the battery 142 , it can also provide power for electronic devices through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide the second generation (2th generation, 2G) network/third generation (3th generation, 3G) network/fourth generation (4th generation, 4G) network/fifth generation network applied on the electronic device 100. (5th generation, 5G) network and other wireless communication solutions. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 may implement a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1. The camera 193 may include, but not limited to, a traditional color camera (RGB camera), a depth camera (RGB depth camera), a dynamic vision sensor (dynamic vision sensor, DVS) camera, and the like. In some embodiments, camera 193 may be a depth camera. The depth camera can collect the spatial information of the real environment.

In some embodiments, the electronic device 100 can take pictures of users through the camera 193, recognize different users by face, correspondingly enable different user accounts, and store the information of different users in the accounts of different users, which can ensure that the accounts of different users are not confused, and further protect users. data privacy.

In some embodiments, the camera 193 may collect hand images or body images of the user, and the processor 110 may be configured to analyze the images collected by the camera 193 to recognize hand motions or body motions input by the user. For example, the camera 193 can be used to recognize the user's hand movements to implement user gesture control.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the electronic device by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device.

In some embodiments of the present application, the internal memory 121 may be used to store application programs of one or more applications, and the application programs include instructions. When executed by the processor 110, the application program causes the electronic device 100 to generate content for presentation to the user. Exemplarily, the application may include an application for managing the electronic device 100, such as a game application, a meeting application, a video application, a desktop application or other applications, and the like.

The internal memory 121 may include one or more random access memories (random access memory, RAM) and one or more non-volatile memories (non-volatile memory, NVM).

Random access memory has the characteristics of fast read/write speed and volatility. Volatile means that once the power is turned off, the data stored in RAM will disappear. Usually, the static power consumption of the random access memory is extremely low, and the operating power consumption is relatively large. The data in RAM is memory data, which can be read at any time and disappears when the power is turned off.

Non-volatile memory has the characteristics of non-volatility and stable data storage. Non-volatile means that the stored data will not disappear after the power is turned off, and the data can be saved for a long time without power. The data in NVM includes application data, which can be stored stably in NVM for a long time. Application data refers to the content written during the running of an application or service process, such as photos or videos obtained by a camera application, text edited by a user in a document application, and so on.

Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (synchronous dynamic random access memory, SDRAM), double data rate synchronous Dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM, such as the fifth generation DDR SDRAM is generally called DDR5SDRAM), etc.

The non-volatile memory may include magnetic disk storage, flash memory, and the like.

The disk storage device is a memory with a disk as the storage medium, which has the characteristics of large storage capacity, high data transmission rate, and long-term storage of stored data.

According to the operating principle, flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. According to the potential order of storage cells, it can include single-level storage cells (single-level cell, SLC), multi-level storage cells (multi-level cell, MLC), triple-level cell (TLC), quad-level cell (QLC), etc., can include universal flash storage (English: universal flash storage, UFS) according to storage specifications , embedded multimedia memory card (embedded multi media Card, eMMC), etc.

The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other running programs, and can also be used to store data of users and application programs.

The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be loaded into the random access memory in advance for the processor 110 to directly read and write.

The external memory interface 120 can be used to connect an external non-volatile memory, so as to expand the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and video are stored in an external non-volatile memory.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. The electronic device can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or a voice message, it can listen to the voice by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device may be provided with at least one microphone 170C. In some other embodiments, the electronic device can be provided with two microphones 170C, which can also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device can also be equipped with three, four or more microphones 170C to realize sound signal collection, noise reduction, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The electronic device 100 may include one or more keys 190 , and these keys 190 may control the electronic device 100 and provide a user with access to functions on the electronic device 100 . The key 190 may be in the form of a mechanical case such as a button, a switch, or a dial, or may be a touch or near-touch sensing device (such as a touch sensor). The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 . The keys 190 may include a power key, a volume key and the like.

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the electronic device 100 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging status, the change of the battery capacity, and can also be used to indicate messages, notifications and the like.

The electronic device 100 may also include other input and output interfaces, and other devices may be connected to the electronic device 100 through a suitable input and output interface. Components may include, for example, audio/video jacks, data connectors, and the like.

The electronic device 100 is equipped with one or more sensors, including but not limited to a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, Temperature sensor 180J, touch sensor 180K, ambient light sensor 180L, bone conduction sensor 180M, etc.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed. In some embodiments, touch operations acting on the same touch position but with different touch operation durations may correspond to different operation instructions. For example: when a touch operation whose duration is shorter than the first time threshold acts on the pressure sensor 180A, a confirmation instruction is executed. When a touch operation whose duration is greater than or equal to the first time threshold acts on the pressure sensor 180A, a power on/off command is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device about three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of shaking of the electronic device, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the opening and closing of the clamshell can be detected according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object nearby. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device may reduce the performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention uses a layered architecture

The system is taken as an example to illustrate the software structure of the electronic device 100 .

FIG. 3 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the

The system is divided into four layers, from top to bottom are application layer, application framework layer, Android runtime (

runtime) and system libraries, and the kernel layer.

The application layer can consist of a series of application packages.

As shown in Figure 3, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and settings. Wherein, the font size, thickness and the like can be set in the setting application.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 3, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.

Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

Telephony Manager is used to provide communication functions. For example, the management of call status (including connected, hung up, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.

Runtime includes core library and virtual machine.

The runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

A system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

The above introduction to the software architecture of the electronic device 100 is only an example, and it should be understood that the software architecture shown in the embodiment of the present invention does not constitute a specific limitation to the present application. In some other embodiments of the present application, the software architecture of the electronic device 100 may include more or fewer modules than shown in the figure, or combine some modules, or split some modules, or arrange different architectures. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The technical solutions of the present application will be described below in conjunction with the relevant schematic user interfaces provided by some embodiments.

After the core device (such as a mobile phone) and the associated device (such as a smart watch) establish a communication connection, the core device and the associated device can send instructions, data, etc. to each other through the communication connection. The communication connection may be a USB connection, a Bluetooth connection, a Wi-Fi P2P connection, etc., which is not limited in this embodiment.

In the following embodiments of this application, the mobile phone is used as the core device, and other devices such as smart watches, smart TVs, etc. are connected to the mobile phone as associated devices to form a super terminal. This example does not describe the core device in practical applications. and the device type of the associated device constitute no restrictions.

The core device and the associated device may be logged in with the same account or different accounts, which is not limited in this embodiment.

The core device is the control center of the super terminal system, carrying functions such as device management, service management, and service scheduling.

The associated device may be an electronic device located on the same communication network as the core device (such as a mobile phone), for example, an electronic device located on the same Wi-Fi network. The associated device can also be an electronic device whose login account is the same as the core device login account (such as a Huawei account), or an electronic device whose login account and the core device login account belong to the same group (such as the same family account). The associated device can also be an electronic device that establishes a trusted relationship (or pairing) with the core device in other ways, such as an electronic device that has been paired with Bluetooth, an electronic device that has connected to a hotspot shared by a mobile phone, or a mobile phone that has been connected to the electronic device. The hotspot of the device, or the mobile phone has established a Wi-Fi P2P connection with the electronic device, etc. The embodiment of the present application does not limit the specific manner of associating the foregoing associated device with the core device.

It can be understood that, in the following example interface provided by the embodiment of the present application, the core device is mainly used as the mobile phone to display the relevant user interface, but this does not constitute a limitation to the present application. The following sample interfaces can also be transferred to other types of devices, as long as they are based on the same inventive idea provided by this application, they are all within the protection scope of this application.

It can be understood that each user interface described in the following embodiments of the present application is only an example interface, and does not limit the solution of the present application. In other embodiments, different interface layouts may be used in the user interface, more or less controls may be included, and other functional options may be added or reduced, as long as it is based on the same inventive idea provided by this application, it is within the scope of protection of this application Inside.

Referring to the embodiment shown in FIG. 4, FIG. 4 shows that the user can enter the hyperterminal interface by pulling down the control center interface.

As shown in FIG. 4 , the user interface 400 is a schematic diagram of a main interface (or called a main screen interface, etc.) displayed by the mobile phone. The user interface 400 may include a top status bar 401 , a desktop 402 , and a bottom tray bar 403 . Wherein, the top status bar 401 includes a mobile signal indicator, a wireless network signal indicator, a power indicator, a time indicator and the like. Icons of one or more application programs, application function components, and the like may be displayed on the desktop 402 . The bottom tray bar 403 can display icons of commonly used applications, so that users can quickly open these applications. In the example of the user interface 400 , the bottom tray bar 403 displays icons for phone, text message, address book, and camera applications.

As shown in FIG. 4 , the user can call out the control center interface through user operation 404 . The user operation 404 shown in FIG. 4 may be a touch operation in which the user slides down from the top of the display interface. In some other embodiments, the user operation 404 may also be a touch operation in which the user performs an upward slide from the bottom of the display interface.

After the mobile phone detects the user operation 404 , it displays the control center interface, as shown in the user interface 410 . Multiple functional controls may be displayed on the user interface 410, or may also be called cards or modules. For example, the date and time display control 411 shown in the user interface 410, the setting control 412, the shortcut setting card 413, the hyper terminal function card 414, the music card 415, the desk lamp shortcut control card 416, the smart screen shortcut control card 417, etc.

Wherein, the current time and date are displayed in the date and time display control 411 . The setting control 412 can be used for the user to quickly enter the setting interface. The shortcut setting card 413 includes multiple controls, which can be used by the user to quickly turn on/off or adjust corresponding functions, such as quickly turning on/off Wi-Fi network, Bluetooth, mobile network, sound, alarm clock, and quickly adjusting the brightness of the screen display, etc. . One or more icons of devices are displayed in the hyperterminal function card 414, and these devices can include devices that have been connected with the mobile phone to form a hyperterminal, or devices that have not yet established a connection detected by the mobile phone, and devices that have not yet established a connection It may be a device that has been paired before, or a device that has not been paired yet, which is not limited in this embodiment. The user clicks the hyperterminal function card 414, and the mobile phone can display the user interface 500 shown in FIG. 5 . The music card 415 can display information such as currently played music and singers, and also includes controls such as previous song, play/pause, next song, and song list. The desk lamp shortcut control card 416 can be used for the user to quickly turn on or off the desk lamp. The smart screen shortcut control card 417 can be used for the user to quickly turn on or turn off the smart screen.

In other embodiments, the user can also enter the hyperterminal interface by selecting the hyperterminal option in the setting page, that is, the user operation 404 can include a series of operations, for example, the user clicks the "settings" application icon on the main interface. , and the operation of selecting the corresponding function option in the "Settings" application interface, etc. Or the user can also enter the HyperTerminal interface from the negative screen of the mobile phone, pull up the toolbar, pull down the notification bar, pull left/right shortcut icon bar, desktop card or an application (such as a smart home application), etc. The embodiment does not limit the way of entering the hyper terminal interface.

The user clicks the hyperterminal function card 414, and the mobile phone can display the user interface 500 shown in FIG. 5 . Fig. 5 is the device interaction interface related to the HyperTerminal function.

In some embodiments, the mobile phone can turn on the wireless communication function, search for electronic devices around, and display the names of the electronic devices found in the device list. Then the user selects the electronic device to be interacted with according to the name in the device list, and then initiates a connection to the electronic device. In some embodiments, when the user has no special remarks, the name of the electronic device displayed in the device list of the mobile phone is usually the type and/or model of the electronic device, such as HUAWEI Vision, HUAWEI MateBook, sometimes two or For two or more electronic devices with the same name, the user cannot determine which electronic device corresponds to each.

The embodiment of the present application provides a brand-new interactive interface, which can provide users with an easier-to-understand interface and a more humanized and friendly interactive experience, such as the super terminal user interface 500 shown in FIG. 5 .

The user interface 500 may display a hyperterminal title bar 501 , a return control 502 for returning to a previous page, a prompt description 503 , a device connection status icon 504 , and the like.

In the prompt description 503, the prompt text "drag the device to the local machine to establish collaboration" is displayed.

The device connection status diagram 504 may be presented as a circular diagram, and a mobile phone icon 505 is displayed in the center of the device connection status diagram 504, indicating that the core device is a mobile phone. The device icons and names surrounding the mobile phone are the surrounding devices searched by the mobile phone, such as the smart watch icon 506 shown in Figure 5, the Sound X icon, the Vision icon, the FreeBuds icon, the smart bracelet icon, the MatePad icon, and the MateBook icons etc.

It can be noted that, in the device connection state diagram 504 , the layout relationship between the core device and the associated device can be visually indicated. For example, the relative position of the associated device icon and the core device icon displayed in the device connection status diagram 504 can indicate the positional relationship between the associated device and the core device in real space, such as relative orientation information, relative distance information, etc., which is helpful for users Quickly find and distinguish individual electronic devices. For example, the mobile phone searches for two devices named smart watches, and the user can distinguish the device to be selected according to the relative position (relative orientation or distance) of the logo and the mobile phone in the super terminal interface of the mobile phone. For example, in the real space, smart watch A is located on the left side of the mobile phone, and smart watch B is located on the right side of the mobile phone. In the device connection status icon, the user can distinguish that the smart watch icon on the left of the mobile phone icon represents smart watch A. The smart watch icon to the right of the phone icon represents smart watch B.

In some embodiments, in the device connection state diagram 504, the distances between different associated devices and the core device mobile phone may also be different, and the distance between the associated device icon and the core device icon in the interface may indicate the distance in real space Far and near, for example, the farther the associated device is from the core device in the real space, the farther the distance between the associated device icon and the core device icon displayed on the interface will be. This embodiment does not limit the specific technology for realizing this function. For example, the mobile phone can obtain the orientation and the measured distance of the directional information carried by the associated device based on ultra wide band (UWB) technology, and obtain the actual distance between the associated device and the mobile phone. The spatial mapping relationship under the space.

In some other embodiments, the distance between the icon of the associated device and the icon of the core device in the interface may indicate the signal strength of the associated device detected by the core device, such as the Bluetooth signal strength or the WiFi signal strength. The stronger the signal strength of the associated device, the closer the distance between the associated device icon and the core device icon displayed on the interface.

The present application does not limit the layout, expression form and indication meaning of the device connection status diagram 504 . In other embodiments, other layouts or presentation forms may also be used to indicate the detected associated devices.

In some embodiments, the mobile phone can automatically detect surrounding associated devices periodically. For example, the mobile phone can display an interface in the form of a radar scan, indicating that the mobile phone is currently performing detection tasks periodically. Alternatively, the phone can respond to a user action on the refresh control and then detect surrounding associated devices. For example, a mobile phone can use the wireless communication function to broadcast query messages to search for associated devices around the mobile phone. After receiving the query message from the mobile phone, the attached electronic device can return a response message to the mobile phone, and the response message can carry the media access control address (media access control address, MAC address), device type, name, etc. of the electronic device. information.

The mobile phone can determine whether the electronic device is associated with the mobile phone according to the response message of the electronic device, that is, whether the electronic device is an associated device of the mobile phone. For example, according to the MAC address of the electronic device carried in the response message, the mobile phone can query the account management server for the account logged in by the electronic device to determine whether the account logged in by the electronic device is the same as the account logged in by the mobile phone, or whether it belongs to the same group. account. Alternatively, the mobile phone may query the router of the Wi-Fi network where the mobile phone is located according to the MAC address of the electronic device carried in the response message, whether the electronic device is located in the same Wi-Fi network as the mobile phone. Or, the mobile phone determines whether the electronic device has established a trusted relationship with the mobile phone according to the historical records stored by itself. The embodiment of the present application does not limit the specific method for determining the associated device of the mobile phone.

The above-mentioned wireless communication functions include but are not limited to WLAN, radio frequency identification (radio frequency identification, REID), Wi-Fi P2P, hotspot, infrared, ultrasonic, Bluetooth, ZigBee, UWB and other wireless communication functions. No limit.

Understandably, many wireless communication technologies support a positioning function. For example, the positioning function based on Wi-Fi, the positioning function based on Bluetooth iBeacon, the positioning function based on the signal angle of arrival (arrival of angle, AOA) of Bluetooth version 5.1, the positioning function based on UWB, etc., are not limited in this embodiment. Among them, the positioning function based on Wi-Fi and the positioning function based on Bluetooth iBeacon can measure the distance between the mobile phone and other electronic devices. The AOA positioning function based on Bluetooth 5.1 and the UWB-based positioning function can measure the distance between the mobile phone and other electronic devices, as well as the direction of other electronic devices relative to the mobile phone.

After the mobile phone searches the surrounding associated devices, it can further use the positioning capability of the wireless communication function to measure the relative direction and/or relative distance between the associated device and the mobile phone in real space, and then display the device connection status diagram as shown in Figure 5 Show 504. The layout of the icons (and names) used to represent each electronic device in the device connection state diagram 504 may be consistent with the layout of each electronic device's position in real space. For example, in the embodiment provided by this application, Fig. 1 shows a plurality of associated devices (such as smart watch 102, notebook computer 103, tablet computer 104, smart bracelet 105, earphone 106, smart TV 107, smart speaker 108) and mobile phone The positional relationship between 101 (the distance relationship is not shown) is in one-to-one correspondence with the layout of each associated device icon and mobile phone icon shown in the device connection state diagram 504 in FIG. 5 . For example, as shown in FIG. 1 , the smart watch 102 is located directly in front of the mobile phone 101 . Correspondingly, the device connection status diagram 504 in FIG. 5 shows that the smart watch icon 506 is directly above the mobile phone icon 505 .

The associated device icons shown in Figure 5 are floating around the core device, and are not adsorbed, overlapped or attached to the core device icon, indicating that these associated devices have not yet established a connection with the core device. That is to say, in the process of the core device detecting which associated devices are around, the mobile phone and each associated device need to enable the wireless communication function, but it is not required that the mobile phone and each associated device have established a wireless connection.

If the user wants an associated device and the core device mobile phone to form a hyper terminal to achieve some collaborative functions, in one implementation, the user can select the icon of the associated device on the hyper terminal interface shown in Figure 5, and drag The icon corresponding to the associated device is close to the icon of the core device, refer to the user interface 600 shown in FIG. 6 .

Assume that before the devices are coordinated, the distance between the associated device icon and the core device icon is the first distance. When the associated device icon is dragged close to the core device icon and the distance between them is less than or equal to the second distance, the user releases the finger. The associated device icon can be adsorbed around the core device icon, refer to the user interface 700 shown in FIG. 7 .

As shown in FIG. 6 , if the user wants the smart watch and the mobile phone to form a super terminal and work together, the user can hold down the smart watch icon 506 and drag the smart watch icon 506 close to the mobile phone icon 505 . In the user interface 600, a prompt description 602 "smart watch has been selected" may also be displayed to illustrate the selection status of the current user.

When the distance between the smart watch icon 506 and the mobile phone icon 505 is less than or equal to the second distance, or the smart watch icon 506 enters a circular area with the center of the mobile phone icon 505 as the center and a radius of the first radius distance R, the user loosens the Open your finger to display the device connection status diagram 701 shown in FIG. In the user interface 700, a prompt description 703 "cooperated with the smart watch" may also be displayed to illustrate the current connection status of the hyper terminal.

Of course, if the smart watch has established a connection with the mobile phone before opening the super terminal interface, then when the mobile phone detects that the connection has been established, it will directly display the interface where the smart watch icon 702 is attached to the edge of the mobile phone icon 505, expressing that the smart watch In the situation that the connection has been established with the mobile phone, the user interface 700 shown in FIG. 7 is shown.

It can be understood that being able to establish a communication connection with the mobile phone is a necessary but not sufficient condition for the associated device. The device that establishes a communication connection with the mobile phone is not necessarily an associated device that can form a HyperTerminal. The associated device must be a device that can establish a communication connection with the mobile phone.

In FIG. 7 , the smart watch icon 702 that has been connected with the mobile phone to form a hyperterminal can display a distinctive logo, which is convenient for distinguishing it from the associated device icon in the unconnected state. For example, the smart watch icon 702 is displayed in a dark filled form, indicating that the smart watch is connected to the mobile phone. In addition, the size of the smart watch icon 702 is smaller than that of the smart watch icon 506 when it is not connected. The diameter of the round icon of the smart watch icon 702 is R2 is adsorbed on the edge of the mobile phone icon 505 , and the smart watch icon 702 is tangent to the edge of the mobile phone icon 505 .

In some other embodiments, the two device icons being attracted together may also mean that the two device icons overlap in whole or in part. This embodiment does not limit the form of adsorption.

Correspondingly, if the user wants to detach an associated device from the HyperTerminal system, he can select the associated device icon and drag the associated device icon away from the core device icon when the associated device icon is in the state of adsorption to the core device. When the mobile phone detects that the user drags the icon of the associated device away from the icon of the core device for a distance greater than the third distance, when the user releases the finger, the icon of the associated device returns to the position before it was not connected, the adsorption state is released, and the associated device is also disconnected If the connection with the core device is separated from the HyperTerminal system, the service provided by the device in the HyperTerminal will also be terminated. In some examples, the core device icon remains stationary, and the user selects the associated icon and drags it away from the core device icon for a certain distance, and the two devices can be disconnected. In other examples, the user can select the associated icon and the core device icon at the same time, press and hold the two icons respectively, and drag the two icons in opposite directions over a certain distance, and the two devices can be disconnected.

In this embodiment, there is no limitation on conditions or user operations for triggering adsorption and releasing adsorption.

In some other embodiments, when the user drags the associated device icon closer to the core device icon for less than the second distance, the user does not release the associated device icon, but after the user stays at this position for more than the first preset time, the associated Device icons can also snap together with core device icons. Correspondingly, the user can also drag the icon of the associated device to move away from the icon of the core device for a distance greater than the third distance. To the position before it is not connected, the adsorption state is released, and the associated device is also disconnected from the core device at the same time, and it is separated from the hyperterminal system, and the service provided by the device in the hyperterminal is also terminated.

Or, in some other embodiments, when the user clicks an icon of an unconnected associated device, the connection between the associated device and the core device can be triggered, and the interface shows that the associated device icon is attached to the core device icon together. Correspondingly, when the user clicks an associated device that is in the connected state, it can trigger the associated device to disconnect from the core device. On the interface, the icon of the associated device is separated from the icon of the core device, and the icon of the associated device returns to The location of the unconnected state.

In some embodiments, the selected device icon can change its position along with the track of the user's finger dragging on the screen. This embodiment does not limit the finger movement trajectory when the user drags the device icon, and the user's dragging trajectory may be any curve. For example, when the user selects an associated device icon, and the user drags the associated device icon on the screen for a certain distance or stays on a certain position on the screen for a second preset time, the mobile phone can detect the current association between the associated device icon and the mobile phone. Whether the distance between the icons is less than or equal to a certain distance, such as the second distance, if so, the associated device icon and the mobile phone icon are adsorbed together. When the user's finger moves, the associated device icon moves along with the track of the user's finger.

In some other embodiments, the selected device icon may not change its position along with the dragging track of the user's finger on the screen. For example, when the user selects an associated device icon, and the user slides a finger on the screen for a certain distance or stays on a certain position on the screen for a second preset time, the mobile phone can determine the distance between the current position of the user's finger and the mobile phone icon Whether it is less than or equal to a certain distance, such as the second distance, if so, the associated device icon and the mobile phone icon are attracted together. The associated device icon does not move when the user's finger is moved.

In some other embodiments, the user can also select the icon of the core device to be close to the icon of the associated device, which also triggers the connection of the two devices. For example, if the user holds down the phone icon and drags the phone icon close to the smart watch icon, when the phone icon and the smart watch icon overlap or overlaps, the user releases the finger, and the smart watch icon can also be attached to the phone icon, indicating that The phone is connected to the smart watch and can work together.

In some embodiments, a one-key disconnection control can also be set on the HyperTerminal interface. Clicking the one-key disconnection control can disconnect all associated devices from the core device, and restore the core device to be independent without any associated device connection. state, the services provided by each associated device will also be terminated.

It can be understood that the hyperterminal is not limited to the connection between the associated device and the core device. In other embodiments, the user can also connect an associated device with another associated device in the hyperterminal interface to form a hyperterminal. That is to say, the user can connect any two or more devices, and the connected devices are not limited to associated devices or core devices. For example, associated devices, smart speakers and smart TVs can be directly connected to form a super terminal. For functions, schematic interfaces or user operations, reference may be made to the descriptions in the preceding or subsequent embodiments, and details are not repeated here.

In the embodiment provided by this application, when multiple devices are connected to form a hyper terminal, the hyper terminal system can be based on the currently connected device type and/or device characteristics, or according to the detected environment where the user is currently located, or the device Analyze the current user’s usage scenarios, and further, enumerate the scenario-based services or functions that the HyperTerminal system can provide, and display them on the user interface, so that users can quickly start related services or functions.

Specifically, different device types have different device features. For example, smart TVs have larger screens, which provide users with a better visual viewing experience and are more suitable for watching videos; smart speakers have strong audio output capabilities and better speaker effects. It is more suitable for playing audio; the earphone is small in size, easy to carry, does not need to be placed outside, does not disturb people around, and can also have a noise reduction effect, shielding external environmental noise, suitable for listening to audio in public places, such as talking in public places, listening to music, etc.; Smart watches and smart bracelets can measure the user's exercise data or heart rate and other health information more accurately than mobile phones; laptops have powerful processing capabilities and are more suitable for office work. The device type and device features will not be described in detail here, and the above examples do not limit other embodiments.

Alternatively, the device may also detect the current environment of the user, and analyze the usage scenario of the current user. This embodiment does not limit the method of detecting the user's environment. For example, it can be determined whether the device is indoors or outdoors based on the current location information of the device; or use a camera to collect photos of the current environment, or use ultrasonic waves, infrared rays, etc. The spatial depth or size of the environment to determine whether the current user is indoors or outdoors.

In some embodiments, the hyperterminal system can be divided into various consumer usage scenarios. The HyperTerminal system can match relevant usage scenarios based on the device type, device features, product positioning, common functions and other information of the devices in the current system.

In an example, consumer usage scenarios may include the following five categories: mobile office, smart home, audio-visual entertainment, sports and health, and smart travel. Of course, the classification of consumer usage scenarios is not static, and the hyperterminal system can continuously update, expand or refine more usage scenarios, which are not limited in this embodiment, and can be changed according to actual conditions. For example, the addition of associated devices or functions that have not appeared before may lead to new usage scenarios. For example, adding a smart car as an associated device can form a super terminal with a mobile phone, and then add a mobile phone and a smart car. The scene is an intelligent travel scene.

After the HyperTerminal system matches the usage scenario according to the device, it can also analyze and query the scenario-based services and atomized services that the device combination can provide based on the usage scenario.

Among them, the scenario-based service refers to a functional service based on a usage scenario, which is a collection of multiple functions or services that can be provided for a specific scenario. The atomic service refers to the smallest capability unit that can run independently. It is a concept of abstract packaging of a single function/capability. It can be a hardware service or a software service. Generally, a scene service can be supported by multiple atomic services, and an atomic service combination includes atomic services composed of atomic capabilities of multiple devices. The atomic capabilities may include one or more of the following: audio output capabilities, audio input capabilities, display capabilities, camera capabilities, touch input capabilities, keyboard and mouse input capabilities, and the like. For example, the audio output capability may include whether the device supports mono or multi-channel, supported sound effects, supported frequency response range, noise reduction capability, audio analysis capability, etc. when the device is playing audio; the audio input capability may include receiving Acoustic range, noise reduction capability, etc.; display capability may include the device’s screen size, display resolution parameters, refresh rate, color expressiveness, etc.; camera capability may include the device’s camera type, shooting pixels, night scene shooting capability, and image adjustment capability, etc.; touch input capability, keyboard and mouse input capability refer to whether the device can support touch input or keyboard and mouse input, etc.

For example, the multi-screen collaborative scene service between the mobile phone and the large screen is supported by the following atomic services: display service, audio input service, audio output service, touch input service, keyboard and mouse input service, camera service, etc. Among them, the display service can be provided by the large screen, the audio input service can be provided by the mobile phone, the audio output service can be provided by the large screen, the touch input service can be provided by the mobile phone, the keyboard and mouse input service can be provided by the large screen, and so on.

It can be understood that scene-based service and atomized service are only words used in this embodiment, and their representative meanings have been recorded in this embodiment, and their names do not constitute any limitation to this embodiment. For example, in some other embodiments, scenario-based services may also be called scenario-based functions, scenario functions, scenario services, scenario-based services, business functions, and other terms; atomic services may also be called meta-capabilities, atomic capabilities (Ability ), atomic services, functional components, and other nouns. In the embodiment of this application, it is mainly described as "scenario-based service" and "atomized service".

Taking a super terminal composed of a mobile phone and a smart watch as an example, it can match its usage scenario as a sports and health scene. The super terminal system can query the scene-based services and atomized services that can be provided by the mobile phone + smart watch combination. For example, the combination of mobile phone + smart watch can provide services including: supporting sports services, supporting health services, counting exercise, measuring heart rate, measuring blood oxygen saturation, etc. Combined with different scenarios, the scenario-based services recommended for mobile phone + smart watch combinations include: indoor running, outdoor running, heart health, blood oxygen health, etc. Query the atomic services supported by smart watches, including: GPS positioning, sound output, vibration reminder, etc.

In an example, see Table 1. Table 1 lists the corresponding classification of usage scenarios when the core device is a mobile phone, and some different associated devices form a hyper terminal with the mobile phone, and the scenario-based services that are analyzed and recommended after querying the capabilities of the device. It should be noted that Table 1 only shows the case where the associated device is a single device, but it is not limited thereto. In other embodiments, there are also cases where multiple associated devices and mobile phones form a hyper terminal, for example, the associated device is a smart TV and a smart speaker, and a mobile phone to form a super terminal, and its corresponding execution operations and function descriptions can refer to the foregoing or subsequent embodiments, and will not be repeated here.

Table I

Scenario-based services can be based on the type of core device and/or associated devices, device usage, user's commonly used collaborative work content, current environment of the device, device operating status, user's recently used applications, currently running applications, etc. Situation, scenario-based service options based on comprehensive statistics.

In some embodiments, the HyperTerminal can recommend scenario-based services based on device type, product positioning, and device usage. For example, the combination of a mobile phone and a smart watch is often used to count exercise, so the HyperTerminal system is based on the device type, device Uses and common collaborative work content can be classified as sports and health scenarios for the combination of mobile phones and smart watches. Scenario-based service options that can be provided in sports and health scenarios include outdoor running, indoor running, heart health, and vascular health.

In some embodiments, when listing the scenario-based service options, the probability of each scenario-based service option being selected can be calculated according to the user's recent use frequency or the environment in which the current device is detected. In the scenario-based service option area The scenario-based service options that are estimated to have a high probability of being selected are displayed first.

The hyper terminal system can display the options with high usage rate in the front according to the usage frequency of various scenario-based service options by users within a certain period of time according to the statistics. The certain time can be all the time in history, or a certain period of time, such as the last three months. The higher the historical usage rate of the scenario-based service option, the higher the display.

In some other embodiments, the core device or the associated device may detect the environment where the current device is located, and sort the scenario-based service options according to the environment. If the core device or associated device detects that the user's environment is outdoors, it will give priority to recommending outdoor-related service options, such as outdoor running. If the core device or associated device detects that the user's environment is indoors, it will give priority to recommending indoor related service options, such as indoor running. This embodiment does not limit the method of detecting the user's environment. For example, the environment can be judged according to the current positioning information of the device; or the camera can be used to collect photos of the current environment, or ultrasonic waves, infrared rays, etc. size to determine the current user's environment.

In some other embodiments, the HyperTerminal system can also determine the environment or scene where the device is located according to the application recently opened or used by the user, or the currently running application, and then recommend scene-based service options. For example, when a mobile phone and a smart TV form a super terminal, if it is detected that the application currently running on the mobile phone is an office presentation application, then the multi-screen collaborative service option can be recommended first; or, if it is detected that the application currently running on the mobile phone is a video application, then the Recommend home theater service options; or, when it is detected that the webpage browsed by the user is a game-related webpage, preferentially recommend game entertainment-related service options, etc.

In some other embodiments, the hyperterminal system can also recommend scene-based services according to the running status of the equipment. For example, when a mobile phone is connected to a smart TV to form a super terminal, if it is detected that the smart TV is in the on-screen state, then the scene-based service is recommended as screen mirroring; if the smart TV is detected to be off-screen, the scene-based service is recommended as Casting service.

In some other embodiments, the hyperterminal system can also recommend scenario-based services according to the atomized services that the device can provide.

In some embodiments, the user can customize and add scene-based services, for example, the user sets a name for a usage scene and associates the scene with other application services or system services. The newly added scenario-based service is displayed side by side with other scenario-based services on the HyperTerminal interface.

The foregoing embodiments do not limit other embodiments. The hyperterminal system may comprehensively consider one or more of the above factors and reasonably recommend scene-based service options.

The hyper terminal system can be based on the current device combination, the atomized services that can be provided by each device that can be queried, and then according to the scene service actually selected by the user, it can be based on the user's usual service combination, or the default service combination set by the user, or based on life Cognition is the most suitable service combination in this scenario, or the most frequently used service combination in this scenario is obtained from the server, and the atomic service combination in this scenario is automatically set, saving users from tedious selection operations.

For example, after a mobile phone and a smart TV form a super terminal, if the scene service selected by the user is a multi-screen conference, the atomic service combination is automatically set as follows: the audio output function is provided by the smart TV, the audio input (microphone) function is provided by the mobile phone, and the camera The function is provided by the smart TV, and the display function is provided by the smart TV. For another example, after a super terminal is composed of a mobile phone, a smart watch, earphones, and a treadmill, if the scene service selected by the user is indoor running, the atomic service combination is automatically set as follows: the audio output function is provided by the earphone, and the audio input (microphone) function is provided by the The earphones provide, the camera function is provided by the mobile phone, and the display function is provided by the treadmill.

In some embodiments, the hyperterminal system can analyze the optimal atomic service combination according to the device characteristics of different device types. For example, a smart TV has a larger screen, and the user has a better viewing experience and is more suitable for watching videos. Then, when a smart TV and a mobile phone form a super terminal, the smart TV is given priority to provide display functions. Smart speakers have strong audio output capabilities, better sound effects, and are more suitable for playing audio. When a smart speaker and a mobile phone form a super terminal, the smart speaker is given priority to provide audio output. The headset is small in size, easy to carry, does not need to be placed outside, does not disturb people around, and can also have a noise reduction effect, shielding external environmental noise, suitable for listening to audio in public places, such as talking in public places, listening to music, etc. When forming a HyperTerminal, it is preferred to set the earphone to provide audio output function. Smart watches and smart bracelets can measure the user's exercise data or heart rate and other health information more accurately than mobile phones. When smart watches or smart bracelets and mobile phones form a super terminal, the smart watch or smart bracelet is preferred. The acquired motion data; the laptop has powerful processing capability and is more suitable for office work, so when the laptop and mobile phone form a super terminal, the laptop is given priority to provide display functions, keyboard input functions, etc. The device type and device features will not be described in detail here, and the above examples do not limit other embodiments.

In some embodiments, the hyper terminal supports users to personalize the atomic service combination, and the user can manually modify the atomic service combination according to personal preferences or actual conditions, and adjust a certain atomic service to be provided by other devices. For example, in a multi-screen conference, if the surrounding environment is noisy, the user can change the audio output function provided by the smart TV to the headset.

In the embodiment provided by this application, the HyperTerminal system can also analyze the current scene based on the types of devices that currently make up the HyperTerminal system, and list and recommend other devices suitable for the current scene to the user, so as to help the user obtain a better scene experience; Or speculate on more scalable usage scenarios, and then list and recommend other devices to the user. If the user adopts the device recommended by the system, more and richer usage scenarios can be further expanded.

For example, when a mobile phone is coordinated with a smart TV, wireless microphones, wireless game controllers and other equipment can be recommended to users, and scene-based services such as karaoke and games can be expanded correspondingly. For another example, after the collaboration between the mobile phone and the smart watch, it can recommend users such as treadmills, heart rate belts, blood glucose meters and other equipment, and can correspondingly expand scene-based services such as indoor running, heart health, and blood sugar health. After adding a recommended device, the system can also recommend new devices according to the newly formed hyperterminal. Users can choose to add more devices to form a hyper terminal, making the user use scenarios more abundant. After adding more associated devices, the HyperTerminal can provide more and richer scene-based services, improving the HyperTerminal ecosystem.

The functions provided by the above-mentioned embodiments are illustrated below with reference to the accompanying drawings.

Referring to the user interface 700 shown in FIG. 7, in the example of FIG. 7, when the mobile phone and the smart watch form a super terminal system, a scene-based service option area 705 can be displayed below the device connection status icon 701. The scene-based service option area In step 705, one or more scenario-based services recommended by the hyperterminal system according to the current device combination are listed. In addition, a recommended device area 704 is also displayed, which is used to list devices that can be linked with the current hyperterminal system device to provide users with richer scenarios or better experience, and the user can select according to needs. In the user interface 700, an atomized service entry option 706 is also displayed. Select the atomized service entry option 706 to display the atomized service combination, and the user can choose the atomized service provider by himself. Refer to the embodiment shown in FIG. 10 .

In the example of FIG. 7, when the mobile phone and the smart watch form a super terminal system, multiple scenario-based service options are displayed in the scenario-based service option area 705, such as outdoor running, indoor running, heart health, and vascular health. The user can also use a finger to slide left and right on the scene service option area 705 on the screen to view more options that are not displayed.

These scenario-based service options can be based on the device type of the mobile phone and smart watch, the purpose of the device, the content of the user's common collaborative work, the current environment of the device, the operating status of the device, the application recently used by the user, the currently running application, etc. , the scenario-based service options obtained from comprehensive statistics. In addition, the probability of each scene-based service option being selected can be calculated according to the user's recent frequency of use or the environment in which the current device is detected, and the scene-based service options that are estimated to have a high probability of being selected are displayed first in the scene-based service option area 705. Service options.

For example, the combination of mobile phones and smart watches is often used to count the amount of exercise. Based on the combination of mobile phones and smart watches, the super terminal system can classify the combination of mobile phones and smart watches as sports and health scenarios based on the device type, device use, and commonly used collaborative work content. The scenario-based service options that can be provided in the sports health scenario include outdoor running, indoor running, heart health, blood vessel health, etc., which are displayed in the scenario-based service option area 705 .

The hyper terminal system can display the options with high usage rate in the front according to the usage frequency of various scenario-based service options by users within a certain period of time according to the statistics. The certain time can be all the time in history, or a certain period of time, such as the last three months. The higher the historical usage rate of the scenario-based service option, the higher the display. For example, it is obtained that the most frequently used scene service option of the user is outdoor running, followed by indoor running, etc., then in the scene service option area 705, the first displayed scene service option is outdoor running, and the second displayed service option is outdoor running. Scenario-based service options include outdoor running, etc.

In some other embodiments, the mobile phone or the smart watch can detect the environment where the current device is located, and sort the scene-based service options according to the environment. If the mobile phone or smart watch detects that the user's environment is outdoors, then the outdoor running service option is recommended first. If the mobile phone or smart watch detects that the user's environment is indoors, then the indoor running service option is recommended first. This embodiment does not limit the method of detecting the user's environment. For example, it can be determined whether the device is indoors or outdoors based on the current location information of the device; or use a camera to collect photos of the current environment, or use ultrasonic waves, infrared rays, etc. The spatial depth or size of the environment to determine whether the current user is indoors or outdoors.

In some other embodiments, the HyperTerminal system can also determine the environment or scene where the device is located according to the application recently opened or used by the user, or the currently running application, and then recommend scene-based service options. For example, before the user opens the super terminal interface to connect the smart watch and the mobile phone, the application the user runs is a sports and health application, and the service used is the outdoor running sub-service in the sports and health application, then the user connects the smart watch to the mobile phone to form a hyper terminal system , in the scenario-based service option area 705, the first displayed scenario-based service option is outdoor running.

The above examples do not limit other embodiments. The hyper terminal system may comprehensively consider one or more of the above factors and reasonably recommend scene-based service options.

In some embodiments, when the devices are connected to form a hyper terminal, a certain scenario-based service may be automatically selected as a default service. For example, setting the first scenario service option in the scenario service option area 705 as a default service, when devices are connected to form a hyper terminal, each device is configured according to the atomic service combination corresponding to the scenario service by default. If the user wants to choose other scenario-based services, he can choose other scenario-based service options. Setting the default scene-based service can facilitate the seamless switching of services on devices and save user operations to a certain extent. For example, when the headset is connected to the mobile phone, the default service setting is that the headset provides audio output and audio input functions. Then when the headset is successfully connected to the mobile phone, the audio output and audio input functions are immediately switched to the headset, and the user does not need to perform additional operations. , saving the tedious operation. The setting of the default service may be a default service selected by the user, including scenario-based services and/or atomic services. The setting of the default service may also be that the system uses the user's frequently-used services obtained from statistics as the default service, etc. , which is not limited in this embodiment.

In some other embodiments, when the devices are connected to form a hyper terminal, the scene-based service may not be selected first, but one or more scene-based service options are listed. After the user selects a certain scene-based service, the scene The service or the corresponding atomic service switches to the set state. The advantage of this is to fully consider the user's willingness to use, and avoid automatically switching to the wrong service, causing trouble or distress to the user. For example, when connecting a mobile phone to a PC, the user intends to transfer files. If the default service is screen mirroring, the screen of the mobile phone will be mirrored to the PC. At this time, the user needs to switch services, especially if the current PC is being used for other purposes. , If you are presenting a document, the screen mirroring service will hinder the display of the presentation document, causing trouble to the user, or the user has private content on the mobile phone and does not want to display it on the PC. If you directly switch to the screen mirroring service, it will cause trouble for the user. Therefore, when the mobile phone is connected to the PC, the scene-based service options can be displayed on the hyperterminal interface, and the user first selects the required scene-based service options and/or atomized service combination options, and then executes switching services.

It is explained here that when two or more devices form a hyper terminal and the user selects a scene-based service, these devices can have the ability or status to perform interactive functions, but this is not limited to the two or more Interaction functions are performed immediately between devices. For example, if the user triggers the audio casting function between the mobile phone and the smart speaker by dragging the device icon. If at this time, the mobile phone does not have an audio playback task or the audio playback task of the mobile phone is suspended, the mobile phone will not immediately send audio data to the smart speaker. However, when the mobile phone has an audio playback task, the mobile phone will send the audio data to be played to the smart speaker, and the smart speaker will play it.

In the user interface 700 of FIG. 7 , icons of treadmills and spinning bikes are displayed in the recommended equipment area 704 , prompting the user to add the treadmills or spinning bikes to the HyperTerminal to obtain a better scene experience. In the example shown in FIG. 7, the hyperterminal system can analyze the user's usage scene as a sports and health scene based on the current devices that make up the hyperterminal as a mobile phone and a smart watch. The recommended device area 704 can display recommended sports equipment or health monitoring equipment, etc. Such as treadmills, spinning bikes, blood glucose meters, heart rate meters, etc.

When the user selects (clicks or drags) the device icon in the recommended device area 704, if the device is an associated device detected by the current core device, that is, the device displayed in the device connection status diagram 701, then the device can initiate a connection Join HyperTerminal. When the user selects (clicks or drags) the device icon in the recommended device area 704, if the device is not displayed in the device connection status icon 701, it can jump to a page related to the device product, such as introducing the device to the user. The function of the recommended device, user experience, how to use and expand the experience when forming a hyper terminal with other devices, purchase links, purchase channels, etc.

Adding these recommended devices can give users a better scene experience, or expand more usage scenarios. After adding a recommended device, the system can also recommend new devices according to the newly formed hyperterminal. Users can choose to add more devices to form a hyper terminal, making the user use scenarios more abundant. For example, a mobile phone is connected to a smart watch to form a hyperterminal. The system recommends devices including a treadmill, and the user then selects the treadmill to connect to the hyperterminal. The system recommends devices including earphones, and the recommended user can listen to music while running. With the addition of more devices, a more complete HyperTerminal ecosystem can be formed.

In some embodiments, the mobile phone can display the prompt information of the recommended device on the user interface. For example, after detecting that the mobile phone is coordinated with the smart TV, the mobile phone finds that the sound box is also around, so it can display the prompt information, such as "Recommended sound box Sound X Join the hyperterminal, is it allowed?" and give the options "yes" and "no". If the user selects "yes", the speaker Sound X will be connected to the hyperterminal. If the user selects "no", no other operations will be performed. Or after the prompt message is given, only the option "Yes" will be displayed. If the user selects "Yes", the speaker Sound X will be connected to the HyperTerminal. If the user does not operate for a certain period of time, such as two seconds, the prompt message will be automatically canceled. , and no other operations are performed.

In the embodiment of the present application, the cross-device services or functions that the HyperTerminal can provide include but are not limited to system-level services or functions, and application-level services or functions. Here, for example, system-level services or functions may include but not limited to audio projection, screen projection, etc., and application-level services or functions include but not limited to application relay, content sharing, etc.

Among them, casting sound refers to sending the audio stream played on one electronic device to another electronic device, and the other electronic device plays it. For example, audio projection from a mobile phone to a smart speaker, audio projection from a mobile phone to a smart screen with the screen off, screen projection from a mobile phone to a car, etc.

Screen projection includes stream projection based on digital living network alliance (DLNA) technology (also known as heterogeneous projection), screen mirroring based on Miracast technology (also known as homologous projection), And screencasting based on other technologies such as AirPlay based. For example, screen projection between a mobile phone and a smart TV/PC/tablet. If the user wants to cast the video or audio played on the mobile phone to other electronic devices, they can perform stream casting. Then, the mobile phone can send the playback address of the video or audio to an electronic device (such as a PC), and the PC plays the corresponding video or audio according to the received playback address. Alternatively, the mobile phone sends the video data or audio data played by itself to the PC, and the PC can directly play the received video data or audio data. During the screen projection process, the interface displayed on the mobile phone may be different from the content displayed on the PC. If the user wants to cast the display interface on the mobile phone screen to other electronic devices, screen mirroring can be performed. Then, the mobile phone sends the mobile phone interface data to other electronic devices (such as a PC), and the PC displays the mobile phone interface. During the screen casting process, the interface displayed on the mobile phone is the same as that displayed on the PC.

Application relay means that one electronic device can relay an application running on another electronic device.

Content sharing includes sharing files/photos/videos/audio/text among different electronic devices. For example, application relay between mobile phones and smart TVs/PCs/tablets. Wherein, the relayed applications are, for example, Changlian application (a video call application), video application, music application, and document application.

It can be seen from the above that the example user interface 700 shown in FIG. The devices form a HyperTerminal. In addition, the HyperTerminal can judge the usage scenarios based on the device type, device features, product positioning, common functions, device status, current running applications, current device environment, etc., and provide one or more scenarios based on the usage scenarios. service options, atomized service composition options, or interactive function options. Users can quickly select scene-based services and/or atomized services according to their personal needs, without cumbersome decentralized settings. Therefore, it can solve the problem that the settings of interactive functions between electronic devices are scattered and cumbersome, and it is impossible to list rich uses for users. scene problem.

The embodiment of the present application provides the scenario-based services and/or atomized services available under the device combination through the user-selected device combination, lists detailed and rich usage scenarios for the user, and guides the user to use the hyper terminal with sufficient functions. Explore more application scenarios and achieve the effect of recommending scenario-based services based on device combinations.

Referring to FIG. 8 , when the mobile phone detects that the user clicks and selects the outdoor running option 707 listed in the scene service option area 705 of the user interface 700 , the mobile phone can jump to a page related to outdoor running, as shown in the user interface 800 . Wherein, the outdoor running option 707 can be highlighted when selected.

The user interface 800 is a sports and health application page, and you can see that there are sub-options displayed under the sports and health title bar: outdoor running sub-option 801 , indoor running sub-option, heart health sub-option, and vascular health sub-option. Since the user selects the outdoor running option 707 in the user interface 700, the page corresponding to the outdoor running sub-option 801 is currently displayed on the user interface 800, such as a map icon, a user's positioning icon, and a user's exercise route. , Statistical user exercise volume, etc.

Correspondingly, the smart watch displays an interface related to outdoor running, refer to the smart watch user interface 802 . The user interface 802 of the smart watch may not be consistent with the user interface 800 related to outdoor running of the mobile phone, and exercise data may be shared between the smart watch and the mobile phone. The smart watch user interface 802 can display exercise data such as pace, exercise distance, heart rate, and exercise time detected by the smart watch.

Figure 9 shows that the scenario-based service options and recommended devices will change with different device combinations, and the HyperTerminal will provide more suitable scenario-based service options and recommended devices according to different device combinations.

Referring to the user interface 900 shown in FIG. 9 , in the user interface 900, the device connection status icon indicates that the mobile phone is connected to the smart TV Vision to form a hyper terminal, and the mobile phone icon and the smart TV Vision icon are adsorbed together in the reference indication area 901, and Under the title of HyperTerminal, it states "Synergy with Vision".

In the user interface 900, the scene-based service option area 903 includes scene-based service options such as multi-screen collaboration, screen extension, multi-screen conference, and somatosensory games. The recommended device area 902 includes device icons for speakers, laptops, and gamepads. The above scenario-based service options and recommended device options are based on the recommended options obtained based on the fact that the devices included in the hyper terminal are mobile phones and Smart TV Vision.

If the user selects the speaker icon in the recommended device area 902 to join the hyperterminal, or drags the speaker Sound X icon in the device connection status icon to be absorbed around the mobile phone icon, the user interface 910 can be displayed.

In the user interface 910, the device connection status icon indicates that the mobile phone is connected to the smart TV Vision and the speaker Sound X to form a super terminal. Under the title of the HyperTerminal, it says "Synergy with Vision and Sound X".

In the user interface 910, the scene-based service option area 913 includes scene-based service options such as home theater, motion-sensing games, enjoying karaoke, and multi-screen conference. The recommended device area 912 includes device icons for laptops and gamepads. The above scenario-based service options and recommended equipment options are based on the recommended options obtained from the mobile phone, smart TV Vision, and speaker Sound X, which are included in the hyperterminal. In this example, compared to the device combination of mobile phone and smart TV Vision, the HyperTerminal system can infer that the combination of mobile phone, smart TV Vision, and speaker Sound X is more entertainment-oriented, so it will give priority to recommending entertainment-oriented scene services.

If the user selects the atomic service entry option in the user interface 910, one or more atomic service combinations can be displayed, as shown in the user interface 1000 and user interface 1010 shown in Figure 10, and the user can choose different atomic services by different devices supply.

Referring to FIG. 10 , a service setting window 1001 is displayed in the user interface 1000, and multiple atomized service options are displayed in the service setting window 1001, such as screen display, sound output, sound input, camera, etc. shown in the user interface 1000. Among them, the screen display service/function is provided by the smart TV Vision, the sound output service/function is provided by the smart speaker Sound X, the sound input service/function is provided by the mobile phone HuaWei P50, the camera service/function is provided by the mobile phone HuaWei P50, etc.

In this example, the option bar 1002 corresponding to the sound output includes a sound output service/function identifier, and the icon and name of the device smart speaker Sound X that currently provides the sound output service/function.

In some embodiments, the hyper terminal supports users to personalize the atomic service combination, and the user can manually modify the atomic service combination according to personal preferences or actual conditions, and adjust a certain atomic service to be provided by other devices. For example, the user may choose to switch the audio playback service provided by the mobile phone to the audio playback service provided by the smart speaker.

If the user wants to change the device providing the service, he can click on the control 1003 and the mobile phone can display the user interface 1010 .

It can be seen that in the service setting window 1011 in the user interface 1010, the option bar 1012 corresponding to the sound output is expanded to display multiple device options, namely the mobile phone HuaWei P50, the smart TV Vision, and the speaker Sound X in the hyperterminal. The user can click the selected control 1013 corresponding to the right side of the device name to confirm that the device provides the sound output service/function.

In some embodiments, the hyperterminal system can analyze the optimal atomic service combination according to the device characteristics of different device types, and allocate different devices to carry different services. For example, in this example, a smart TV, a smart speaker, and a mobile phone form a super terminal. The smart TV has a larger screen, and the user has a better visual viewing experience and is more suitable for watching videos. Therefore, it is preferred to set the smart TV to provide display functions. The smart speaker has strong audio output capability, better speaker effect, and is more suitable for playing audio, so it is preferred to set the smart speaker to provide the audio output function. The mobile phone is relatively light, easy for the user to grasp, and the microphone is convenient for listening to the sound, so it is preferred to set the mobile phone to provide audio input and camera shooting functions.

In some other embodiments, it can also be based on the user's frequently used service combination, or the default service combination set by the user, or the service combination that is most suitable for the scene based on life cognition, or obtain the most frequently used service combination in the scene from the server. Service combination, etc., automatically set the atomic service combination in this scenario, saving users from tedious selection operations.

In some embodiments, a blank option may also be set in the service setting window 1011, that is, the atomized service is not provided by any device.

In some embodiments, corresponding prompt information may be displayed on the mobile phone interface to remind the user of the device that can provide the best atomization service, for example, the current speaker is the best device for playing audio.

In some embodiments, when the user completes the setting, the user can slide down the service setting window, and then return to the previous interface.

Fig. 11, Fig. 12, Fig. 13A, Fig. 13B, Fig. 13C, and Fig. 13D show other interaction modes and schematic interfaces for selecting scene-based service options.

As shown in the user interface 1100 shown in Figure 11, the scene-based service options can be displayed in the form of floating controls, and different scene-based service options correspond to different areas. convenient.

In the user interface 1100, when the user presses the smart watch icon, the super terminal interface can display a circular floating area 1101 with the mobile phone as the center. The circular floating area 1101 can be divided into multiple areas, and each area corresponds to a A scenario-based service option. For example, in the user interface 1100, the circular floating area 1101 is divided into four areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health.

If the user wants to select a scene-based service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then drag along the direction inside the area corresponding to the outdoor running service option, close to the For the mobile phone icon, the trajectory of the user's finger can refer to the schematic trajectory in the user interface 1100 . When the smart watch icon reaches the adsorption area, the user releases the finger, the service corresponding to the outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . Of course, if there is only one scene-based service option, the circular floating area 1101 does not need to be divided into blocks, and the user will select the same scene-based service when approaching the mobile phone icon from any direction. The selected scene-based service option can be displayed as highlighted, as shown in Figure 11, the area corresponding to the outdoor running service option is displayed as highlighted.

One operation in Figure 11 is equivalent to the two operations in Figure 6 and Figure 8, that is, one user operation in Figure 11 includes the two functions of associating the associated device to the core device and selecting the scene-based service, which improves the efficiency of user operations .

It should be noted that the scene-based service options displayed in the floating area may also have a recommended order, and this embodiment does not limit the display manner of the recommended order of the scene-based service options. For example, take four scene-based service options corresponding to four areas as an example. Taking the center as the origin, the first quadrant is called area A, the second quadrant is called area B, the third quadrant is called area C, and the third quadrant is called area C. The four-quadrant orientation is called zone D. The recommended order of scenario-based service options is outdoor running, indoor running, heart health, and vascular health, and the corresponding areas can be area A, area B, area C, and area D; or, area B, area A, area C, area D; or, area B, area C, area A, area D; or, area B, area A, area D, area C; or, area A, area B, area D, area C; or, area B, area C, area D, area A; or, area A, area D, area B, area C; or, area A, area D, area C, area B; etc., this embodiment is not limited, and the details can be determined according to the actual situation Settings, such as settings based on the user's operating habits, common point area, etc.

FIG. 12 shows another user interaction operation. Referring to the user interface 1200, when the user presses the smart watch icon, the hyper terminal interface can display a circular floating area 1201 centered on the mobile phone, and the circular floating area 1201 can be Divided into multiple areas, each area corresponds to a scenario-based service option. For example, in the user interface 1200, the circular floating area 1201 is divided into four areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health.

If the user wants to select a scene-based service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then release the finger, which means that the outdoor running service option is selected, and the outdoor running service option is selected. The corresponding service starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . The trajectory of the user's finger can refer to the schematic trajectory in the user interface 1200 . Of course, if there is only one scene-based service option, the circular floating area 1201 does not need to be divided into blocks, and the user drags the device icon to any position in the floating area 1201 and releases the icon to select the same scene-based service. The selected scenario-based service option can be displayed as highlighted, as shown in Figure 12, the area corresponding to the outdoor running service option is displayed as highlighted. Likewise, this embodiment does not limit the display manner of the recommendation ranking of the scenario-based service options.

Compared with the interaction mode shown in FIG. 11 , the interaction mode shown in FIG. 12 is easier for the user to operate, and the probability of wrong selection is lower.

The circular floating areas shown in FIG. 11 and FIG. 12 are only schematic diagrams. In other embodiments, the areas representing scene-based service options may also be in other forms.

Referring to the user interface 1300 shown in FIG. 13A , the scene-based service option area is a square floating area. The user interface 1300 displays a mobile phone as the center, and a square floating area 1301 is divided into four areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health. If the user wants to choose the outdoor running service, he can drag the smart watch icon to the area corresponding to the outdoor running service option in the interactive mode shown in Figure 11, and then drag along the direction inside the area corresponding to the outdoor running service option, close to the mobile phone icon, when the smart watch icon reaches the adsorption area, the user releases the finger, the service corresponding to the outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . The interaction method shown in Figure 12 can also be used. The user drags the smart watch icon to the area corresponding to the outdoor running service option, and then releases the finger, which means that the outdoor running service option is selected, the service corresponding to outdoor running starts, and the mobile phone jumps. Turn to display the outdoor running interface shown in the user interface 800 . The selected scene-based service option may be highlighted, for example, the area corresponding to the outdoor running service option shown in Figure 13A is highlighted. Likewise, this embodiment does not limit the display manner of the recommendation ranking of the scenario-based service options.

In some examples, the user interface 1310 shown in FIG. 13B , compared to FIG. 12 , may not display the mobile phone icon in the center, but after the user selects the smart watch icon, the displayed circular floating area 1311 includes only Multiple scenario-based service options. The circular floating area 1311 shown in FIG. 13B is divided into four areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health. If the user wants to choose the outdoor running service, he can drag the selected smart watch icon to the area corresponding to the outdoor running service option, the area corresponding to the outdoor running service option will be highlighted, and then release the finger, which means that the outdoor running service is selected For service options, the service corresponding to outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . Likewise, this embodiment does not limit the display manner of the recommendation ranking of the scenario-based service options.

The user interface 1320 in Figure 13C shows another display style. After the user selects the smart watch icon, the mobile phone displays a circular floating area 1321. The circular floating area 1321 includes a plurality of ring-shaped areas, and each ring The shape area corresponds to a scenario-based service option. As shown in FIG. 13C , the circular floating area 1321 has four circular areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health. If the user wants to choose the outdoor running service, he can drag the selected smart watch icon to the area corresponding to the outdoor running service option, the area corresponding to the outdoor running service option will be highlighted, and then release the finger, which means that the outdoor running service is selected For service options, the service corresponding to outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . Likewise, this embodiment does not limit the display manner of the recommendation ranking of the scenario-based service options. In one implementation, the scenario-based service options that are recommended with higher priority are displayed closer to the inner ring. Here, the ring with a smaller diameter is called the inner ring relative to the ring with a larger diameter, or the ring that is closer to the center of the circle is the inner ring. The ring, such as the outdoor running service option being the first recommended scenario-based service option, may be displayed in the innermost circle. The indoor running service option is the second recommended scenario-based service option, and is displayed in the circle of the second inner ring. The heart health service option is the third recommended scenario-based service option, displayed in the third inner circle. The vascular health service option is the fourth recommended scenario-based service option, displayed in the outermost circle.

The user interface 1330 in Figure 13D shows another display style. After the user selects the smart watch icon, the mobile phone displays a fan-shaped floating area 1331. The circular floating area 1331 includes multiple fan-shaped ring areas, and each fan-shaped ring area Corresponds to a scenario-based service option. The fan-shaped floating area 1331 shown in FIG. 13D has four fan-ring areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health. If the user wants to choose the outdoor running service, he can drag the selected smart watch icon to the area corresponding to the outdoor running service option, the area corresponding to the outdoor running service option will be highlighted, and then release the finger, which means that the outdoor running service is selected For service options, the service corresponding to outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 .

In the example shown in FIG. 13D , the fan-shaped floating area 1331 is a quarter-circle fan-shaped area with the center of the circle as the lower right corner. This is considering that the user often controls the mobile phone with the right hand. For the convenience of the user, this design is more reasonable . Of course, it is not limited to the example shown in FIG. 13D , the fan-shaped floating area 1331 may also be a quarter-circle fan-shaped area with the center of the circle as the lower left corner, which is convenient for left-handed users to manipulate. The mobile phone can switch different display control styles according to the user's favorite hand as the right hand or left hand, or the mobile phone can determine whether the user is currently using the left hand or the right hand when the user controls the mobile phone, and then confirm the display style of the floating control, etc. .

Likewise, this embodiment does not limit the display manner of the recommendation ranking of the scenario-based service options. In one implementation, as shown in the user interface 1330, considering that the user often holds the mobile phone with his right hand in the lower half of the mobile phone, the scene-based service options that are recommended with higher priority are displayed closer to the inner ring, which is referred to here as the fan that is closer to the center of the circle. The ring is relatively an inner ring, or the fan ring with a smaller radius is relatively an inner ring. If the outdoor running service option is the first recommended scenario-based service option, it may be displayed in the innermost fan ring. The indoor running service option is the second recommended scenario-based service option, which is displayed in the fan ring of the second inner ring. The heart health service option is the third recommended scenario-based service option, which is displayed in the fan ring of the third inner ring. The vascular health service option is the fourth recommended scenario-based service option, displayed in the outermost fan ring.

The above embodiments are only examples, and the embodiments of the present application do not limit the form of selecting scene-based service options and atomized service options, and technical solutions with the same inventive concept are within the scope of protection of the present application.

In the embodiments shown in Figure 11, Figure 12, Figure 13A, Figure 13B, Figure 13C, and Figure 13D, after the user selects the outdoor running service option, the service corresponding to outdoor running starts, and the mobile phone jumps to display the user interface 800 The interface of the outdoor running service. When the user chooses to return to the hyperterminal page, the user interface 1400 shown in FIG. Collaborative work.

In some embodiments, when multiple devices form a hyperterminal and start scene-based services to work together, in the interface, the interactive orientation information can also be used as an input parameter, and the input of different orientation information can correspond to different feedbacks. For example, when a mobile phone is connected to an associated device such as a PAD/PC/smart TV to start a multi-screen collaborative service, the user drags the icon of the associated device to attach and absorb it from the left or right side or the middle area of the mobile phone icon, which will produce different effects. For example, if the user drags the icon of the associated device from the left area of the mobile phone icon to stick it together, then the screen of the mobile phone will be projected on the right side of the screen of the associated device; if the user drags the icon of the associated device from the mobile phone If the right area of the icon fits and sticks together, the screen of the mobile phone will be projected on the left side of the screen of the associated device; Cast full screen on the associated device screen.

Of course, in some embodiments, the dragging directions of the attracting device icon and the adsorbed device icon are exchanged, and the service or interactive function performed by them is the same. If the user can also press and hold the mobile phone icon and drag it to the left or right side of the associated device icon or the middle area to fit and adsorb, the service feedback will be different, which is the same as the effect described in the previous embodiment, that is, if the user drags the mobile phone icon Paste from the right area of the associated device icon and stick together, then the mobile phone screen will be projected on the right side of the associated device screen; if the user drags the mobile phone icon to paste it from the left area of the associated device icon, the adsorption together, the screen of the mobile phone will be projected on the left side of the screen of the associated device; if the user drags the icon of the mobile phone to the middle area of the icon of the associated device and snaps them together, the screen of the mobile phone will be projected in full screen on the screen of the associated device . In a word, the effect displayed by the scene-based service may be different according to the orientation of the connection between the core device icon and the associated device icon.

In some other embodiments, the dragging directions of the attracting device icon and the adsorbed device icon are exchanged, and the services or interactive functions executed by them may be different. If the data transmission direction is different, the device as the adsorbing party or as the adsorbed party indicates that the device is the sender or receiver of the data. For example, the data transmission direction is stipulated that the adsorbing device sends data streams to the adsorbed device. For example, when sharing files, if you press and hold the PC icon and drag to attach to the mobile phone icon, the corresponding service is to send the PC file to the mobile phone; if you press and hold the mobile phone icon and drag to attach to the PC icon, the corresponding The service is to send files from the phone to the PC.

For another example, the scenario-based services triggered by the device icon as the absorber or the absorber can also be different. For example, if you press and hold the PC icon and drag it to attach to the mobile phone icon, the triggered service is to send the files of the PC to the mobile phone; Screen mirroring to the PC screen.

Refer to the examples shown in Figure 15, Figure 16, and Figure 17. Figure 15, Figure 16, and Figure 17 describe that when the user drags the MateBook icon of the laptop computer close to the icon of the mobile phone to establish a multi-screen collaborative connection, different orientations and different areas will have different adsorption effects.

As shown in Figure 15, in the user interface 1500 of the mobile phone, the user drags the MateBook icon of the notebook computer to fit it from the left area 1501 of the mobile phone icon. When the MateBook icon of the notebook computer is located in the left area 1501, a prompt " Screen on the right” prompts the user to choose this option to mirror the phone screen on the right side of the laptop screen. When the user drags the icon of the notebook computer MateBook to the left area 1501 and then releases the finger, the icon of the mobile phone can be attached to the icon of the notebook computer MateBook. It can be seen that, on the laptop computer screen, the mobile phone display interface 1502 is projected on the right side of the laptop computer display interface.

As shown in Figure 16, in the user interface 1600 of the mobile phone, the user drags the MateBook icon of the notebook computer to fit it from the right area 1601 of the mobile phone icon. When the MateBook icon of the notebook computer is located in the right area 1601, a prompt " Screen on the left”, prompting the user to choose this option to mirror the mobile phone screen on the left side of the laptop screen. When the user drags the icon of the notebook computer MateBook to the right area 1601 and then releases the finger, the icon of the mobile phone can be attached to the icon of the notebook computer MateBook. It can be seen that, on the laptop computer screen, the mobile phone display interface 1602 will be projected on the left side of the laptop computer display interface.

As shown in Figure 17, in the user interface 1700 of the mobile phone, the user drags the icon of the notebook computer MateBook to the middle area 1701 of the mobile phone icon. ", prompting the user to select this option will make the phone fill the full screen and display it on the laptop screen. When the user drags the icon of the notebook computer MateBook to the middle area 1701 and then releases the finger, the icon of the mobile phone can be adsorbed or overlapped with the icon of the notebook computer MateBook. It can be seen that, on the laptop computer screen, the display interface 1702 of the mobile phone will be displayed in full-screen projection on the laptop computer.

When the core device is connected to the associated device to perform an interactive function, the contents of the interfaces displayed by the core device and the associated device may be the same or different.

In some embodiments, although the core device and the associated device display different interface content, the core device and the associated device can be used together, and the displayed interfaces are related instead of displaying irrelevant content.

As shown in FIG. 18 , in the user interface 1800 shown on the mobile phone, the mobile phone is connected to the smart TV, and the scene service area displays service options such as multi-screen collaboration, home theater, multi-screen conference, and somatosensory games. The user selects the home theater service option 1801, and the mobile phone and the smart TV can display a page related to the home theater service.

As shown in Figure 18, the smart TV plays a video, and the video screen is displayed on the interface 1806, and the current application title "Home Theater" is displayed on the top of the video screen, as well as the prompt "This device has been coordinated with HuaWei P50", weather, time wait. The mobile phone user interface 1810 displays the remote control interface of the home theater application. The user interface 1810 displays the title "Home Theater", a prompt 1802 "This device has been coordinated with Vision, click to remotely control Vision", a remote control operation area 1803, and a switching control 1804 wait. The remote control operation area 1803 can be used for the user to control the smart TV, and can include homepage control, switch control, volume up control, volume down control, return control, menu control, voice control, confirmation control, up, down, left, and right direction controls, etc.

The switching control 1804 is used to switch between the video screen and the remote control interface on the mobile phone interface, and the switching control 1804 prompts "switch to the display screen" on the remote control interface. As shown in FIG. 19 , when the user clicks on the switch control 1804 , the mobile phone can display a video image 1901 and a switch control 1902 on the user interface 1900 . The switching control 1902 on the video playback interface of the mobile phone prompts “Switch to the remote control interface”. If the user clicks the switching control 1902, the mobile phone can return to display the remote control interface, that is, the user interface 1810 .

In other embodiments, after the core device cooperates with the associated device, some functions of the core device can be migrated to the associated device, and the associated device can provide services, and the core device can hide the service in the background and run other applications in the foreground. For example, after a mobile phone and a smart TV are coordinated, the service of playing video is provided by the smart TV, and the user can hide the service in the background, open the chat application in the foreground, and watch the video while chatting.

It can be understood that the user interfaces described above in FIG. 4 to FIG. 19 are only exemplary interfaces for assisting readers in understanding the technical solutions described in the present application, and do not limit the user interfaces in other embodiments of the present application. In other embodiments, more or fewer user interfaces may be added or removed, or more or fewer controls may be added or removed, or different human-computer interaction operations may be designed according to actual conditions, so that the user interface is more suitable for the user experience.

With reference to the foregoing embodiments shown in FIGS. 1 to 19 , a method for device interaction provided by an embodiment of the present application is introduced below.

The following method embodiments may be applied in a communication system composed of multiple devices, such as the aforementioned communication system 10 . The communication system may also be referred to as a HyperTerminal. These multiple electronic devices can specifically be mobile phones, tablet computers, notebook computers, personal computers (personal computers, PCs), smart TVs (also called smart screens, large screens, etc.), or wearable devices such as smart watches and smart bracelets. Equipment, this application does not limit the type of equipment.

A communication connection is established between the core device and the associated device, through which communication can be performed to transmit data or instructions. The communication connection may be a wired connection or a wireless connection, or a combination of several connection methods, which is not limited in this embodiment.

Each terminal device in the communication system can be equipped with

system,

system, HarmonyOS or other types of operating systems, the operating systems of each terminal device in the communication system may be the same or different, which is not limited in this application. In some embodiments, each terminal in the communication system is equipped with HarmonyOS, so the communication system may be called a HarmonyOS super virtual device (super virtual device), or a HarmonyOS super terminal.

In the following method embodiments, the core device/central device (such as a mobile phone) is the control center of the hyperterminal system, carrying functions such as device management, service management, and service scheduling, and can be connected with one or more other associated devices (such as a notebook computer, PCs, tablets, smart watches, smart TVs, etc.) to connect and collaborate, to provide users with a variety of scenario-based services and a variety of atomized service combinations.

In some embodiments, a hyper terminal application is installed on the core device or the associated device, so as to facilitate the user to manage each terminal device in the hyper terminal system.

It can be understood that the user can interact on any device installed with the HyperTerminal application. The instructions and data entered by the user in the HyperTerminal application on any device can be transferred to other devices in the HyperTerminal system synchronously.

FIG. 20 is a flow chart of the device interaction method provided in this embodiment, which specifically includes the following steps:

S101. The first device displays a first interface, where the first interface includes a first control indicating the first device and a second control indicating a second device.

The first control is a device option indicating the first device, and the second control is a device option indicating the second device.

The first device may be the aforementioned core device, the second device may be the aforementioned associated device, and the second device is a peripheral device discovered by the first device.

The first interface is the hyper terminal interactive interface, and the user can control and manage each device in the hyper terminal system through user operations acting on the first interface. This embodiment does not impose any limitation on the presentation form of the hyper terminal interactive interface. For one form of expression, reference may be made to the embodiment described in FIG. 5 .

The first control is an icon indicating the core device, such as the mobile phone icon shown in FIG. 5 . The second control is an icon indicating an associated device, such as the smart watch icon shown in FIG. 5 .

Referring to FIG. 5 , a device connection status diagram may be displayed in the hyperterminal interactive interface. The center of the device connection status diagram is a core device icon surrounded by one or more associated device icons.

The device connection state diagram may indicate the connection relationship, relative position relationship, signal strength relationship and the like between the core device and the associated device.

The connection relationship can be referred to as shown in Figure 7. The icon of the mobile phone and the icon of the smart watch are attached together, indicating that the mobile phone and the smart watch are connected and can work together to provide system-level services. However, the smart bracelet icon is free from the periphery of the mobile phone icon, and is not attached to, overlapped or attached to the mobile phone icon, but separated, indicating that the mobile phone and the smart bracelet have not established a connection.

In some embodiments, the relative position of the associated device icon and the core device icon displayed in the device connection status diagram may indicate the positional relationship between the associated device and the core device in real space, such as relative orientation information, relative distance information, etc. It is helpful for users to quickly find and distinguish each electronic device. For example, the orientation of the core device and the associated device in the real space is consistent with the orientation of the core device icon and the associated device icon in the device connection status diagram. For another example, in the real space, the farther the associated device is from the core device in the real space, the farther the distance between the associated device icon and the core device icon displayed on the interface is. This embodiment does not limit the specific technology for realizing this function. For example, the mobile phone can obtain the orientation and measured distance of the directional information carried by the associated device based on UWB technology, and obtain the spatial mapping relationship between the associated device and the mobile phone in real space.

S102. The first device detects a first user operation.

If the user wants the second device to form a hyper terminal with the first device to realize some collaborative functions, the first user operation of selecting the second device can be implemented. This embodiment does not impose any limitation on the specific manner of the first user's operation. The first user operation is for the user to associate the first device with the second device.

In one implementation, the first user operation may be a user operation acting on the second control. For example, the user presses and holds the second control and drags the second control close to the first control. When the second control reaches the designated area Release the second control. In some embodiments, the designated area may be a circular area centered on the first control and having a radius of the first radius distance.

With reference to the embodiment shown in Figure 6, the user can select the smart watch icon 506 corresponding to the smart watch on the super terminal interface, and drag the smart watch icon 506 close to the mobile phone icon 505, when the smart watch icon 506 and the mobile phone icon 505 When the distance between them is less than or equal to the second distance (R2 as shown in Figure 6), or, when the smart watch icon 506 enters the specified area (in Figure 6, it is an annular area formed by the nearest circle around the mobile phone icon, that is, The mobile phone icon 505 is the center and the radius is a circular area of R), and the user releases the finger to display the device connection status diagram 701 shown in FIG.

Wherein, in some embodiments, the selected device icon may change its position along with the trajectory of the user's finger dragging on the screen. This embodiment does not limit the finger movement trajectory when the user drags the device icon, and the user's dragging trajectory may be any curve. For example, when the user selects an associated device icon, and the user drags the associated device icon on the screen for a certain distance or stays on a certain position on the screen for a second preset time, the mobile phone can detect the current association between the associated device icon and the mobile phone. Whether the distance between the icons is less than or equal to a certain distance, such as the second distance, if so, the associated device icon and the mobile phone icon are adsorbed together. When the user's finger moves, the associated device icon moves along with the track of the user's finger.

In another implementation manner, the first user operation may be that the user clicks the second control in an unconnected state, or clicks the first control and the second control at the same time, which may trigger the connection between the second device and the first device. The interface shows that the icon of the second device is attracted to the icon of the first device.

In another implementation, the first user operation may also be a user operation acting on the first control, for example, the user presses and holds the first control and drags the first control close to the second control, and when the first control reaches the specified When releasing the first control in the region, the first control and the second control can also be adsorbed together.

Optionally, in another implementation, the first user operation may also be a user operation of selecting the second control, such as pressing and holding the second control, refer to Figure 11, Figure 12, Figure 13A, Figure 13B, Figure 13C, Figure 13D depicts the embodiment.

S103, in response to the first user operation, the first device displays a second interface, and the second interface displays one or more service options of scenario-based services, where the scenario-based services can be provided by the cooperation of the first device and the second device services.

Optionally, the second interface displays that the first control is connected to the second control. In response to the first user operation, the second control may be displayed on the second interface to be adsorbed together with the first control. The two device icons being attracted together may refer to that the edges of the icons corresponding to the two devices are tangent, or the two device icons being adsorbed together may also refer to that the two device icons are completely or partially overlapped. In this embodiment, no special limitation is imposed on the form of adsorption.

For the second interface in this embodiment, refer to the user interface 700 shown in FIG. 7 .

In some embodiments, the second control in the second interface may have a different form from the second control in the first interface. For example, in FIG. 7 , the smart watch icon 702 that has been connected to a mobile phone to form a hyperterminal can display a distinctive logo, so as to be easily distinguished from the associated device icon in an unconnected state. For example, the smart watch icon 702 is displayed in a dark filled form, indicating that the smart watch is connected to the mobile phone. In addition, the size of the smart watch icon 702 is smaller than that of the smart watch icon 506 when it is not connected. The diameter of the round icon of the smart watch icon 702 is R2 is adsorbed on the edge of the mobile phone icon 505 , and the smart watch icon 702 is tangent to the edge of the mobile phone icon 505 .

Optionally, in another implementation manner, referring to the embodiments described in FIG. 11, FIG. 12, FIG. 13A, FIG. 13B, FIG. 13C, and FIG. A floating control is displayed on the second interface, and the floating control has one or more areas, and each area corresponds to a scenario-based service option. The floating control is represented as multiple circular areas, or multiple rectangular areas, or multiple fan-shaped areas, or multiple fan-shaped ring areas, or multiple circular ring-shaped areas, or multiple polygonal areas. In the second interface, the first control may or may not be displayed. This embodiment does not limit the layout of the floating control.

Referring to the user interface 1100 shown in FIG. 11, or the user interface 1200 shown in FIG. 12, or the embodiments shown in FIG. 13A, FIG. 13B, FIG. 13C, and FIG. 13D, a floating control is displayed in the second interface. Controls can be divided into one or more areas, each area corresponds to a scene-based service option, and different scene-based service options correspond to different areas. This way of interaction is more interesting, more intuitive, and the interface is more concise. This embodiment does not limit the shape of the floating control, nor does it limit the way of dividing regions.

For example, in the user interface 1100, when the user presses the smart watch icon, the super terminal interface can display a circular floating area 1101 with the mobile phone as the center. The circular floating area 1101 can be divided into multiple areas, and each area corresponds to A scenario-based service option. For example, in the user interface 1100, the circular floating area 1101 is divided into four areas, corresponding to scene-based service options: outdoor running, indoor running, heart health, and blood vessel health.

One or more scenario-based service options are displayed on the second interface, and the scenario-based service mentioned here is a service that can be provided by the first device and the second device in cooperation.

Referring to the descriptions in the foregoing embodiments, the scenario-based service refers to a functional service based on a usage scenario, which is a collection of multiple functions or services that can be provided for a specific scenario. A scene service can be supported by one or more atom services. The service options of the scene-based service displayed on the second interface may be the scene-based service provided by the current hyper terminal system, that is, the combination of the first device and the second device, on the first device or on the second device The atomic capabilities corresponding to these scenario-based services have been installed and deployed.

Scenario-based services can be based on the type of core device and/or associated devices, device features, product positioning, device usage, collaborative services that the device can provide, the current environment of the device, the operating status of the device, and the most recently used Based on the application, the frequency of the user using the scenario-based service, the currently running application, the sorting or default options of the scenario-based service set by the user, etc., the scenario-based service options obtained after comprehensive consideration are displayed on the user interface, which is convenient for the user to quickly Start the related service or function. The service options of the scenario-based service with a higher probability of being selected by the user calculated by the hyperterminal system are recommended and displayed with higher priority.

For another example, different device types have different device characteristics. For example, smart TVs have larger screens, which provide users with a better visual viewing experience and are more suitable for watching videos; smart speakers have strong audio output capabilities and better speaker effects. It is more suitable for playing audio; the earphone is small in size, easy to carry, does not need to be placed outside, does not disturb people around, and can also have a noise reduction effect, shielding external environmental noise, suitable for listening to audio in public places, such as talking in public places, listening to music, etc.; Smart watches and smart bracelets can measure the user's exercise data or heart rate and other health information more accurately than mobile phones; laptops have powerful processing capabilities and are more suitable for office work. The device type and device features will not be described in detail here, and the above examples do not limit other embodiments.

In some embodiments, when listing the scenario-based service options, the probability of each scenario-based service option being selected can be calculated according to the user's recent use frequency of different collaborative services or the environment in which the current device is detected. In the customized service options area, the scenario-based service options that are estimated to be selected with a higher probability are displayed first.

Optionally, in some embodiments, the HyperTerminal system may include a first database, the first database records one or more scenario-based services supported by one or more device combinations, the device combination includes the first device and The associated second device constitutes a device combination, and the first scenario-based service is a scenario-based service queried from the first database. The first database can be stored locally or on a cloud server. The scenario-based service is determined by one or more of the following parameters: device type of device combination, device features, product positioning, device usage, environment or scenario of the device, status of the device, recently run applications, etc.

Optionally, in some embodiments, the hyperterminal system may include a second database, the second database records one or more scenario-based services supported by one or more atomic service combinations, and the first scenario-based service is Scenario-based services queried from the second database. The second database can be stored locally or on a cloud server. The atomic service combination includes an atomic service combination composed of atomic capabilities of the first device and its associated second device. Among them, the atomic capability includes one or more of the following: audio output capability, audio input capability, display capability, camera capability, touch input capability, keyboard and mouse input capability, etc. For example, the audio output capability may include whether the device supports mono or multi-channel, supported sound effects, supported frequency response range, noise reduction capability, audio analysis capability, etc. when the device is playing audio; the audio input capability may include receiving Acoustic range, noise reduction capability, etc.; display capability may include the device’s screen size, display resolution parameters, refresh rate, color expressiveness, etc.; camera capability may include the device’s camera type, shooting pixels, night scene shooting capability, and image adjustment capability, etc.; touch input capability, keyboard and mouse input capability refer to whether the device can support touch input or keyboard and mouse input, etc.

In this embodiment, the layouts of the floating controls are different, and the sequence of the recommended scene-based services is also different.

Optionally, referring to the embodiments shown in FIG. 13C and FIG. 13D , the shape of the floating control is circular or fan-shaped, where multiple circular areas or fan-shaped areas respectively indicate multiple scene-based services, the closer to the center of the circle The scene-based service indicated by the ring-shaped area or fan-shaped area of , is the recommended scene-based service with higher priority.

Of course, the scenario-based service options are not static, and the hyper terminal system can continuously update, expand or refine more scenario-based service options, which are not limited in this embodiment, and can be changed according to actual conditions. In some embodiments, the user can customize and add scene-based services, for example, the user sets a name for a usage scene and associates the scene with other application services or system services. The newly added scenario-based service option is displayed in parallel with other scenario-based service options on the HyperTerminal interface.

For specific examples, reference may be made to the descriptions of the embodiments shown in FIG. 7 and FIG. 9 , which will not be repeated here.

In some embodiments, when the devices are connected to form a hyper terminal, a certain scenario-based service may be automatically selected as a default service. For example, set the first-ranked scenario-based service option in the scenario-based service option area as the default service. When devices are connected to form a hyper terminal, each device will be configured according to the atomic service combination corresponding to the scenario-based service by default. If the user wants to choose other scenario-based services, he can choose other scenario-based service options. Setting the default scene-based service can facilitate the seamless switching of services on devices and save user operations to a certain extent. For example, when the headset is connected to the mobile phone, the default service setting is that the headset provides audio output and audio input functions. Then when the headset is successfully connected to the mobile phone, the audio output and audio input functions are immediately switched to the headset, and the user does not need to perform additional operations. , saving the tedious operation. The setting of the default service may be a default service selected by the user, including scenario-based services and/or atomic services. The setting of the default service may also be that the system uses the user's frequently-used services obtained from statistics as the default service, etc. , which is not limited in this embodiment.

In the second interface, a recommended associated device may also be displayed, which may also be called a third device. The HyperTerminal system can also analyze the current scenario based on the type of devices currently forming the HyperTerminal system, device features, product positioning, device usage, device environment or scene, device status, and recently running applications, and list recommendations to users. Other devices suitable for the current scene to help users obtain a better scene experience; or speculate on more scalable usage scenarios, and then list and recommend other devices to the user. If the user adopts the device recommended by the system, it can further expand more, Richer usage scenarios.

Optionally, the third device is a device discovered by the first device, and the device option of the third device is clicked, and the first device displays the service options of the second scenario-based service. Among them, the service options of the second scenario-based service may be that some scenario-based service options are added on the basis of the service options of the first scenario-based service, or the service options of the second scenario-based service are re-updated scenario-based services, It is different from the service options of the first scenario-based service.

Optionally, the third device is not the device discovered by the first device. Click the device option of the third device, and the first device will jump to display the fifth interface. For example, the fifth interface can be the purchase interface of the third device, and the fifth interface includes A plurality of item options, the plurality of item options come from multiple sources, the plurality of item options includes an item option indicative of a third device.

In the example shown in FIG. 7, the hyperterminal system can analyze the user's usage scene as a sports and health scene based on the current devices that make up the hyperterminal as a mobile phone and a smart watch. The recommended device area 704 can display recommended sports equipment or health monitoring equipment, etc. Such as treadmills, spinning bikes, blood glucose meters, heart rate meters, etc. When the user selects (clicks or drags) the device icon in the recommended device area 704, if the device is an associated device detected by the current core device, that is, the device displayed in the device connection status diagram 701, then the device can initiate a connection Join HyperTerminal. When the user selects (clicks or drags) the device icon in the recommended device area 704, if the device is not displayed in the device connection status icon 701, it can jump to a page related to the device product, such as introducing the device to the user. The function of the recommended device, the user experience, how to use and expand the experience when forming a hyperterminal with other devices, etc.

S104, the first device detects a second user operation for selecting the first contextualized service, and the first contextualized service instructs the first device to provide the first service, and/or the second device to provide the second service.

The first scenario-based service corresponds to a first atomized service combination, and the first atomized service combination provides at least one first service for the first device, and/or at least one second service for the second device. A scene service can be supported by one or more atom services.

Atomic service refers to the smallest capability unit that can run independently. It is a concept of abstract encapsulation of a single function/capability. It can be a hardware service or a software service.

This embodiment does not limit the operation of the second user. Referring to the embodiment shown in FIG. 8 , the second user operation may be that the user clicks on the first scenario-based service option.

Optionally, refer to the embodiments described in FIG. 11 , FIG. 12 , FIG. 13A , FIG. 13B , FIG. 13C , and FIG. 13D . The second user operation may be that the user drags the second control to the area corresponding to the first scene-based service option, and then releases it; the second user operation may also be that the user drags the second control to correspond to the service option of the first scene-based service area close to the first device icon etc. In this embodiment, there is no limitation on the motion track dragged by the user.

In some embodiments, the aforementioned floating control may include a first area, and the first area displays service options of the first scenario-based service. The second user operation is the operation of releasing the second control after the user drags the second control to the first area, or the second user operation is the user dragging the second control through the first area and moving towards the direction close to the first control After arriving at the specified area, release the operation of the second control. The specified area can be a circular area with the first control as the center and a radius equal to the first radius distance.

For example, in the example shown in Figure 11, if the user wants to select a scene-based service option, such as an outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then select the service option via the outdoor running service option. The corresponding area is close to the direction of the mobile phone icon, and the trajectory of the user's finger can refer to the schematic trajectory in the user interface 1100 . When the smart watch icon reaches the adsorption area, the user releases the finger, the service corresponding to the outdoor running starts, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . Of course, if there is only one scene-based service option, the circular floating area 1101 does not need to be divided into blocks, and the user will select the same scene-based service when approaching the mobile phone icon from any direction.

As another example in the example shown in Figure 12, if the user wants to select a scenario-based service option, such as the outdoor running service option, the user can drag the smart watch icon to the area corresponding to the outdoor running service option, and then release the finger , which means that the outdoor running service option is selected, the service corresponding to the outdoor running is started, and the mobile phone jumps to display the outdoor running interface shown in the user interface 800 . The trajectory of the user's finger can refer to the schematic trajectory in the user interface 1200 . Of course, if there is only one scene-based service option, the circular floating area 1201 does not need to be divided into blocks, and the user drags the device icon to any position in the floating area 1201 and releases the icon to select the same scene-based service.

The hyper terminal system can be based on the current device combination, the atomized services that can be provided by each device that can be queried, and then based on the scene service actually selected by the user, or based on the user's usual service combination, or the default service combination set by the user, or based on the life Cognition is the most suitable service combination in this scenario, or the most frequently used service combination in this scenario is obtained from the server, and the atomic service combination in this scenario is automatically set, saving users from tedious selection operations.

For example, after a mobile phone and a smart TV form a super terminal, if the scene service selected by the user is a multi-screen conference, the atomic service combination is automatically set as follows: the audio output function is provided by the smart TV, the audio input (microphone) function is provided by the mobile phone, and the camera The function is provided by the smart TV, and the display function is provided by the smart TV. For another example, after a mobile phone, a smart watch, earphones, and a treadmill form a super terminal, if the scene service selected by the user is indoor running, the atomic service combination is automatically set as follows: the audio output function is provided by the earphone, and the audio input (microphone) function is provided by the The earphones provide, the camera function is provided by the mobile phone, and the display function is provided by the treadmill.

Optionally, the first device displays the first atomic service combination on the seventh interface, the first atomic service combination is selected by the user, or the first atomic service combination is determined according to one or more of the following parameters Service combination: the device type and device characteristics of the first device and/or the second device, or the atomic service combination commonly used by the user, or the default atomic service combination set by the user, or the analyzed service that is most suitable for the first scenario Atomic service combination, or the most frequently used atomic service combination obtained from the server under the first scenario service.

Referring to the aforementioned example shown in FIG. 10, if the user selects the atomic service entry option in the user interface 910, one or more atomic service combinations may be displayed, as shown in the user interface 1000 and the user interface 1010 shown in FIG. 10, Users can choose different atomic services to be provided by different devices.

S105, the first device and/or the second device runs the first scenario-based service, that is, the first device runs the first service, and/or the second device runs the second service.

In this embodiment, there is no limitation on the way of knowing the scenario-based service run by the first device and/or the second device. In some embodiments, the first device may directly notify the second device to run the atomized service corresponding to the first scenario-based service through the communication connection. In some other embodiments, the first device may send an instruction to notify the operation of the first scenario-based service to the cloud server, or the first device may report the instruction or message obtained by itself or its own running status to the cloud server, Then the cloud server generates an instruction to run the first scenario-based service, and sends it to the second device. After receiving the instruction to run the first scenario-based service, the second device runs the atomized service indicated by the first scenario-based service.

Wherein, the third interface displayed by the first device running the first scenario-based service may be different from or the same as the fourth interface displayed by the second device running the first scenario-based service.

Optionally, the first scenario-based service is a motion monitoring service, the second device is a wearable motion device, the third interface can be a picture of a motion track, and the fourth interface can be the detected motion data of the user, and the motion data can be Including: exercise speed, exercise distance, exercise time, heart rate, etc.

Referring to the example shown in FIG. 8 , when the mobile phone detects that the user clicks and selects the outdoor running option 707 listed in the scene service option area 705 in the user interface 700 , the mobile phone can jump to an outdoor running related page, as shown in the user interface 800 . The user interface 800 is a sports and health application page, and you can see that there are sub-options displayed under the sports and health title bar: outdoor running sub-option 801 , indoor running sub-option, heart health sub-option, and vascular health sub-option. Since the user selects the outdoor running option 707 in the user interface 700, the page corresponding to the outdoor running sub-option 801 is currently displayed on the user interface 800, such as a map icon, a user's positioning icon, and a user's exercise route. , Statistical user exercise volume, etc.

Optionally, referring to the embodiment shown in FIG. 18 , the first scenario-based service is a screen projection service, the second device is a large-screen device, the fourth interface is a video or image screen, and the third interface is to control the video or image The control interface of the screen display function.

Optionally, the screen content displayed on the third interface and the fourth interface may be the same. Referring to the embodiment shown in FIG. 19 , both the mobile phone and the TV display the same video screen.

Optionally, when the user chooses to return to the HyperTerminal page, a second control may be displayed that is adsorbed together with the first control. The two device icons being attracted together may refer to that the edges of the icons corresponding to the two devices are tangent, or the two device icons being adsorbed together may also refer to that the two device icons are completely or partially overlapped. In this embodiment, no special limitation is imposed on the form of adsorption. For example, referring to the user interface 1400 shown in FIG. 14 , the device connection status icon area on the hyperterminal page shows that the smart watch icon is attached to the mobile phone icon, indicating that the smart watch has established a connection with the mobile phone and can work together.

In some embodiments, when the user drags the associated device icon to attach to the core device icon from different orientations and different areas, the resulting effects and triggered services may also be different.

In the screen projection scenario, the first device determines the position of the screen interface of the first device on the screen of the second device according to the relative position of the second control and the first control when the drag operation is released. For details, reference may be made to the embodiments described in FIG. 15 , FIG. 16 , and FIG. 17 .

If the relative position is that the second control is on the left side of the first control, then the screen interface of the first device is projected on the right side of the screen of the second device; if the relative position is that the second control is on the right side of the first control, then The screen interface of the first device is projected on the left side of the screen of the second device; if the relative position is that the second control is in the middle of the first control, then the full screen of the screen interface of the first device is projected on the screen of the second device.

If the user wants to detach an associated device from the HyperTerminal system or end the service, he can operate the fourth user operation on the HyperTerminal interface under the condition that the icon of the associated device is attached to the core device. The fourth user operation includes Operation of the first control separately from the second control. For example, select the second control, and drag the second control to move away from the first control until it moves out of the specified area, then release the operation of the second control. In one example, the user can select the associated device icon and drag the associated device icon away from the core device icon. When the mobile phone detects that the user drags the icon of the associated device away from the icon of the core device for a distance greater than the third distance, when the user releases the finger, the icon of the associated device returns to the position before it was not connected, the adsorption state is released, and the associated device is also disconnected If the connection with the core device is separated from the HyperTerminal system, the service provided by the device in the HyperTerminal will also be terminated. Or, when the user clicks an associated device that is in the connected state, or the user presses an associated device icon and a core device icon, and drags them in opposite directions to separate them, the associated device and the core device can be triggered. Disconnection means that the icon of the associated device is separated from the icon of the core device on the interface, and the icon of the associated device returns to the unconnected state. In this embodiment, there is no limitation on conditions or user operations for triggering adsorption and releasing adsorption.

It can be understood that the hyperterminal is not limited to the connection between the associated device and the core device. In other embodiments, the user can also connect an associated device with another associated device in the hyperterminal interface to form a hyperterminal. That is to say, the user can connect any two or more devices, and the connected devices are not limited to associated devices or core devices. For example, associated devices, smart speakers and smart TVs can be directly connected to form a super terminal. For functions, schematic interfaces or user operations, reference may be made to the descriptions in the foregoing embodiments, and details are not repeated here.

In some embodiments, the first device detects a third user operation acting on the second interface, the third user operation is used to associate the fourth device with the first device, and the third user operation includes selecting the third user operation indicating the fourth device. Three control operation. In response to the third user operation, the first device displays a sixth interface, the sixth interface displays the first control, the second control, and the third control are adsorbed together, and the sixth interface also displays a third scene-based service option, The third scenario-based service option is determined according to the first device, the second device, and the fourth device, and the third scenario-based service option is different from the first scenario-based service option. Referring to the embodiment shown in FIG. 9 , details are not repeated here.

It can be seen from the above that the embodiment of the present application provides a unified entrance for the interaction between the core device and multiple associated devices, that is, the hyper terminal interface, and the user can trigger any two or more electronic devices in the hyper terminal interface to form a hyper terminal . In addition, the HyperTerminal can judge the usage scenarios based on the device type, device features, product positioning, common functions, device status, current running applications, current device environment, etc., and provide one or more scenarios based on the usage scenarios. service options, atomized service composition options, or interactive function options. Users can quickly select scene-based services and/or atomized services according to their personal needs, without cumbersome decentralized settings. Therefore, it can solve the problem that the settings of interactive functions between electronic devices are scattered and cumbersome, and it is impossible to list rich uses for users. scene problem.

With reference to the foregoing embodiments, the following describes the functional modules of the communication system 10 provided by the embodiments of the present application.

Referring to FIG. 21 , FIG. 21 shows functional modules of the communication system 10 . The communication system 10 may include a device combination generating module 2101, a scenario service decision module 2102, a scenario service providing module 2103, an atomic service providing module 2104, and the like. Wherein, the communication system 10 may also be called a hyper terminal, and the hyper terminal may include a core device and one or more associated devices. Regarding the explanations of the hyper terminal, the core device, the associated device, etc., reference may be made to the foregoing embodiments, and details are not repeated here.

Among them, the device combination generation module 2101 can be used for the combination of the core device and the associated device in the hyper terminal, etc., including realizing the discovery of the surrounding associated devices by the core device, and indicating the positional relationship or signal strength between the associated device and the core device on the interface Relationships, etc., as well as functions such as connecting core devices and associated devices. A hyper terminal interface may be displayed on the core device, and the discovered surrounding associated devices may be displayed on the hyper terminal interface. In some embodiments, the user can drag the associated device icon close to the core device icon, and when reaching the specified area, the associated device icon can be attached to the edge of the core device icon, indicating that the core device has established a connection with the associated device. Regarding the connection between the core device and the associated device, user operations, schematic interface, etc., you can refer to the above-described embodiments such as FIG. 5 , FIG. 6 , FIG. 15 , FIG. 16 , and FIG. 17 , and will not repeat them here.

The scene service decision module 2102 can be used to base on the currently connected device type and/or device features, product positioning, common functions and other information, or according to the detected environment where the user is currently in, or the operating status of the device, etc., Analyze current user usage scenarios. Scenario service decision module 2102 can classify multiple consumer usage scenarios. In an example, consumer usage scenarios may include the following five categories: mobile office, smart home, audio-visual entertainment, sports and health, and smart travel. Of course, the classification of consumer usage scenarios is not static, and the scenario service decision module 2102 can continuously update, expand or refine more usage scenarios, which are not limited in this embodiment, and can be changed according to actual conditions. For example, the addition of associated devices or functions that have not appeared before may lead to new usage scenarios. For specific description, reference may be made to the above-described embodiments, and details are not repeated here.

The scene service providing module 2103 is used to, when multiple devices are connected, according to the device type, device characteristics, product positioning, device usage, collaborative service that the device can provide, the current environment of the device, the device One or more scenario-based service options based on comprehensive consideration of running status, recently used applications by users, currently running applications, location information input by users, etc., and displayed on the user interface, so that users can quickly start related services or function.

In some embodiments, when listing the scenario-based service options, the scenario service providing module 2103 may detect the environment where the current device is located, or the application recently run by the user, etc. The probability of each scenario-based service option being selected is calculated, and the scenario-based service option that is estimated to have a higher probability of being selected is displayed first in the scenario-based service option area. Of course, the scenario-based service options are not static, and the scenario-based service providing module 2103 can continuously update, expand or refine more scenario-based service options, which are not limited in this embodiment, and can be changed according to actual conditions. In some embodiments, the user can customize and add scene-based services, for example, the user sets a name for a usage scene and associates the scene with other application services or system services. The newly added scenario-based service option is displayed in parallel with other scenario-based service options on the HyperTerminal interface. The scenario service providing module 2103 may comprehensively consider one or more of the above factors, and reasonably recommend scenario-based service options.

The scenario service providing module 2103 may also be used to recommend associated devices. The scene service providing module 2103 can analyze the current scene based on the types of devices that currently make up the hyper terminal system, and list and recommend other devices suitable for the current scene to the user, so as to help the user obtain a better scene experience; or speculate more scalable usage Scenarios, and then list and recommend other devices to the user. If the user adopts the device recommended by the system, more and richer usage scenarios can be further expanded.

For more descriptions of the functions that the scene service providing module 2103 can provide, please refer to Figure 7, Figure 8, Figure 9, Figure 11, Figure 12, Figure 13A, Figure 13B, Figure 13C, Figure 13D, Figure 15, Figure 16, The descriptions of the foregoing embodiments such as FIG. 17 , FIG. 18 , and FIG. 19 will not be repeated here.

The atomic service providing module 2104 can be used to provide atomized services based on the current device combination and each device that can be queried, and then based on the scene service actually selected by the user, or based on the user's common service combination, or the default service set by the user Combination, or the service combination that is most suitable for the scenario based on life cognition, or the service combination that is most frequently used in the scenario is obtained from the server, etc., and the atomic service combination in the scenario is automatically set, saving the user from tedious selection operations . Atomic service refers to the smallest capability unit that can run independently. It is a concept of abstract encapsulation of a single function/capability. It can be a hardware service or a software service.

In some embodiments, the atomic service providing module 2104 supports users to personalize the atomic service combination, and the user can manually modify the atomic service combination according to personal preference or actual situation, and adjust a certain atomic service to be provided by other devices. For example, in a multi-screen conference, if the surrounding environment is noisy, the user can change the audio output function provided by the smart TV to the headset.

For more functional descriptions of the atomic service providing module 2104, reference may be made to the descriptions of the foregoing embodiments such as FIG. 10 , and details are not repeated here.

In some embodiments, the electronic devices that can be added to the HyperTerminal may include: devices with the same account, devices with different accounts, and devices without accounts.

The aforementioned device with the same account may include an electronic device whose login account is the same as the login account of the core device of the hyper terminal. For example, the accounts logged into Zhang San's mobile phone and Zhang San's tablet computer are the same account, that is, Zhang San's account. In the case that the core device of the hyperterminal is Zhang San's mobile phone, Zhang San's tablet computer can join the hyperterminal as a device with the same account.

The device with the same account may also include an electronic device that is paired with the core device of the hyper terminal and associated with the login account on the core device. In some embodiments, some electronic devices (such as earphones, smart watches, smart bracelets, etc.) do not have the ability to independently log in to an account. The user can pair the above-mentioned electronic device through a mobile phone (or an electronic device such as a tablet computer capable of independently logging in to an account), thereby associating the above-mentioned electronic device with the account registered on the mobile phone, so as to facilitate the control and management of the above-mentioned electronic device through the mobile phone. The aforementioned pairing method may include Bluetooth pairing, Wi-Fi pairing, and the like. The embodiment of the present application does not limit the above pairing method. For example, Zhang San's headset does not have the ability to independently log in to the account. Zhang San's mobile phone is paired with Zhang San's headset. Then, Zhang San's headset is associated with the login account on Zhang San's mobile phone (that is, Zhang San's account). That is, Zhang San's earphone may be an electronic device managed under Zhang San's account. In the case that the core device of the hyperterminal is Zhang San's mobile phone, Zhang San's earphone can be added to the hyperterminal as a device with the same account.

The above-mentioned cross-account device may include an electronic device whose login account is different from the login account of the core device of the hyper terminal. For example, the login account on Zhang San's mobile phone is Zhang San's account. The login account on Li Si's tablet computer is Li Si's account. Zhang San's mobile phone login account is different from Li Si's tablet computer login account. In the case that the core device of the hyperterminal is Zhang San's mobile phone, Li Si's tablet computer can join the hyperterminal as a cross-account device.

The above-mentioned cross-account device may also include an electronic device associated with an account through pairing with other electronic devices, and the associated account is different from the login account of the core device of the HyperTerminal. For example, the login account on Li Si's mobile phone is Li Si's account. Li Si's earphone is paired with Li Si's mobile phone, thereby being associated with Li Si's account. In the case that the core device of the hyperterminal is Zhang San's mobile phone (the login account is Zhang San's account), Li Si's earphone can be added to the hyperterminal as a cross-account device.

The aforementioned device without an account may be an electronic device without a login account and no associated account. After other electronic devices (such as mobile phones, tablet computers, etc.) establish a communication connection with the above-mentioned unaccounted devices, they can cooperate with the above-mentioned unaccounted devices to provide users with scene-based services. The aforementioned device without an account may be equivalent to a public device. For example, there is no login account in the printer, and there is no associated account. In the case that the core device of the hyperterminal is Zhang San's mobile phone (the login account is Zhang San's account), the printer can be added to the hyperterminal as a device without an account.

The embodiment of the present application does not limit the types of the above-mentioned same-account device, cross-account device, and non-account device.

In some embodiments, when the above-mentioned cross-account device joins the hyperterminal, device binding may be performed first. Wherein, the core device of the hyper terminal may request to be bound with the above-mentioned cross-account device. After the above-mentioned cross-account device agrees to bind, the core device can be bound with the above-mentioned cross-account device, so that the above-mentioned cross-account device can join the hyperterminal. The HyperTerminal can use the above-mentioned cross-account devices to provide users with scenario-based services.

It can be understood that the user to which the above-mentioned cross-account device belongs is usually not the same user as the user to which the core device of the HyperTerminal belongs. If the user who belongs to the core device wants to use the above-mentioned cross-account device to provide scenario-based services, he needs to obtain the consent of the user who belongs to the cross-account device. Then, when the above-mentioned cross-account device is added to the hyperterminal, the core device can perform device binding with the above-mentioned cross-account device to request the user to which the cross-account device belongs to agree that the user to which the core device belongs to use the cross-account device. This can reduce the situation that the user who belongs to the cross-device account does not agree or know that his electronic device is used by other users, and improves the user experience.

In some embodiments, when the above-mentioned device with the same account joins the hyperterminal, the above-mentioned device binding process may not be performed. It can be understood that the user to which the above-mentioned device with the same account belongs is usually the same user as the user to which the core device of the hyper terminal belongs. If the user who belongs to the core device wants to use other devices of his own to cooperate with the core device, he can use the method shown in Figure 5 to Figure 8 to enable the core device to cooperate with the device with the same account to provide scene-based services. In this way, the user uses his or her multiple electronic devices to cooperate without device binding. This can simplify the user's operation of using the HyperTerminal to provide scene-based services and improve the user experience.

In some embodiments, the above-mentioned unaccounted device joins the hyperterminal, and device binding may be performed, or device binding may not be performed.

The following specifically introduces a scenario in which an electronic device joins a hyper terminal and performs device binding provided by the embodiment of the present application.

22A to 22H exemplarily show schematic diagrams of scenarios where an electronic device joins a hyperterminal and performs device binding.

As shown in FIG. 22A , the electronic device 100 may display a user interface 2210 . Here, the electronic device 100 is taken as an example for illustration. Not limited to mobile phones, the electronic device 100 may also be other types of electronic devices (such as tablet computers, notebook computers, etc.). For the user interface 2210, reference may be made to the introduction of the aforementioned user interface 500 shown in FIG. 5 .

Wherein, the user interface 2210 may include a device connection status icon 2211 and an adding control 2212 . The device connection status icon 2211 may include a mobile phone icon 505 . The mobile phone icon 505 may indicate that the core device is a mobile phone, that is, the electronic device 100 . The electronic devices represented by the device icons and names surrounding the mobile phone icon 505 may be the electronic devices searched by the electronic device 100 . For example, the Sound X icon in the device connection status icon 2211, the FreeBuds icon of Zhang San, the MatePad icon of Zhang San, and the MateBook icon of Zhang San.

In some embodiments, when an account is registered on the electronic device, the name of the electronic device may be account name + type and/or model of the electronic device. When other electronic devices search for the electronic device, they can display the name of the electronic device. For example, the type of electronic device is MatePad. An account is registered on the electronic device, and the account name is "Zhang San". Then the name of the electronic device may be Zhang San's MatePad. After the electronic device 100 searches for the electronic device, it may display the icon of Zhang San's MatePad and the name of the electronic device "Zhang San's MatePad" in the device connection status diagram 2211 shown in FIG. 22A .

In some embodiments, the name of the electronic device may be noted by the user. Wherein, the user can manage the electronic device paired with the electronic device 100 through the electronic device 100 , for example, modify the name of the electronic device paired with the electronic device 100 . Exemplarily, the electronic device 100 is paired with an earphone. The user can modify the name of the headset to Zhang San's FreeBuds through the electronic device 100 . After the electronic device searches for the headset, the icon of Zhang San's FreeBuds and the name of the electronic device "Zhang San's FreeBuds" can be displayed in the device connection status diagram 2211 shown in FIG. 22A .

The embodiment of the present application does not limit the method for determining the name of the electronic device.

The add control 2212 can be used to trigger the electronic device 100 to search for other electronic devices. For example, electronic devices located around the electronic device 100, electronic devices that are connected and communicate through the Internet, and so on.

In some embodiments, in the above-mentioned device connection status diagram 2211 , the electronic devices represented by the device icons around the mobile phone icon 505 may include devices with the same account, cross-account devices bound with the electronic device 100 , and devices without accounts. That is to say, the user can directly realize the cooperative work between the electronic device represented by the device icon and the electronic device 100 by dragging the device icon around the mobile phone icon 505 close to the mobile phone icon 505 .

In addition to the electronic device represented by the device icon in the device connection state diagram 2211 , there may be other electronic devices that can work together with the electronic device 100 . The electronic device 100 may search for electronic devices whose device icons are not displayed in the device connection state illustration 2211 in response to an operation acting on the add control 2212 . For example, a cross-account device, a non-account device, etc. that have not completed device binding with the electronic device 100 .

When searching for an electronic device whose device icon is not displayed in the device connection status diagram 2211 , the electronic device 100 may display the device list 2213 in FIG. 22B on the user interface 2210 . The device identifications of the electronic devices searched by the electronic device 100 may be displayed in the device list 2213, so that the user may select an electronic device among them for device binding. Wherein, the device identification of the electronic device in the device list 2213 may include a device icon and a device name.

As shown in FIG. 22B , the device list 2213 may include a smart bracelet ID 2213B, a Vision ID 2213C, Li Si's MatePad ID 2213D, and Li Si's FreeBuds ID 2213E. The device identifier of any one of the above electronic devices may be used to trigger the electronic device 100 to perform device binding with this electronic device. A search control 2213A and a cancel control 2213F may also be included in the device list 2213 . The search control 2213A can be used to trigger the electronic device 100 to continue searching for other electronic devices. The cancel control 2213F can be used to trigger the electronic device 100 to cancel displaying the device list 2213 .

As shown in FIG. 22B , in response to the operation on Li Si's MatePad logo 2213D, the electronic device 100 may display the connection code setting box 2214 and the input keyboard 2215 shown in FIG. 22C . in:

The connection code setting box 2214 can be used to set the connection code for device binding between the electronic device 100 and the electronic device named Li Si's MatePad. A connection code setup box 2214 may contain setup prompts. The setting prompt can be, for example, a text message: to connect to Li Si's MatePad, please key in the connection code verified on the other party's device. The connection code setup box 2214 may also include a cancel control 2214A and an OK control 2214B. The cancel control 2214A can be used to cancel the device binding between the electronic device 100 and the electronic device named Si Li's MatePad. The determination control 2214B can be used to trigger the electronic device 100 to send a binding request for device binding to the electronic device named Li Si's MatePad.

The input keyboard 2215 can be used for the user to input the connection code in the connection code setting box 2214 . The input keyboard 2215 may include number input keys, letter input keys, character input keys and the like. That is, the connection code may contain one or more of the following: numbers, letters, characters, and so on. The embodiment of the present application does not limit the content of the connection code. For example, the connection code can consist of 6 digits: 710710.

As shown in FIG. 22C , if a connection code is input in the connection code setting box 2214, the electronic device 100 may send a binding request for device binding to the electronic device 200 in response to the operation on the determination control 2214B. The electronic device 200 may be the above-mentioned electronic device named Li Si's MatePad. A communication connection may be established between the electronic device 100 and the electronic device 200 .

As shown in FIG. 22D , during the process of the electronic device 100 requesting device binding with the electronic device 200 , the electronic device 100 may display a waiting prompt box 2216 . The waiting prompt box 2216 may be used to prompt the user to wait for the consent of the user to which the electronic device 200 belongs before adding the electronic device 200 to the core device as a hyper terminal of the electronic device 100 . The waiting prompt box 2216 may contain a text prompt: waiting for the other party to agree. The embodiment of the present application does not limit the content in the waiting prompt box 2216 .

When receiving the binding request sent by the electronic device 100, the electronic device 200 may display a binding prompt box 2221 shown in FIG. 22D. The binding prompt box 2221 can be used to prompt the user that the electronic device 100 requests to add the electronic device 200 to the hyperterminal, so that the electronic device 200 can provide scene-based services in the hyperterminal, and ask the user whether to agree.

The binding prompt box 2221 may include a reject control 2221A and an agree control 2221B. The rejection control 2221A can be used to reject the electronic device 200 from joining the hyper terminal whose core device is the electronic device 100 . In response to the manipulation of the rejection control 2221A, the electronic device 200 may send a rejection message to the electronic device 100 . Then, the electronic device 100 may prompt the user that the electronic device 200 refuses to join the HyperTerminal. The consent control 2221B can be used to agree that the electronic device 200 joins the core device as the hyper terminal of the electronic device 100 . In response to the operation of the consent control 2221B, the electronic device 200 may display a connection code input box 2222 shown in FIG. 22E. The connection code input box 2222 may be used to enter a connection code. When the connection code input in the connection code input box 2222 is the same as the connection code set by the user in the connection code setting box 2214 shown in FIG. 22C , the electronic device 100 and the electronic device 200 can complete the binding.

As shown in FIG. 22E, the connection code input in the connection code input box 2222 is 710710, which is the same as the connection code set in the connection code setting box 2214 shown in FIG. 22C. Then, in response to the operation of the determination control 2222A in the connection code input box 2222, the electronic device 200 may transmit a consent message to the electronic device 100. The connection code entered in the connection code input box 2222 may be included in the above-mentioned consent message. When receiving the above-mentioned consent message, the electronic device 100 may verify whether the connection code included in the consent message is correct. When it is detected that the connection code included in the consent message is correct, the electronic device 100 may display the user interface 2210 shown in FIG. 22F .

As shown in FIG. 22F , the electronic device 100 may display a MatePad icon 2211A of Li Si in the device connection state illustration 2211 of the user interface 2210 . Li Si's MatePad icon 2211A may represent an electronic device named Li Si's MatePad, that is, the electronic device 200 .

It can be seen that when the electronic device 200 agrees to join the HyperTerminal, the electronic device 100 can display the identification of the electronic device 200 in the device connection status icon 2211 , so that the user can quickly cooperate with the electronic device 100 and the electronic device 200 .

As shown in FIG. 22G , the operating electronic device 100 may display the user interface 2210 shown in FIG. 22H in response to dragging the MtaPad icon 2211A of Johnny's icon 2211A close to the phone icon 505 .

As shown in FIG. 22H , John's MtaPad icon may be displayed on user interface 2210 in the style of icon 2211B. Wherein, the MtaPad icon of Li Si can be adsorbed on the edge of the mobile phone icon 505 .

The user interface 2210 may include a prompt description 2217 "Already coordinated with Li Si's MatePad". The above prompt description 2217 may prompt the user that the electronic device 100 and the electronic device 200 are currently connected and can work together to provide the user with scene-based services.

The user interface 2210 shown in FIG. 22H may also include a recommended device area 2218A, a scene-based service option area 2218B, and an atomized service entry option 2219 . Among them, the recommended device area 2218A can be used to list the devices that can be linked with the hyperterminal formed by the currently coordinated devices (such as the hyperterminal composed of the electronic device 100 and the electronic device 200 ) to bring richer scenarios or better experience to the user. The scenario-based service option area 2218B can be used to list one or more scenario-based services recommended by the hyper terminal according to the current device combination. For example, the scenario-based services that can be provided by the mobile phone and Li Si's MatePad (i.e. tablet computer) working together include: multi-screen collaboration, camera collaboration, multi-screen conference, screen expansion, etc. The atomic service entry option 2219 can be used to display the atomic service combination, and the user can choose the atomic service provider by himself.

The electronic device 100 and the electronic device 200 work together to provide a scene-based service. For details, reference may be made to the embodiments described in FIG. 6 to FIG. 10 . I won't go into details here.

It can be known from the scenarios shown in FIGS. 22A to 22H that the hyperterminal with the electronic device 100 as the core device can join the cross-account electronic device. In this way, the user to whom the electronic device 100 belongs can not only use his own electronic device to perform collaborative work, but also use his own electronic device to perform collaborative work with other users' electronic devices, thereby enjoying more convenient and rich scene-based services. Among them, the user can request cross-account devices to complete device binding by setting a connection code, so as to establish a connection relationship that can work together. This can reduce the situation that the user who belongs to the cross-device account does not agree or know that his electronic device is used by other users, and the verification process of the above-mentioned connection code can ensure that the electronic device bound with the electronic device 100 The accuracy of the equipment improves the user experience.

FIG. 22I and FIG. 22J exemplarily show a unified entry for interaction between electronic devices in some hyperterminals.

As shown in FIG. 22I , the electronic device 100 may display a user interface 2210 . User interface 2210 may include a device connection status icon 2231 . Device icons of one or more electronic devices may be displayed in the device connection state diagram 2231 . For example, cell phone icon 505 . The mobile phone icon 505 may represent the core device of the hyper terminal, that is, the electronic device 100 . The device connection status diagram 2231 may also include device icons of electronic devices searched by the electronic device 100 .

In some embodiments, the electronic device 100 may display the device icon of the device with the same account on a circle with the mobile phone icon 505 as the center and a radius of A2. For example, Zhang San's MateBook icon, Zhang San's MatePad icon, and Zhang San's FreeBuds icon. The electronic device 100 may display the device icon of the cross-account device and the device icon of no account on a circle with the mobile phone icon 505 as the center and a radius of A3. For example, Li Si's FreeBuds icon, Li Si's MatePad icon, smart watch icon. The above distance A3 may be greater than the distance A2.

When the device icon of the electronic device is dragged to a circular area centered on the mobile phone icon 505 and the radius is A1, the user releases the finger, the icon of the electronic device can be attached to the mobile phone icon 505, and the electronic device can be connected to the mobile phone icon 505. Devices 100 cooperate. For the above process, reference may be made to the scenarios shown in FIG. 6 and FIG. 7 . The above-mentioned distance A1 may be smaller than the distance A2.

As shown in FIG. 22J , the electronic device 100 may display a user interface 2210 . The user interface 2210 may include a device connection status icon 2211 and a display area 2232 for devices with different accounts. For the above-mentioned device connection status diagram 2211, reference may be made to the introduction of the foregoing embodiments. The above non-same-account device display area 2232 may be used to display device icons of cross-account devices and device icons of non-account devices. For example, Li Si's FreeBuds icon, Li Si's MataPad icon, smart watch icon and so on. The electronic devices represented by the device icons in the device display area 2232 of the non-same account may all be searched by the electronic device 100 .

In some embodiments, the electronic device 100 can connect the two devices according to the operation of dragging the device icon in the device connection status diagram 2211 to a distance less than a preset distance from the device icon in the device display area 2232 of the non-same account. Icon snapping. Wherein, the electronic devices respectively represented by the two device icons can establish collaboration to provide scene-based services. In some embodiments, the electronic device 100 can connect the two devices according to the operation of dragging the device icon in the device display area 2232 of the non-same account to a distance less than the preset distance from the device icon in the device connection status icon 2211. Icon snapping. Wherein, the electronic devices respectively represented by the two device icons can establish collaboration to provide scene-based services.

That is to say, the user can drag the device icon in the device connection status icon 2211 close to the device icon in the device display area 2232 of the non-same account, so that the two device icons are attracted, and the electronic devices represented by the two device icons respectively establish collaboration. . The user can also drag the device icon in the device display area 2232 of the non-same account close to the device icon in the device connection status icon 2211, so that the two device icons are attracted, and the electronic devices represented by the two device icons establish collaboration.

The embodiment of the present application does not limit the expression form of the unified entry for interaction between electronic devices in the hyper terminal. The electronic device 100 may also display the core device and associated devices of the HyperTerminal in other presentation forms.

In some embodiments, the electronic devices represented by the device icons in the device connection status diagram 2211 shown in FIG. 22A may also include: cross-account devices searched by the electronic device 100 and not yet bound to the electronic device 100 . In response to the operation of coordinating the above-mentioned cross-account device whose device binding has not been completed with the electronic device 100, the electronic device 100 may perform device binding with the above-mentioned cross-account device whose device binding has not been completed. Then, the electronic device 100 and the cross-account device can work together to provide scene-based services.

The following describes another scenario in which an electronic device joins a HyperTerminal and performs device binding provided by the embodiment of the present application.

FIG. 23A to FIG. 23D exemplarily show schematic diagrams of scenarios in which an electronic device joins a hyperterminal and performs device binding.

In response to the operation of opening the device interaction interface related to the hyperterminal function (such as the touch operation on the hyperterminal function card 414 shown in FIG. 4 ), the electronic device 100 may display the user interface 2210 as shown in FIG. 23A .

The user interface 2210 may include a device connection status icon 2211 . For the device connection state diagram 2211, reference may be made to the introduction of the foregoing embodiments, and details are not repeated here. In the device connection state diagram 2211 in FIG. 23A , around the mobile phone icon 505 , there may be a MatePad icon 2311 of Li Si. Li Si's MatePad icon 2311 may represent an electronic device named Li Si's MatePad, such as the electronic device 200 shown in FIG. 22D above. The account registered on the electronic device 200 may be, for example, the account of Li Si, which is different from the account registered on the electronic device 100 . Wherein, the electronic device 100 can search for the electronic device 200 , so that Li Si's MatePad icon 2311 is displayed in the device connection state diagram 2211 . The electronic device 100 is not bound to the electronic device 200 .

The user interface 2210 may also include an add control 2212 . In response to the operation on the add control 2212, the electronic device 100 can search for more electronic devices for the user to choose to join the HyperTerminal.

As shown in FIG. 23B , in response to the operation of coordinating the electronic device 200 with the electronic device 100 , for example, the operation of dragging Lisi's MtaPad icon 2311 close to the mobile phone icon 505 , the electronic device 100 can perform device binding with the electronic device 200 . Certainly.

In some embodiments, in response to the above operation of coordinating the electronic device 200 with the electronic device 100, the electronic device 100 may first determine whether the electronic device 200 is a cross-account device. When it is determined that the electronic device 200 is a cross-account device, the electronic device 100 may determine whether the electronic device 200 has been bound with the electronic device 100 . If it is determined that the electronic device 200 is not bound to the electronic device 100 , the electronic device 100 may be bound to the electronic device 200 .

In some embodiments, when the above-mentioned Li Si's MtaPad icon 2311 is dragged close to the mobile phone icon 505, the electronic device 100 may display Li Si's MtaPad icon in the style of the icon 2312 shown in FIG. 23B. The icon 2312 is different from the MtaPad icon 2311 of Li Si, so that it is convenient for the user to distinguish which electronic devices have established connections with the electronic device 100 that can work together.

As shown in FIG. 23C , during the device binding process between the electronic device 100 and the electronic device 200 , the electronic device 100 may display a connection code setting box 2313 and an input keyboard 2314 for the user to input a device binding connection with the electronic device 100 code. For an implementation of device binding between the electronic device 100 and the electronic device 200, reference may be made to the embodiments described in FIGS. 22C to 22E. I won't go into details here.

When the electronic device 100 and the electronic device 200 complete the device binding, the electronic device 100 may display the user interface 2210 shown in FIG. 23D . The user interface shown in FIG. 23D may refer to the aforementioned user interface shown in FIG. 22H. I won't go into details here.

In some embodiments, the electronic device 100 may display the device icon of the searched electronic device in the device connection status diagram 2211 according to a preset rule. For example, the number of device icons displayed in the device connection status icon 2211 is limited. The electronic device 100 may be sorted according to the following priorities: same-account device>non-account device>cross-account device, select the electronic device with higher priority to display the device icon in the device connection status icon 2211 . For another example, the number of device icons displayed in the device connection status icon 2211 is limited. The electronic device 100 may select an electronic device that is close to the electronic device 100 to display a device icon in the device connection status diagram 2211 according to the distance between the searched electronic device and the electronic device 100 . For another example, the number of device icons displayed in the device connection status icon 2211 is limited. The electronic device 100 may select an electronic device with a high signal strength to display a device icon in the device connection status diagram 2211 according to the level of the signal strength corresponding to the searched electronic device. The embodiment of the present application does not limit the above preset rules for displaying device icons in the device connection status icon 2211 .

It can be seen from the scenarios shown in FIGS. 23A to 23D that the electronic device 100 can identify the searched electronic devices (such as account devices, bound cross-account devices, unbound cross-account devices, and no-account devices). Displayed around the core device icon of HyperTerminal. In this way, the user can quickly establish a connection relationship between different electronic devices for collaborative work by dragging and dropping the device icon. When the electronic device represented by the dragged device icon includes an unbound cross-account device, the electronic device 100 may establish a binding relationship with the above-mentioned unbound electronic device, so as to establish a cooperative working connection relationship. Through the above method, the user to which the electronic device 100 belongs can not only use his own electronic device to perform collaborative work, but also use his own electronic device to perform collaborative work with other users' electronic devices, thereby enjoying more convenient and rich scene-based services.

In some embodiments, when an electronic device (such as a cross-account device) that has been bound to the core device of the hyperterminal joins the hyperterminal again, the process of binding the core device to the hyperterminal can be simplified. For example, electronic device 200 and electronic device 100 have performed device binding. When adding the core device as the hyperterminal of electronic device 100 again, electronic device 100 and electronic device 200 can mutually verify each other's identities and complete the device binding without requiring the user to Enter the connection code for device binding again. The above method can simplify the user operation of adding the electronic device to the hyperterminal again, facilitate the user to quickly use the scene-based service provided by the hyperterminal, and improve the user experience.

FIG. 24A to FIG. 24D exemplarily show schematic diagrams of scenarios in which an electronic device joins a hyperterminal and performs device binding.

As described in FIG. 24A , the electronic device 100 may display a user interface 2210 . For the user interface 2210, reference may be made to the introduction of the foregoing embodiment described in FIG. 22A.

In FIG. 24A , user interface 2210 may include device connection status illustration 2211 . Surrounding the mobile phone icon 505 in the device connection status icon 2211 , there may be a MatePad icon 2411 of Li Si and a FreeBuds icon 2412 of Li Si. Wherein, Si Li's MatePad icon 2411 may represent an electronic device named Si Li's MatePad, such as the electronic device 200 shown in FIG. 22D . The login account on the electronic device 200 may be Li Si's account, which is different from the login account on the electronic device 100 represented by the mobile phone icon 505 . Johnny's FreeBuds icon 2412 may represent an electronic device named Johnny's FreeBuds. The electronic device named Li Si's FreeBuds may be an electronic device associated with Li Si's account.

It can be seen that the above-mentioned MatePad electronic device named as Li Si and the electronic device named as Li Si's FreeBuds can both be cross-account devices whose core device is the hyper terminal of the electronic device 100 .

In some embodiments, the device icons of cross-account devices that have been device-bound with the electronic device 100 and the device icons of cross-account devices that have not been device-bound with the electronic device 100 may have different representation forms. For example, an electronic device named as Li Si's MatePad (hereinafter referred to as Li Si's MatePad) was once bound to the electronic device 100 . The electronic device named Li Si's FreeBuds (hereinafter referred to as Li Si's FreeBuds) has not been bound to the electronic device 100 . Lisi's MatePad icon 2411 may be displayed in a dark filled form as shown in FIG. 24A . John's FreeBuds icon 2412 may be displayed in a light filled form as shown in FIG. 24A . The different presentation forms of the above-mentioned device icons can facilitate the user to distinguish which cross-account devices have been bound with the electronic device 100 and which cross-account devices have not been bound with the electronic device 100 .

In some embodiments, the cross-account device joins the hyperterminal of the electronic device 100 as the core device for the first time, and the cross-account device and the electronic device 100 can perform device binding as shown in FIGS. 22C to 22E . During the above-mentioned device binding process, the cross-account device and the electronic device 100 may store each other's device identifiers. When the above-mentioned cross-account device exits the HyperTerminal and joins the HyperTerminal again, the electronic device 100 and the above-mentioned cross-account device can use the above-mentioned device identification to verify each other's identities and complete device binding. That is to say, when the user adds a cross-account device that has joined the HyperTerminal to the HyperTerminal again, there is no need to set the connection code for device binding again.

Exemplarily, Li Si's MatePad (that is, the electronic device 200 ) has been bound with the electronic device 100 once. In response to the operation of dragging Li Si's MatePad icon 2411 close to the mobile phone icon 505 as shown in FIG. 24B (that is, the operation of adding the electronic device 200 to the core device as the hyperterminal of the electronic device 100 again), the electronic device 100 can send a message to the electronic device. 100 Send a bind request requesting a bind. The binding request may include a device identity for authentication.

In the process of the electronic device 100 requesting the electronic device 200 to complete the device binding, the electronic device 100 may display the user interface shown in FIG. 24C to remind the user that cooperation with the electronic device 200 needs to wait for the consent of the user to which the electronic device 200 belongs.

When receiving the binding request from the electronic device 100, the electronic device 200 can determine that it has completed device binding with the electronic device 200 through the connection code according to the device identification in the binding request. The electronic device 200 may also display the user interface 2220 shown in FIG. 24C to ask the user whether to agree to the user to which the electronic device 100 belongs to use the electronic device 200 for collaborative work. For the user interface 2220 shown in FIG. 24C, reference may be made to the introduction of the user interface 2220 shown in FIG. 22D. I won't go into details here.

When it is detected that the user agrees to the request of the electronic device 200 to complete the binding with the electronic device 100 , the electronic device 200 may send a consent message to the electronic device 100 . When receiving the above-mentioned consent message, the electronic device 100 can add the electronic device 200 to the HyperTerminal, so as to establish a cooperative connection with the electronic device 200 and provide the user with scenario-based services. After the electronic device 200 joins the HyperTerminal, the electronic device 100 can display the user interface 2210 shown in FIG. 24D for the user to control and manage the scenario-based services provided by the cooperative work between the electronic device 100 and the electronic device 200 . The user interface 2210 shown in FIG. 24D may refer to the user interface 2210 described above in FIG. 22H . I won't go into details here.

It can be understood that the user to which the electronic device 100 belongs may be different from the user to which the electronic device 200 belongs. Although the electronic device 200 once joined the hyperterminal whose core device is the electronic device 100 , if the user of the electronic device 100 wants to use the electronic device 200 for collaborative work again, the consent of the user who belongs to the electronic device 200 should still be obtained. This can reduce the situation that the user who belongs to the cross-device account has his own electronic device used by other users without consent or knowledge.

It can be known from the scenarios shown in FIG. 24A to FIG. 24D that, in the case that the electronic device has joined the HyperTerminal, the user operation of adding the electronic device to the HyperTerminal again for device binding can be simpler. This can improve user experience in using the HyperTerminal.

In some embodiments, the cross-account device added to the HyperTerminal is paired with other electronic devices, thereby associating the account logged in on the paired electronic device. In the process of the above-mentioned cross-account device joining the hyper terminal whose core device is the electronic device 100, the electronic device 100 can complete the device binding between the above-mentioned cross-account device and the electronic device 100 through the electronic device paired with the above-mentioned cross-account device.

FIG. 25A to FIG. 25F exemplarily show schematic diagrams of scenarios in which an electronic device joins a hyperterminal and performs device binding.

As shown in FIG. 25A , the electronic device 100 may display a user interface 2210 . The user interface 2210 may include a device connection status icon 2211 . In the device connection status diagram 2211 , the device icons displayed around the mobile phone icon 505 include Li Si's FreeBuds icon 2511 . Li Si's FreeBuds icon 2511 may represent an electronic device named Li Si's FreeBuds (hereinafter referred to as Li Si's FreeBuds). Among them, Li Si's FreeBuds can be paired with an electronic device whose login account is Li Si's account, such as Li Si's mobile phone. The above-mentioned account number of Li Si is different from the account number registered on the electronic device 100 . That is, Li Si's FreeBuds can be used as a cross-account device to join the hyperterminal whose core device is the electronic device 100 .

For other content in the user interface 2210, reference may be made to the introduction of the foregoing embodiments. I won't go into details here.

In response to the operation of dragging Li Si's FreeBuds icon 2511 close to the mobile phone icon 505, the electronic device 100 can perform device binding with Li Si's FreeBuds. Wherein, the electronic device 100 may display a connection code setting box 2513 and an input keyboard 2514 shown in FIG. 25B for the user to set a connection code for device binding. For the user interface shown in FIG. 25B above, reference may be made to the introduction of the user interface shown in FIG. 22C above. I won't go into details here.

When detecting that the connection code setting operation has been completed in the connection code setting box 2513, the electronic device 100 may send a binding request for device binding to Li Si's FreeBuds, and display the user interface 2210 shown in FIG. 25C .

In some embodiments, Johnny's FreeBuds are paired with electronic device 300 . The aforementioned electronic device 300 may be, for example, Li Si's mobile phone. The account registered on the electronic device 300 may be Li Si's account. When receiving the above-mentioned binding request from the electronic device 100 , Li Si's FreeBuds can send the binding request to the electronic device 300 . Then, the electronic device 300 may display the user interface 2520 shown in FIG. 25D.

As shown in FIG. 25D , the user interface 2520 may include a binding prompt box 2521 . The binding prompt box 2521 may include a reject control 2521A and an agree control 2521B. For the binding prompt box, refer to the aforementioned binding prompt box 2221 shown in FIG. 22D . In response to the operation on the consent control 2521B, the electronic device 300 may display the connection code input box 2522 shown in FIG. 25E for the user to input the connection code for device binding between Li Si's FreeBuds and the electronic device 100 . The connection code entry box 2522 may include a determination control 2522A. For the connection code input box 2522, refer to the aforementioned connection code input box 2222 shown in FIG. 22E.

As shown in FIG. 25E, the connection code input in the connection code input box 2522 is 710710, which is the same as the connection code set in the connection code setting box 2513 shown in FIG. 25B. Then, in response to the operation on the determination control 2522A, the electronic device 300 may send a consent message to the electronic device 100 through Li Si's FreeBuds. The connection code entered in the connection code input box 2522 may be included in the above-mentioned consent message. When receiving the above-mentioned consent message, the electronic device 100 may verify whether the connection code included in the consent message is correct. When it is detected that the connection code included in the consent message is correct, the electronic device 100 may display the user interface 2210 shown in FIG. 25F . In the user interface shown in FIG. 25F , Li Si's FreeBuds icon 2512 can be adsorbed on the mobile phone icon 505 . The Li Si's FreeBuds icon 2512 may be different from the Li Si's FreeBuds icon 2511 shown in FIG. 25A , so that the user can easily distinguish whether Li Si's FreeBuds has been coordinated with the electronic device 100 .

The user interface 2210 shown in FIG. 25F may also include a prompt description 2531 "cooperated with Li Si's FreeBuds". The above prompt description 2531 can prompt the user that the current electronic device 100 and Li Si's FreeBuds have established a communication connection that can work together, so as to provide the user with scene-based services.

The user interface 2210 shown in FIG. 25F may also include a scene-based service option area 2532 and an atomized service entry option 2533 . Among them, the scenario-based service option area 2532 can be used to list one or more scenario-based services recommended by the hyperterminal composed of currently coordinated devices (such as the hyperterminal composed of the electronic device 100 and Li Si's FreeBuds) according to the current device combination . For example, the scenario-based services that the mobile phone and Li Si's FreeBuds can work together include: audio sharing and audio playback. The above-mentioned audio sharing can be used to share one or more channels of audio played by the electronic device 100 to Li Si's FreeBuds. Li Si's FreeBuds can play the audio shared by the electronic device 100 . Wherein, the electronic device 100 can continue to play the shared audio while sharing the audio. The above audio delivery can be used to deliver one or more channels of audio played by the electronic device to Li Si's FreeBuds. Li Si's FreeBuds can play the audio delivered by the electronic device 100 . Wherein, after the electronic device 100 puts the audio onto Li Si's FreeBuds, it may no longer play the above-mentioned put audio.

It can be known from the scenarios shown in Figure 25A to Figure 25F above that when the electronic device 100 is bound to a cross-account device, the above-mentioned device binding can be completed through the electronic device paired with the cross-account device, so that the cross-account device can join the core device It is the hyper terminal of the electronic device 100 . The electronic device paired with the above-mentioned cross-account device can better remind the user that the cross-account device belongs to that there are other users who want to use the cross-account device for collaborative work. The electronic device paired with the above-mentioned cross-account device can facilitate the user to confirm whether the cross-account device agrees to join the hyperterminal.

In some embodiments, the electronic device paired with the cross-account device is also paired with other electronic devices. The above-mentioned other electronic devices may also display a message inviting the above-mentioned cross-account device to join the HyperTerminal for device binding.

Exemplarily, it can be known from the scenarios shown in FIGS. 25A to 25C that the electronic device 100 may send a binding request for device binding to the FreeBuds of the cross-account device Li Si. As shown in FIG. 25G , Li Si's FreeBuds are bound to the electronic device 300 . The above-mentioned electronic device 300 may be Li Si's mobile phone. The electronic device paired with the electronic device 300 may include the smart watch shown in FIG. 25G in addition to Li Si's FreeBuds mentioned above. The smart watch may be Li Si's smart watch. Li Si's FreeBuds can send the binding request from the electronic device to the electronic device 300 . When receiving the binding request, the electronic device 300 may display the aforementioned binding prompt box 2521 shown in FIG. 25D . In addition, the electronic device 300 may also send the binding request to the smart watch paired with itself. The smart watch may display user interface 2520 shown in Figure 25G. The user interface 2520 can be used to prompt the user that Zhang San's mobile phone (that is, the electronic device 100) requests to use Li Si's FreeBuds so that Li Si's FreeBuds can provide scene-based services in the hyperterminal, and ask the user whether to agree.

The user interface 2520 may include a reject control 2521 and an agree control 2522 . The rejection control can be used to reject Li Si's FreeBuds from joining the hyperterminal whose core device is the electronic device 100 . The consent control 2522 can be used to agree that Li Si's FreeBuds joins the core device as the hyper terminal of the electronic device 100 . In response to manipulation of consent control 2522, smart watch may display user interface 2530 shown in Figure 25H. The user interface 2530 may be used to input a connection code for completing device binding with the electronic device 100 . User interface 2530 may include an OK control 2531 .

As shown in FIG. 25H , when a connection code (such as 710710) is input on the user interface 2530, in response to the operation on the confirmation control 2531, the smart watch can send a consent message to Li Si's FreeBuds. Wherein, the above-mentioned consent message may be sent to Li Si's FreeBuds through the electronic device 300 . Then, Li Si's FreeBuds can send the consent message to the electronic device 100 . The electronic device 100 and Li Si's FreeBuds can complete device binding.

It can be seen from the above embodiment that the user can use the above electronic device 300 or the above smart watch to complete the device binding between the electronic device 100 and Li Si's FreeBuds.

FIG. 26A to FIG. 26C exemplarily show schematic diagrams of scenarios in which an electronic device joins a hyperterminal and performs device binding.

As shown in FIG. 26A , the electronic device 100 may display a user interface 2210 . For the user interface 2210, reference may be made to the introduction of the aforementioned user interface 2210 shown in FIG. 26A. I won't go into details here. In response to the operation of dragging Li Si's FreeBuds icon 2511 close to the mobile phone icon 505, the electronic device 100 can perform device binding with Li Si's FreeBuds.

In some embodiments, during the device binding process between the electronic device 100 and Li Si's FreeBuds, the electronic device 100 may display the binding prompt box 2611 shown in FIG. 26B , and send a device binding message to Li Si's FreeBuds. The binding request.

As shown in FIG. 26B , the binding prompt box 2611 can be used to prompt the user to perform operations on Li Si's FreeBuds to complete device binding with Li Si's FreeBuds. For example, the binding prompt box 2611 may contain a text prompt: It has been requested to establish a cooperative connection with Li Si's FreeBuds, and the establishment of the cooperative connection can be completed after the other party agrees, and the other party agrees to press and hold the earphone handle of the left earphone/right earphone. The embodiment of the present application does not limit the prompt content in the binding prompt box 2611.

As shown in FIG. 26B , in response to a long press operation on the earphone handle of Li Si's FreeBuds (such as the earphone handle of the right earphone), Li Si's FreeBuds can send a consent message to the electronic device 100 . When receiving the above-mentioned consent message, the electronic device 100 may display the user interface 2210 shown in FIG. 26C . For the user interface 2210 shown in FIG. 26C, reference may be made to the introduction of the user interface 2210 shown in FIG. 25F. I won't go into details here.

In some embodiments, when receiving the binding request sent by the above-mentioned electronic device 100, Li Si's FreeBuds can prompt the user electronic device 100 to request device binding with Li Si's FreeBuds through voice playback, light flashing, etc. And the user operations required to complete the device binding.

In some embodiments, the device binding process shown in FIG. 26A to FIG. 26C may be a process in which Li Si's FreeBuds joins the HyperTerminal again and binds with the electronic device 100 . It can be seen that the device binding process shown in FIGS. 26A to 26C is simpler than the device binding process shown in FIGS. 25A to 25F . When the electronic device 100 and Li Si's FreeBuds are bound for the first time, the device binding can be performed according to the process shown in Figure 25A-25F, so that the electronic device 100 and Li Si's FreeBuds can more accurately verify each other's identities through the connection code , reducing the possibility of device binding errors. In the case of device binding once, Li Si's FreeBuds joins the HyperTerminal again, and when performing device binding with the electronic device 100 again, the device binding can be performed according to the process shown in Fig. 26A to Fig. 26C. This can simplify the user operation of device binding when the user adds the cross-account device to the HyperTerminal again, and improve the user experience.

It can be seen from the foregoing embodiments that the device connection status diagram 2211 shown in FIG. 22A can provide a unified entry for interaction between multiple electronic devices, which is convenient for users to use different electronic devices for collaborative work. In some embodiments, the user may also remove one or more device icons in the device connection status diagram 2211 according to his needs.

The following introduces a scenario of stopping the cooperative work between electronic devices and removing the electronic devices from the HyperTerminal provided by the embodiment of the present application.

FIG. 27A to FIG. 27E exemplarily show a schematic diagram of a scene where the cooperation among electronic devices is stopped and the electronic device is removed from the HyperTerminal.

As shown in FIG. 27A , the electronic device 100 may display a user interface 2210 . For the user interface 2210, reference may be made to the introduction shown in FIG. 25F. I won't go into details here. Among them, Li Si's FreeBuds icon 2512 is attached to the mobile phone icon 505, which may indicate that Li Si's FreeBuds and the electronic device 100 have established a connection relationship capable of working together. In response to an operation on Li Si's FreeBuds icon 2512 , such as a long press operation, the electronic device 100 can move the position of Li Si's FreeBuds icon 2512 according to the continuous operation on Li Si's FreeBuds icon 2512 .

As shown in FIG. 27B , in response to the operation of long pressing and dragging Li Si's FreeBuds icon 2512 , the electronic device 100 may display Li Si's FreeBuds icon 2512 according to the position indicated by the above operation. The electronic device 100 may also display a delete control 2711 on the user interface 2210 . The delete control 2711 can be used to remove the device icon dragged to the location of the delete control 2711 from the device connection status icon 2211 .

In some embodiments, in response to the operation of dragging and dropping Lisi's FreeBuds icon 2512 to the position shown in FIG. 27B and releasing Lisi's FreeBuds icon 2512 , the electronic device 100 may display the user interface 2210 shown in FIG. 27B . The aforementioned release of Li Si's FreeBuds icon 2512 may indicate that the finger operating on Li Si's FreeBuds icon 2512 leaves the screen of the electronic device 100 .

As shown in FIG. 27C , the device icon of Si Li's FreeBuds can change from Si Li's FreeBuds icon 2512 to Si Li's FreeBuds icon 2511 . Among them, Li Si's FreeBuds icon 2512 can be attached to the mobile phone icon 505, which can indicate that Li Si's FreeBuds and the electronic device 100 have been coordinated to provide users with scene-based services. Li Si's FreeBuds icon 2511 is not attached to the mobile phone icon 505, which may indicate that Li Si's FreeBuds has not been coordinated with the electronic device 100 yet.

Exemplarily, Li Si's FreeBuds has cooperated with the electronic device 100 . Wherein, the electronic device 100 shares the audio to Li Si's FreeBuds. Li Si's FreeBuds can play the audio shared by the electronic device 100 . In response to the operation of dragging the Doe's FreeBuds icon 2512 away from the phone icon 505, the electronic device 100 may stop coordinating with Doe's FreeBuds. Wherein, the electronic device 100 can stop sharing the audio to Li Si's FreeBuds.

That is to say, when Li Si's FreeBuds and the electronic device 100 have been coordinated, the user can stop the cooperative work between the electronic devices by moving the Li Si's FreeBuds icon away from the mobile phone icon 505 .

As shown in FIG. 27D , in response to the operation of dragging Li Si's FreeBuds icon 2512 to the location of the deletion control 2711 , the electronic device 100 may display the user interface 2210 shown in FIG. 27E . Wherein, the electronic device 100 may remove the device icon of Li Si's FreeBuds from the device connection status icon 2211 . It is also in time that users can remove electronic devices incorporated into HyperTerminal from HyperTerminal.

It can be known from the scenarios shown in FIGS. 27A to 27B that the user operations of stopping the cooperation between the electronic devices and removing the electronic device from the HyperTerminal are relatively simple. Users can quickly manage electronic devices in the HyperTerminal and control the collaborative work between electronic devices.

In some embodiments, when the cross-account device joins the above-mentioned hyper terminal whose core device is the electronic device 100, the user to which the cross-account device belongs can stop the cross-account device from cooperating with other electronic devices in the hyper terminal, and remove the cross-account device from the hyper terminal. removed from the terminal.

The following describes another scenario of stopping the collaboration between electronic devices and removing the electronic devices from the HyperTerminal provided by the embodiment of the present application.

FIG. 28A-FIG. 28E exemplarily show a schematic diagram of a scene where the cooperation among electronic devices is stopped and the electronic device is removed from the HyperTerminal.

As shown in FIG. 28A , the electronic device 100 may display a user interface 2210 . For the user interface 2210, reference may be made to the introduction of the aforementioned user interface shown in FIG. 25F. Among them, Li Si's FreeBuds icon 2512 is attached to the mobile phone icon 505, which may indicate that Li Si's FreeBuds and the electronic device 100 have been coordinated. Li Si's FreeBuds is a cross-account device in the hyper terminal whose core device is the electronic device 100 .

As shown in FIG. 28B , the electronic device 300 may display a user interface 2810 . The above-mentioned electronic device 300 may be an electronic device paired with the above-mentioned Li Si's FreeBuds. For example, electronic device 300 may be a mobile phone. The account of Li Si can be registered in the electronic device 300 . The user interface 2810 may be a device interaction interface related to the hyperterminal function of the electronic device 300 as the core device.

User interface 2810 may include device connection status icon 2811 . The device connection status diagram 2811 may be presented as a donut diagram. A mobile phone icon 2801 may be displayed in the center of the device connection status icon 2811 . The mobile phone icon 2801 may indicate that the core device is a mobile phone, that is, the electronic device 300 . The mobile phone icon 2801 may be surrounded by device identifications of electronic devices searched by the electronic device 300 . The device identification of the electronic device may include a device icon and a device name. For example, surrounding the mobile phone icon 2801 may include: Si Li's FreeBuds icon 2811A, a Vision icon, a smart watch icon, Si Li's MateBook icon, Si Li's MatePad icon, and the like. The above-mentioned Li Si's FreeBuds icon 2811A may represent Li Si's FreeBuds.

As shown in FIG. 28C , in response to the operation of dragging Li Si's FreeBuds icon 2811A close to the mobile phone icon 2801 , the electronic device 100 may display the user interface 2810 shown in FIG. 28D . Among them, Li Si's FreeBuds can establish a connection relationship capable of cooperating with the electronic device 300, and stop the cooperating work with the electronic device 100 shown in FIG. 28A.

As shown in FIG. 28D , when attached to the mobile phone icon 2801 , the device icon of Li Si's FreeBuds can display the expression form shown in FIG. 2811B of Li Si's FreeBuds icon 2811B. The adsorption between Li Si's FreeBuds icon 2811B and the mobile phone icon 2801 may indicate that Li Si's FreeBuds and the electronic device 300 have been coordinated.

When Li Si's FreeBuds stops cooperating with the electronic device 100, the electronic device 100 may display the user interface 2210 shown in FIG. 28E. In FIG. 28E , Li Si's FreeBuds icon 2511 is not attached to the mobile phone icon 505 . That is, the collaborative work between the electronic device 100 and Li Si's FreeBuds has stopped. Comparing Fig. 28A and Fig. 28E, it can be seen that after the process of coordinating Lisi's FreeBuds with electronic device 300 as shown in Fig. 28B-Fig. terminal.

In some embodiments, when Li Si's FreeBuds stops cooperating with the electronic device 100, the electronic device 100 may also display a prompt message to remind the user that Li Si's FreeBuds has stopped cooperating.

Exemplarily, Li Si's FreeBuds cooperates with the electronic device 100 , and Li Si's FreeBuds can play audio shared by the electronic device 100 . During the process of coordinating the icon of Li Si's FreeBuds with the electronic device 100 , Li Si's FreeBuds receives the execution of coordinating with the electronic device 300 that is paired with it. Li Si's FreeBuds can stop cooperating with the electronic device 100 and instead cooperate with the electronic device 300 . Among them, Li Si's FreeBuds can stop playing the audio shared by the electronic device 100 and play the audio delivered by the electronic device 300 .

The embodiment of the present application does not limit the above user operation for stopping the coordination between the electronic device and other electronic devices in the hyper terminal. For example, in response to an operation (such as a long press operation) on the icon 2811 of Li Si's FreeBuds shown in FIG. 28B , the electronic device 300 may display a control for stopping the cooperation between Li Si's FreeBuds and the electronic device 100 . In this way, the user to whom Li Si's FreeBuds belongs can stop other users from using their own headphones for collaborative work.

It can be seen from the scenarios shown in Fig. 28A to Fig. 28E above that when the user's electronic device is added to the HyperTerminal by other users to provide collaborative services, the user can stop his electronic device from being used by other users at any time according to his needs.

In some embodiments, the user can actively add his own electronic device to other users' hyperterminals as a cross-account device. In this way, users can use their own electronic devices to cooperate with other users' electronic devices. For example, Li Si can request to add his earphone as a cross-account device to the HyperTerminal of Zhang San's mobile phone as the core device. When joining the hyper terminal, Li Si's earphone can cooperate with Zhang San's mobile phone. Zhang San's mobile phone can share the audio to Li Si's earphone. In this way, Li Si can listen to the audio shared from Zhang San's mobile phone through his earphone.

Exemplarily, as shown in FIG. 28F , the electronic device 300 may display a user interface 2810 . The user interface 2810 may include a device connection status icon 2811 and a display area 2821 for devices with different accounts. Wherein, a mobile phone icon 2801 may be displayed in the device connection status icon 2811 . The mobile phone icon 2801 may represent the core device of the hyper terminal, that is, the electronic device 300 . Here, the account for logging in to Li Si in the electronic device 300 is taken as an example for illustration. That is, the electronic device 300 may be Li Si's mobile phone. The device connection status diagram 2811 may also include device icons of devices with the same account that are searched by the electronic device 300 . For example, Li Si's MateBook icon, Li Si's MatePad icon, Li Si's FreeBuds icon 2811A, Vision icon, smart watch icon. The non-same-account device display area 2821 can be used to display device icons of cross-account devices and device icons of non-account devices searched by one or more electronic devices 300 . For example, Zhang San's MateBook icon, Zhang San's MatePad icon, and Zhang San's P50 icon 2822 . The above-mentioned Zhang San's P50 icon 2822 may represent an electronic device named Zhang San's P50, and the electronic device may be the aforementioned electronic device 100 . That is, the electronic device 100 may be Zhang San's mobile phone.

In response to the operation of dragging Li Si's FreeBuds icon 2811A close to Zhang San's P50 icon 2822 shown in FIG. 28F , the electronic device 300 can instruct Li Si's FreeBuds to perform device binding with the electronic device 100 . Understandably, the user to which Li Si's FreeBuds belongs (such as Li Si) is different from the user to which the electronic device 100 belongs (such as Zhang San). Then, the electronic device 300 instructs Li Si's FreeBuds to cooperate with the electronic device 100, and needs to ask the user to whom the electronic device 100 belongs whether to agree. That is to say, before Li Si's FreeBuds and the electronic device 100 establish cooperation, the device can be bound first.

During the device binding process between Li Si's FreeBuds and the electronic device 100 , the electronic device 300 may display the connection code setting box 2831 and the input keyboard 2832 shown in FIG. 28G . The connection code setting box 2831 can be used to set the connection code for device binding between Li Si's FreeBuds and the electronic device 100 . The input keyboard 2832 can be used for the user to input the connection code in the connection code setting box 2831 .

As shown in FIG. 28G , when a connection code is set in the connection code setting box 2831 , the electronic device 300 may send a binding request to the electronic device 100 in response to an operation acting on a control determined in the connection code setting box 2831 . The binding request can be used to instruct Li Si's FreeBuds to request cooperation with the electronic device 100 . In some embodiments, the above binding request may be sent to the electronic device 100 through Li Si's FreeBuds.

When receiving the above-mentioned binding request, the electronic device 100 may display the binding prompt box 2833 shown in FIG. 28H . The binding prompt box 2833 may refer to the aforementioned binding prompt box 2221 shown in FIG. 22D . In response to the operation on the consent control in the binding prompt box 2833, the electronic device 100 may display the connection code input box 2834 shown in FIG. 28I. The connection code input box 2834 may refer to the connection code input box 2222 shown in FIG. 22E.

As shown in FIG. 28I , when a connection code is input in the connection code input box 2834 , the electronic device 100 may send a consent message to the electronic device 300 in response to the operation of the control determined in the connection code input box 2834 . The consent message may include the connection code (such as 710710) entered in the connection code input box 2834 . When receiving the consent message, the electronic device 300 may verify whether the connection code in the consent message is the same as the connection code set in the connection code setting box 2831 shown in FIG. 28G . If they are the same, the electronic device 300 may instruct Li Si's FreeBuds to cooperate with the electronic device.

When Li Si's FreeBuds cooperate with the electronic device 100, the electronic device 100 can display the user interface 2210 shown in Figure 28J, and the electronic device 300 can display the user interface 2810 shown in Figure 28K. In FIG. 28J , Li Si's FreeBuds icon is attached to the mobile phone icon 505 . In FIG. 28K , Li Si's FreeBuds icon 2823 is attached to Zhang San's P50 icon 2822 .

In some embodiments, the electronic device 100 can respond to the action of acting on the icon of Li Si's FreeBuds shown in FIG. 28J , dragging the icon of Li Si's FreeBuds to separate from the mobile phone icon 505, and stop the electronic device 100 and Li Si's FreeBuds synergy.

In some embodiments, the electronic device 300 may respond to the operation of dragging Li Si's FreeBuds icon to separate from Zhang San's P50 icon 2822 acting on Li Si's FreeBuds icon shown in FIG. Li Si's FreeBuds collaboration.

It can be seen from the above-mentioned embodiments that the cross-account device joins the hyperterminal, either by the core device of the hyperterminal inviting the cross-account device to join, or by applying for cross-account device to join. When the core device of the hyperterminal invites the cross-account device to join, the core device may ask the cross-account device whether to agree to join the hyperterminal. In the case of a cross-account device applying to join, the cross-account device can ask the core device whether to allow the cross-account device to join the HyperTerminal.

It should be noted that, in some embodiments, one electronic device can work cooperatively with multiple electronic devices at the same time. For example, the aforementioned FreeBuds of Li Si can also cooperate with the electronic device 300 while cooperating with the electronic device 100 . Among them, Li Si's FreeBuds can include two earphones (left earphone and right earphone). One earphone in Li Si's FreeBuds can play audio from the electronic device 100 . Meanwhile, another earphone in Li Si's FreeBuds can play audio from the electronic device 300 . For another example, a printer can be added to different hyperterminals at the same time. This one printer can receive printing requests from multiple electronic devices, and provide corresponding printing services for the multiple electronic devices.

In some embodiments, in the case that the coordinated electronic devices in the hyperterminal include cross-account devices, the cross-account devices may stop coordinating with other electronic devices in the hyperterminal at any time in response to user operations. For example, the coordinated electronic devices in the hyperterminal with the above-mentioned electronic device 100 as the core device include the above-mentioned FreeBuds of Li Si. In the process of cooperating with the electronic device 100 in the hyperterminal, Li Si's FreeBuds can play audio from the electronic device 100 . Li Si's FreeBuds can stop playing audio in response to an operation acting on the Freebuds to stop playing. Li Si's FreeBuds can also receive a stop coordination instruction sent by an electronic device (such as Li Si's mobile phone) that is paired with him, and exit the above-mentioned hyper terminal according to the stop coordination instruction, and stop coordinating with the electronic device 100 . That is to say, when Li Si's FreeBuds joins the HyperTerminal as a cross-account device and cooperates with other electronic devices in the HyperTerminal, the user to which Li Si's FreeBuds belongs can take back Li Si's FreeBuds at any time and stop other users from using their own. Electronic equipment.

In some embodiments, a device without an account may only join one HyperTerminal at a time, and cooperate with one electronic device. In the case that the unaccounted device has joined the hyperterminal, if it receives a collaboration request to join other hyperterminals for collaboration again, the unaccounted device may send a waiting message to the device that sent the aforementioned collaboration request. The waiting message can be used to remind the user that the device without an account has been occupied and can only be used after the cooperation of other users is completed.

In some embodiments, the electronic device 100 can record the electronic device removed from the HyperTerminal in the device recycle bin. In this way, the user can view the electronic device that he has removed from the HyperTerminal in the device recycle bin, and can add the removed electronic device to the HyperTerminal again.

The following introduces a scenario of managing a hyperterminal provided by the embodiment of the present application.

FIG. 29A and FIG. 29B exemplarily show a schematic diagram of a scene for managing a hyperterminal.

As shown in FIG. 29A , the electronic device 100 may display a user interface 2210 . The user interface 2210 may include a device recycle bin control 2901 . For other content contained in the user interface 2210, refer to the introduction of the user interface shown in FIG. 22A.

In response to the operation of the device recycle bin control 2901, the electronic device 100 may display a recycle bin list 2902 shown in FIG. 29B. The recycle bin list 2902 can be used to display icons of devices removed from the device connection status diagram 2211 , so that the user can check which electronic devices he has removed.

Exemplarily, the electronic device 100 may remove Li Si's FreeBuds icon from the device connection status icon 2211 according to the aforementioned operations shown in FIG. 27D . Then, the electronic device 100 may display Li Si's FreeBuds icon in the above-mentioned recycle bin list 2902 .

As shown in FIG. 29B , the recycle bin list 2902 may include an icon of John's FreeBuds 2902A, a restore control 2902B. Among them, Li Si's FreeBuds icon 2902A may represent Li Si's FreeBuds. The restore control 2902B can be used to trigger the electronic device 100 to add Lisi's FreeBuds to the core device as the hyperterminal of the electronic device 100 . For example, in response to an operation (such as a touch operation) on the recovery control 2902B, the electronic device 100 may display an icon of Li Si's FreeBuds in the device connection status illustration 2211 . The recycle bin list 2902 may also include device icons of other electronic devices and corresponding restore controls. The embodiment of the present application does not limit the method for displaying electronic equipment in the above-mentioned recycle bin list. For example, electronic devices may be represented only by device names in the recycle bin list.

In some embodiments, when the device icon of the electronic device is removed from the device connection state diagram 2211 , the electronic device 100 may no longer display the device icon of the electronic device in the device connection state diagram 2211 . That is, the device icon of the electronic device in the recycle bin list 2902 can be displayed in the device connection status icon 2211 again. When the user restores the electronic device in the recycle bin list 2902 through the restore control in the recycle bin list 2902 , the electronic device 100 may display the device identifier of the electronic device in the device connection status diagram 2211 again.

In some embodiments, when the cross-account device in the recycle bin list 2902 is added to the hyperterminal, the device binding process between the electronic device 100 and the cross-account device can refer to the process shown in FIGS. 24A-24D . It can be understood that the cross-account devices in the recycle bin list 2902 have been bound with the electronic device 100 once. The user can add the cross-account device in the recycle bin list 2902 to the hyperterminal without setting the connection code. This can simplify the user's operation of adding cross-account devices in the recycle bin list 2902 to the hyperterminal, and improve user experience.

It should be noted that the embodiment of the present application uses a cross-account device as an example to illustrate the device binding process of an electronic device joining the hyperterminal and the process of removing the electronic device from the hyperterminal. Other electronic devices, such as devices without an account, can also be bound with the core device of the hyper terminal when joining the hyper terminal, and the binding process can refer to the introduction of the foregoing embodiments.

Another hyperterminal management scenario provided by the embodiment of the present application is introduced below.

FIG. 30A to FIG. 30C exemplarily show the scenario of managing hyperterminals.

As shown in FIG. 30A , the electronic device 100 may display a user interface 3010 . The user interface 3010 may be a control center interface of the electronic device 100 . The user interface 3010 may include a hyperterminal function card 3011 .

One or more device icons of electronic devices may be displayed in the hyperterminal function card 3011 . The above one or more electronic devices may be searched by the electronic device 100 . The hyperterminal function card 3011 may also include a setting control 3012 . The setting control 3012 can be used to open the setting interface of the hyper terminal.

For other content contained in the user interface 3010, reference may be made to the aforementioned user interface 410 shown in FIG. 4 . I won't go into details here.

In response to the manipulation of the setting control 3012, the electronic device 100 may display the user interface 3020 shown in FIG. 30B. The user interface 3020 is the setting interface of the hyper terminal.

As shown in FIG. 30B , the user interface 3020 may include a hyperterminal entry 3021 , a video introduction 3022A, a usage prompt 3022B, a local control 3023 , and a same-account device control 3024 . in:

The hyperterminal entry 3021 can be used to trigger the electronic device 100 to provide a unified entry for interaction between the core device in the hyperterminal and multiple associated devices. For example, in response to an operation on the hyperterminal entry 3021, the electronic device 100 may display the aforementioned user interface shown in FIG. 22A.

The video introduction 3022A may include videos for introducing the functions and usage methods of the HyperTerminal. This can improve users' awareness of HyperTerminal and help users use HyperTerminal better.

The usage prompt 3022B may include text information for introducing the functions and usage methods of the HyperTerminal, so as to help the user understand the HyperTerminal. The embodiment of the present application does not limit the content in the usage prompt 3022B.

The local control 3023 can be used to view related device information of the electronic device 100 . The electronic device 100 may be a core device of a hyper terminal. For example, the electronic device 100 may be a mobile phone modeled as Hua Wei P50. The login account on the electronic device 100 may be 123xxxxxx. The above-mentioned model and login account of the electronic device 100 are only exemplary descriptions, and should not be construed as limiting the present application. In response to the operation on the local control 3023, the electronic device 100 can also display more related device information of the electronic device 100.

The same-account device control 3024 can be used to view electronic devices with the same account as the electronic device 100 (ie, the core device) in the HyperTerminal. For example, the account registered on the electronic device 100 may be Zhang San's account. The accounts logged in on Zhang San's MateBook and Zhang San's MatePad are both Zhang San's accounts. Then Zhang San's MateBook and Zhang San's MatePad may be devices with the same account in the hyper terminal whose core device is the electronic device 100 . The same-account device control 3024 may include the device identifier of Zhang San's MateBook and the device identifier of Zhang San's MatePad. In response to an operation on a device identifier in the same-account device control 3024, the electronic device 100 may display a device management interface. The device management interface can facilitate the user to manage the electronic device whose login account (or associated account) is Zhang San's account, such as removing the electronic device from the HyperTerminal, canceling the association of the electronic device with Zhang San's account, and so on.

User interface 3020 may also contain more content.

Exemplarily, in response to the user operation of sliding upward on the user interface 3020 described in FIG. 30B , the electronic device 100 may display the user interface 3020 shown in FIG. 30C .

As shown in FIG. 30C , the user interface 3020 may further include a shared device control 3025 , a cross-account device control 3026 , a binding control 3027 , and a more setting control 3028 . in:

Shared devices control 3025 can be used to view family group devices, shared devices. The aforementioned family group device may be an electronic device located in the same family group as the electronic device 100 . The above-mentioned shared device may be an electronic device jointly managed and controlled by multiple electronic devices including the electronic device 100 . For example, the MatePad icon in the shared device control 3025 shown in FIG. 30C may represent a family group device, and the MateBook icon may represent a shared device. The aforementioned family group devices and shared devices can all be associated devices in the HyperTerminal, and can cooperate with other electronic devices in the HyperTerminal.

In some embodiments, in response to manipulation of the shared device control 3025, the electronic device 100 may display a user interface of a smart living application. The above-mentioned smart life application program may be an APP for managing the above-mentioned family group equipment and shared equipment.

A cross-account device control 3026 can be used to view cross-account devices. For example, Li Si's FreeBuds and Li Si's MatePad, which are cross-account devices, are both bound to the electronic device 100 , and a hyperterminal whose core device is the electronic device 100 is added. The electronic device 100 may display Si Li's FreeBuds icon and Si Li's MatePad icon in the cross-account device control 3026 . Wherein, the cross-account device control 3026 may also include an unbinding control. For example, the unbinding control corresponding to Li Si's FreeBuds icon, and the unbinding control corresponding to Li Si's MatePad icon. The unbinding control can be used to contact the device binding relationship between the corresponding electronic device and the electronic device 100 . When unbinding, the relevant electronic device can exit the core device as the hyper terminal of the electronic device 100 .

The binding control 3027 can be used to trigger the electronic device 100 to search for other electronic devices and perform device binding.

The more setting control 3028 may include service flow recommendation options and multi-device task center options. The above-mentioned service flow recommendation option can be used to turn on or turn off the function of recommending scene-based services that can be provided by the coordinated electronic devices in the hyperterminal. For example, when the status of the service transfer recommendation option is turned on, the HyperTerminal can, according to the device type, device features, product positioning, device usage, collaborative services that the device can provide, the current environment of the device, the operating status of the device, One or more scenario-based service options based on comprehensive consideration of the user's recently used applications, currently running applications, and location information input by the user, and displayed on the user interface, so that users can quickly start related services or functions. The above multi-device task center option can be used to enable or disable the function of distributing atomized services provided by the coordinated electronic devices in the hyperterminal. For example, when the option of the multi-device task center is turned on, the hyperterminal may display an option for selecting an atomized service on the user interface according to the atomized service that can be provided by each electronic device in the hyperterminal. In this way, the user can choose which electronic device provides the atomization service in the scenario-based service provided by the hyper terminal according to the needs.

Not limited to the controls for managing the HyperTerminal shown in FIG. 30B and FIG. 30C , the user interface 3020 may also include more or less controls for managing the HyperTerminal. The layout of the user interface 3020 is only an exemplary description of the embodiment of the application, and should not limit the application.

It can be seen from the scenarios shown in FIGS. 30A to 30C above that the electronic device 100 can provide rich management controls to facilitate the user to manage the hyperterminal and provide the user with an experience of using the hyperterminal.

In some embodiments, when multiple electronic devices in the hyperterminal are coordinated, the core device of the hyperterminal can provide one or more options of scene-based services for users to choose. One or more of the above scenario-based services can be based on the device type, device features, product positioning, device usage, device-provided collaborative services, device current environment, device operating status, user recent Based on comprehensive consideration of the applications used, currently running applications, and location information input by the user. The aforementioned coordinated electronic devices may provide corresponding scenario-based services according to the option of the scenario-based service selected by the user. For the above specific scenarios, reference may be made to the scenarios shown in FIGS. 6 to 8 .

In some other embodiments, when multiple electronic devices in the hyper terminal are coordinated, the multiple electronic devices can directly provide a scenario-based service. This scenario-based service can be based on the device type, device features, product positioning, device usage, device-provided collaborative services, current environment of the device, device operating status, and recently used applications of the electronic device that the HyperTerminal uses. , the currently running application, the location information input by the user, etc., and comprehensively consider the scenario-based services that the user most likely needs.

The following introduces a scenario of collaboration between electronic devices in a hyper terminal provided by an embodiment of the present application.

FIG. 31A to FIG. 31C exemplarily show schematic diagrams of scenarios of collaboration between electronic devices in a hyper terminal.

As shown in FIG. 31A , the electronic device 100 may display a user interface 2210 . The user interface 2210 may include a device connection status icon 2211 . A mobile phone icon 505 and Zhang San's MatePad icon 3111 may be displayed in the device connection status icon 2211 . In response to the operation of dragging San Zhang's MatePad icon 3111 close to the mobile phone icon 505, the electronic device 100 can cooperate with the electronic device named Zhang San's MatePad (hereinafter referred to as Zhang San's MatePad). Among them, the electronic device 100 may comprehensively consider the scenario-based services most likely to be required by the user for collaboration between the electronic device 100 and Zhang San's MatePad, for example, multi-screen collaborative services. Then, in response to the operation shown in FIG. 31A , the electronic device 100 and Zhang San's MatePad can cooperate to provide a multi-screen collaborative service.

In response to the operation shown in FIG. 31A, the electronic device 100 may also display the user interface 2210 shown in FIG. 31B.

As shown in FIG. 31B , Zhang San's MatePad icon 3112 is attached to the mobile phone icon 505 , which may indicate that Zhang San's MatePad and the electronic device 100 have cooperated. The user interface 2210 shown in FIG. 31B may also include a service settings control 3114 . In response to the operation of the service setting control 3114, the electronic device 100 may display a service selection box 3115 shown in FIG. 31C.

As shown in FIG. 31C , the service selection box 3115 can be used for the user to select the scenario-based service provided by Zhang San's MatePad and the electronic device 100 in cooperation. The service selection box 3115 may include a multi-screen collaboration option 3115A and a camera collaboration option 3115B. An option in the service selection box 3115 being selected may indicate that the scenario-based service corresponding to this option is the scenario-based service provided by Zhang San's MatePad and the electronic device 100 . It can be seen from the above embodiment that the multi-screen collaborative service is the scenario-based service that the user most likely needs when Zhang San's MatePad cooperates with the electronic device 100 . Then, after receiving the operation of attaching the icon of Zhang San's MatePad to the mobile phone icon 505 , the electronic device 100 can provide multi-screen collaborative service with Zhang San's MatePad. The multi-screen collaboration option 3115A in the service selection box 3115 is selected. If the scene service required by the user is not a multi-screen collaborative service, the user can adjust the scene service provided by the electronic device 100 and Zhang San's MatePad through the options in the service selection box 3115 . For example, the user may select the above-mentioned camera cooperation option 3115B, so that the electronic device 100 and Zhang San's MatePad provide the camera cooperation service.

From the above scenarios shown in Figures 31A to 31C, it can be known that the HyperTerminal can predict the scenario-based services required by the user, and directly provide the predicted scenario-based services after detecting the user's operation of triggering the electronic device to establish a coordinated connection. In this way, the user can quickly make the electronic devices cooperate to provide the services they need by pulling and closing the device icons. If what the user needs is not the scenario-based service provided by the hyper terminal according to the prediction, the user can also adjust the service provided by the electronic device cooperation.

Another scenario of collaboration between electronic devices in a hyper terminal provided by an embodiment of the present application is introduced below.

FIG. 32A and FIG. 32B exemplarily show a schematic diagram of a scene where electronic devices in a hyper terminal cooperate.

As described in FIG. 32A , the electronic device 100 may display a user interface 3010 . For the content contained in the user interface 3010, reference may be made to the introduction of the aforementioned FIG. 30A. Wherein, the hyperterminal function card 3011 in the user interface 3010 may include device icons of one or more electronic devices. For example, Zhang San's MateBook icon 3011A, Zhang San's MatePad icon, Zhang San's FreeBuds icon, and Sound X icon.

The device icon of an electronic device in the hyperterminal function card 3011 can be used to trigger the electronic device 100 to cooperate with this electronic device.

Exemplarily, in response to an operation on Zhang San's MateBook icon 3011A, such as a touch operation, the electronic device 100 can cooperate with Zhang San's MateBook. Among them, the electronic device 100 can predict the scenario-based services (such as multi-screen collaborative services) that the user may most likely need when using the electronic device 100 to cooperate with Zhang San's MateBook. The electronic device 100 cooperates with Zhang San's MateBook to provide the above-mentioned predictive scenario-based service. When the electronic device 100 cooperates with Zhang San's MateBook, the electronic device 100 can display the user interface 3010 shown in FIG. 32B .

As shown in FIG. 32B , a MateBook icon 3011B of Zhang San and a collaborative prompt control 3011C may be displayed in the hyperterminal function card 3011 . The presentation forms of Zhang San's MateBook icon 3011B and Zhang San's MateBook icon 3011A shown in FIG. 32A may be different. For example, Zhang San's MateBook icon 3011B may be displayed in a dark filled form. This can be conveniently used to distinguish among the device icons presented by the hyperterminal function card 3011 , which device icons correspond to electronic devices that are coordinating with other electronic devices in the hyperterminal. The collaboration prompt control 3011C may also prompt the user that Zhang San's MateBook is cooperating with the electronic device 100 . The embodiment of the present application does not limit the expression form of the device icon in the hyper terminal function card 3011 .

In some embodiments, if the user wants to adjust the service provided by Zhang San's MateBook in cooperation with the electronic device 100, he can open the device interaction interface related to the hyperterminal function through the hyperterminal function card (refer to the user interface 2210 shown in FIG. 22A above. ). Then, the user adjusts the service provided by Zhang San's MateBook and the electronic device 100 on the device interaction interface related to the hyper terminal function.

As can be seen from the scenarios shown in FIG. 32A and FIG. 32B , the user can quickly trigger the electronic device interaction in the hyper terminal on the control center interface of the electronic device 100 , thereby providing scenario-based services. The above-mentioned operation of triggering the cooperation of the electronic devices in the hyperterminal is simple and convenient, which can improve the user's experience of interacting with multiple electronic devices.

FIG. 33A to FIG. 33C exemplarily show schematic diagrams of scenarios of cooperation of electronic devices in a hyper terminal.

As shown in FIG. 33A , the electronic device 100 may display a user interface 3010 . For the content contained in the user interface 3010, reference may be made to the introduction of the aforementioned FIG. 30A. Wherein, the hyper terminal function card 3011 in the user interface 3010 may include a Mate 40 icon 3011D. The Mate 40 icon 3011D may represent an electronic device named Mate 40 (hereinafter referred to as Mate 40). In response to an operation, such as a touch operation, on the Mate 40 icon 3011D, the electronic device 100 may display a service selection box 3311 shown in FIG. 33B . The service selection box 3311 can be used for the user to select the scenario-based services that Mate 40 and the electronic device 100 can provide in cooperation.

As shown in FIG. 33B , the service selection box 3311 may include a camera cooperation option 3311A and a microphone cooperation option 3311B. Wherein, the option of the scenario-based service included in the service selection box 3311 may be determined according to the atomized service of the electronic device 100 and the Mate40. Not limited to the above-mentioned camera cooperation option 3311A and microphone cooperation option 3311B, the service selection box 3311 may also include more or less options. As shown in FIG. 33B, in response to the operation of the camera coordination option 3311A, the electronic device 100 and the Mate 40 can cooperate to provide a camera coordination service. For example, the electronic device 100 can call the Mate 40 to turn on the camera to collect images.

When the electronic device cooperates with the Mate 40, the electronic device 100 can display a user interface 3010 as shown in FIG. 33C. In FIG. 33C, a Mate 40 icon 3011E and a cooperative prompt control 3011F can be displayed in the hyperterminal function card 3011. The representation forms of the Mate 40 icon 3011E and the Mate 40 icon 3011D shown in FIG. 33A may be different. For example, the Mate 40 icon 3011E can be displayed in a dark filled form. The above-mentioned cooperation prompt control 3011F can be used to prompt the user that the Mate 40 is cooperating with the electronic device 100.

From the scenarios shown in FIGS. 33A to 33C , the user can quickly trigger the electronic device interaction in the hyper terminal on the control center interface of the electronic device 100 , thereby providing scenario-based services. The scenario-based service provided by the electronic device in the hyper terminal may be selected by the user. The operation of triggering the coordination of the electronic devices in the hyper terminal is simple and convenient, which can improve the user's experience of interacting with multiple electronic devices.

In some embodiments, the printer can join HyperTerminal. The electronic device 100 may cooperate with the printer, and use the printer to print the file.

The following describes the scenario where the HyperTerminal provides collaborative services using printers provided by the embodiment of the present application.

FIG. 34A to FIG. 34O exemplarily show a schematic diagram of scenarios where a hyperterminal uses a printer to provide collaborative services.

As shown in FIG. 34A , the electronic device 100 may display a user interface 2210 . The electronic device 100 may display the device icon of the searched electronic device in the device connection state illustration 2211 of the user interface 2210 .

Exemplarily, the electronic device 100 may search for an electronic device named PixLab X1 printer (hereinafter referred to as PixLab X1 printer). A PixLab X1 printer icon 3401 may be displayed in the device connection status icon 2211. In response to the operation of dragging the PixLab X1 printer icon 3401 close to the mobile phone icon 505 shown in FIG. 34A , the electronic device 100 may display the print setting option box 3402 in FIG. 34B .

As shown in FIG. 34B , the setting option box 3402 may contain one or more setting options for instructing the printer to print the file. For example, number of copies option, print area option, printing paper single/double sided option, printing paper landscape/vertical option, printing paper size option, page margin option, etc.

Wherein, the above-mentioned number of copies option can be used to set the number of copies that the printer prints for the received file that needs to be printed. For example, 1 part, 2 parts, etc. The print area option can be used to set the area to be printed in the file. For example, the area to be printed can be the entire document, or it can be the first page of a document. The Print Paper Single/Double Sided option can be used to set whether to print on one side or two sides. The Print Paper Landscape/Portrait option can be used to set whether to print in landscape or portrait orientation on the paper. The Print Paper Size option can be used to set the size of the paper used for printing. For example, the size of the paper is A3, A4, and so on. The page margin option can be used to set the margins of the printed content on the paper. Not limited to the setting options shown in FIG. 34B, the setting options box 3402 may also contain more or fewer options.

When the electronic device 100 cooperates with the aforementioned PixLab X1 printer, the electronic device 100 can display the user interface 2210 shown in FIG. 34C. In the user interface 2210 shown in FIG. 34C, the PixLab X1 printer icon 3402 is attached to the mobile phone icon 505, which may indicate that the PixLab X1 printer and the electronic device 100 have been coordinated. The representation of the PixLab X1 printer icon 3402 may be different from the representation of the aforementioned PixLab X1 printer icon 3401 shown in FIG. 34A . For example, the PixLab X1 printer icon 3402 may appear as a dark fill. The user interface 2210 may also include a printing hover 3405 . Print control 3405A may be included in print hover 3405 . The print control 3405A can be used to trigger the electronic device 100 to send the print file to the PixLab X1 printer for printing. The printing floating ball 3405 can be suspended on the top layer of the user interface.

In some embodiments, during the cooperation between the electronic device 100 and the aforementioned PixLab X1 printer, if no operation on the printing floating ball 3405 is detected within a preset period of time, the electronic device 100 may display the display shown in FIG. 34D Print levitation ball 3406. The expression form of printing floating ball 3405 and printing floating ball 3406 may be different. For example, printed hover 3406 may be displayed with less transparency than printed hover 3405, with a portion of printed hover 3405 hidden. That is to say, when the user has not used the cooperative service provided by the PixLab X1 printer and the electronic device 100 for a long time, the above-mentioned printed floating ball can still be suspended on the user interface, but the transparency of the display and the display area occupied can be reduced. Thereby reducing the impact of the user using other controls in the user interface.

As shown in FIG. 34E , during the period when the electronic device 100 cooperates with the above-mentioned PixLab X1 printer, the printing floating ball can continuously float on the user interface. The user may open a user interface on the electronic device 100 displaying files to be printed.

Exemplarily, the electronic device 100 may display the user interface 3410 shown in FIG. 34E. The user interface 3410 may be a user interface of a gallery application. The user interface 3410 may include a picture 3411. A print hover 3406 may be suspended over the user interface 3410 . In response to the operation of printing the floating ball 3406, the electronic device 100 may display the printing floating ball 3407 shown in FIG. 34F. The above-mentioned printing suspension ball 3406 may be equivalent to shrinking the suspension ball, that is, it is obtained by shrinking the above-mentioned printing suspension ball 3405 shown in FIG. 34C . The above-mentioned printed floating ball 3407 may be equivalent to unfolding the floating ball, that is, it is obtained by expanding the printed floating ball 3405 .

As shown in Figure 34F, print hover 3407 may contain print control 3405A. The printing floating ball 3407 may also include a text prompt "click the control on the left to print", to remind the user how to use the printing floating ball 3407. In response to an operation on the print control 3405A, such as a touch operation, the electronic device 100 can send the file currently displayed on the user interface 3410 (ie, the picture 3411 ) to the PixLab X1 printer for printing.

In the case that the electronic device 100 has sent the file to be printed by the PixLab X1 printer, the electronic device 100 can display the printing floating ball 3408 shown in FIG. 34G . Print control 3405A may also be included in print hover 3408 . However, the text prompt "printing, click to view the printing task" contained in the printing floating ball 3408 is different from the text prompt in the printing floating ball 3407. The text prompt in the printing floating ball 3408 can prompt the user how to use the printing floating ball to check the printing task. It can be understood that, when there is no printing task between the electronic device 100 and the PixLab X1 printer, the electronic device 100 can display the printing floating ball 3407 shown in FIG. 34F above. When there is a printing task between the electronic device 100 and the PixLab X1 printer, the electronic device 100 can display the printing floating ball 3408 shown in FIG. 34G above. The embodiment of the present application does not limit the expression forms of printing the floating ball under the above-mentioned different situations. For example, in the above different situations, the expression form of printing the floating ball can also be the same.

As shown in FIG. 34F , the user can view different pictures through a user operation of sliding left or right on the user interface 3410 . Exemplarily, in response to a user operation of sliding left on the user interface 3410, the electronic device 100 may display the user interface 3410 shown in FIG. 34H. In FIG. 34H , a picture 3412 may be displayed on the user interface 3410 . In response to the operation on the print control 3405A, the electronic device 100 can send the file (ie, the picture 3412) currently displayed on the user interface to the PixLab X1 printer for printing.

As shown in FIG. 34I , in response to an operation, such as a touch operation, on an area outside the print control 3405A in the print floating ball 3408 , the electronic device 100 may display a print job list 3413 as shown in FIG. 34J . The print task list 3413 may include a file name 3414 , a print status 3415 , a file name 3416 , and a pause control 3417 . Wherein, the file name 3414 may be, for example, cat.jpg, which may represent a print job for printing the picture 3411 shown in FIG. 34F. The print status 3415 can be used to reflect the status of the print job corresponding to the file name 3414 . For example, the printing status 3415 can be completed, which can indicate that the PixLab X1 printer has completed printing the above picture 3411.

The file name 3416 may be, for example, dog.jpg, which may represent a print job for printing the picture 3412 shown in FIG. 34H. The pause control 3417 may indicate that the print task corresponding to the file name 3416 is being printed but not yet completed. The pause control 3417 can be used to pause the print job corresponding to the file name 3416 . Exemplarily, in response to the operation on the above-mentioned pause control 3417, the electronic device 100 may send an instruction to the PixLab X1 printer to pause printing the picture 3412. When the above picture 3412 is printed, the electronic device 100 can change the above pause control 3417 to the control shown in the print state 3415 to remind the user that the PixLab X1 printer has completed printing the above picture 3412.

The embodiment of the present application does not limit the implementation method for the electronic device 100 and the PixLab X1 printer to cooperate to provide printing services. For example, in response to the operation on the print control 3405A, the electronic device 100 can send the file currently displayed on the user interface to the PixLab X1 printer for printing. The above-mentioned files displayed on the current user interface are not limited to pictures, but may also be text-type documents, tables, and the like. For another example, in response to the operation on the print control 3405A, the electronic device 100 may perform a screenshot operation on the currently displayed user interface, and send the file obtained by the screenshot operation to the PixLab X1 printer for printing.

In some embodiments, the user can stop the electronic device 100 from cooperating with the PixLab X1 printer by dragging the PixLab X1 printer icon 3403 away from the phone icon 505 in the user interface shown in FIG. 34C .

In some embodiments, the user can also stop the electronic device 100 from cooperating with the PixLab X1 printer by printing the floating ball as described above. Exemplarily, as shown in FIG. 34K , in response to an operation on the printing floating ball 3408, such as a long press operation, the electronic device 100 can move and print on the user interface according to the finger movement track when the user drags the printing floating ball 3408. levitate ball.

Wherein, the electronic device 100 may display the printed floating ball 3409 shown in FIG. 34L. The printed floating ball 3409 and the printed floating ball 3408 may have different expressions. During the process of changing the display position of the printed floating ball 3409 as the user drags the track, the electronic device 100 may display the delete control 3431 shown in FIG. 34L on the user interface 3410 .

As shown in FIG. 34M , in response to the operation of dragging the printing floating ball 3409 to the location of the delete control 3431 , the electronic device 100 may stop the cooperation between the electronic device 100 and the PixLab X1 printer, and cancel the display of the printing floating ball on the user interface.

In some embodiments, when receiving an operation acting on the print control 3405A, the electronic device 100 may obtain a printable file on the currently displayed user interface. If there is no printable file on the currently displayed user interface, the electronic device 100 may prompt the user that there is no printable content on the current page.

Exemplarily, as shown in FIG. 34N , during the cooperation between the electronic device 100 and the PixLab X1 printer, the electronic device 100 may display a user interface 3440 . The user interface 3440 may be a desktop of the electronic device 100 . A print floating ball 3407 is suspended on the user interface 3440 . In response to the operation of the print control 3405A in the print floating ball 3407, the electronic device 100 may display a prompt box 3441 as shown in FIG. 34O. The prompt box 3441 may be used to prompt that there is no printable file on the user interface 3440 and the print operation cannot be performed. The prompt box 3441 may include a print reminder control 3442 . The above-mentioned printing notice control 3442 can be used to trigger the electronic device 100 to display the precautions for printing with the PixLab X1 printer, so as to help the user understand how to use the printing service provided by the electronic device 100 and the PixLab X1 printer. The embodiment of this application does not limit the content contained in the prompt box 3441.

In some embodiments, before the PixLab X1 printer cooperates with the electronic device 100, it can be bound with the electronic device 100 first. For the above device binding process, reference may be made to the introduction of the aforementioned embodiments related to device binding. I won't go into details here.

It can be known from the scenarios shown in FIGS. 34A to 34O above that the printer can join the hyperterminal and cooperate with other electronic devices in the hyperterminal to provide users with printing services. The above method can facilitate the user to quickly coordinate the electronic device 100 with the printer. Moreover, during the period when the above-mentioned electronic device 100 is cooperating with the printer, the user can browse and search for files to be printed, and realize fast printing through the above-mentioned printing floating ball. The above-mentioned user operation for printing is simple and convenient, which can improve the user experience of using the printer for collaboration. When all the printing tasks are completed, the user can also quickly stop the coordination between the electronic device 100 and the printer through the printing suspension ball.

The implementation manners described in the foregoing embodiments are only exemplary illustrations, and do not constitute any limitation to other embodiments of the present application. The specific internal implementation may vary depending on the type of electronic device, the operating system installed, the program used, and the interface called. feature functions.

As used in the above embodiments, depending on the context, the term "when" may be interpreted to mean "if" or "after" or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrases "in determining" or "if detected (a stated condition or event)" may be interpreted to mean "if determining..." or "in response to determining..." or "on detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.

The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still apply to the foregoing embodiments Modifications are made to the recorded technical solutions, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of each embodiment of the application.

Claims

A device interaction method, characterized in that the method comprises:

The first device displays a first interface, the first interface includes a first control and a second control, the first control is a device option indicating the first device, and the second control is a device indicating a second device option, the second device is a device discovered by the first device;

The first device detects a first user operation, and the first user operation is used for a user to associate the first device with the second device;

The first device displays a second interface, and the second interface displays service options of a first scenario-based service, and the first scenario-based service is the first device associated with the first user operation and Scenario-based services supported by the second device;

The first device detects a second user operation for selecting the first scenario-based service;

The first device instructs to run the first scenario-based service selected by the second user operation.
The method according to claim 1, wherein in the first interface, the second controls are distributed on a circle centered on the first control, and the first control and the second Controls are detached.
The method according to claim 1 or 2, wherein the first user operation includes selecting the second control, or selecting the first control, or displaying the first control and the second control The operation in which controls are selected simultaneously.
The method according to any one of claims 1-3, wherein the first user operation is that the user selects the second control and drags the second control toward the first After the direction of the control moves to the specified area, release the operation of the second control, the specified area is a circular area with the first control as the center and the radius as the first radius distance; the second interface also displays The first control and the second control are adsorbed together.
The method according to any one of claims 1-3, wherein a floating control is displayed in the second interface, the floating control includes a first area, and the first area displays the first scene The service option of the customized service; the second user operation is the operation of releasing the second control after the user drags the second control to the first area, or the second user operation is the operation of releasing the second control The user drags the second control through the first area, moves toward the direction close to the first control to a designated area, and then releases the operation of the second control, and the designated area is based on the first area. The control is a circular area with a center and a radius of the first radius distance.
The method according to claim 5, wherein the floating control includes multiple circular areas, or multiple rectangular areas, or multiple fan-shaped areas, or multiple fan-shaped ring areas, or multiple circular ring-shaped areas , or multiple polygonal regions.
The method according to any one of claims 1-6, wherein the second interface includes the first control.
The method according to any one of claims 1-7, wherein the positional relationship between the first control and the second control in the first interface indicates that the first device and the second One or more of the following relationships between devices: connection relationship, or orientation relationship, or distance relationship, or signal strength relationship;

Wherein, the first control is separated from the second control, indicating that the first device is not connected to the second device, and the first control is connected with the second control, indicating that the first device is not connected to the second control. said second device is connected;

And/or, the second control is located on the left, or right, or above, or below the first control, respectively indicating that the second device is located on the left, or right, or in front of the first device , or rear;

And/or, the farther the distance between the first control and the second control is, it indicates that the distance between the first device and the second device is farther, or the distance between the first device and the second control is farther. The weaker the signal between the second device is, the closer the distance between the first control and the second control is, indicating that the closer the distance between the first device and the second device is, or, The stronger the signal between the first device and the second device.
The method according to any one of claims 1-8, wherein the first scenario-based service is a scenario-based service queried from a first database, and the first database records one or more One or more scenario-based services supported by a device combination, where the device combination includes a device combination composed of the first device and its associated second device;

The scenario-based service is determined by one or more of the following parameters: device type, device feature, product positioning, device usage, device environment or scene, device status, and recently run applications of the device combination.
The method according to any one of claims 1-8, wherein the first scenario-based service is a scenario-based service queried from a second database, and the second database records one or more One or more scenario-based services supported by an atomic service combination, where the atomic service combination includes an atomic service combination composed of atomic capabilities possessed by the first device and its associated second device;

The atomic capabilities include one or more of the following: audio output capabilities, audio input capabilities, display capabilities, camera capabilities, touch input capabilities, and keyboard and mouse input capabilities.
The method according to any one of claims 1-10, wherein the first scenario-based service calculates the probability of being selected according to one or more of the following parameters: the user uses the first scenario-based service The frequency of the service, the ranking of the first scenario-based service set by the user, the environment or scenario where the first device and/or the second device are located, the first device and/or the second device The state of the second device, the most recently run application of the first device and/or the second device; the service option of the first scenario-based service with a higher probability of being selected by the user is recommended and displayed with higher priority.
The method according to any one of claims 5-11, wherein the shape of the floating control is a circular ring or a fan ring, wherein a plurality of circular ring areas or fan ring areas respectively indicate a plurality of the For the first scenario-based service, the first scenario-based service indicated by the circular area or fan-shaped area closer to the center of the circle is a more preferentially recommended scenario-based service.
The method according to any one of claims 1-12, wherein the second interface also displays device options of a third device, and the third device evaluates the first device and its associated The second device is the recommended device according to the device usage requirements.
The method according to claim 13, wherein the third device is a device discovered by the first device, and the device option of the third device is clicked, and the first device displays the information of the second scene-based service Service options.
The method according to claim 13, wherein the third device is not the device discovered by the first device, and the device option of the third device is clicked, and the first device jumps to display the fifth interface, The fifth interface includes a plurality of item options, the plurality of item options come from multiple supply sources, and the plurality of item options include an item option indicating the third device.
The method according to any one of claims 13-15, wherein the third device is a device recommended based on one or more parameters: the first device and/or its associated The device type, device feature, product positioning, device use, device environment or scene, device state, recently run application of the second device, or the first scene-based service, or predictable and scalable use scene.
The method according to any one of claims 1-16, wherein the first scenario-based service is supported by a first atomic service composition, and the first atomic service composition includes the first device At least one first service provided, and/or at least one second service provided by the second device.
The method according to any one of claims 1-17, further comprising:

The first device runs the first scenario-based service, and/or, the second device associated with the first device runs the first scenario-based service.
The method according to any one of claims 1-18, characterized in that the third interface displayed by the first device running the first scenario-based service is different from the second device running the first scenario The fourth interface displayed by the customized service is different.
The method according to claim 19, wherein if the first scenario-based service is a screen projection service, the second device is a large-screen device, the fourth interface is a video or image screen, and the second The third interface is a control interface for controlling the display function of the video or image screen.
The method according to claim 19, wherein if the first scenario-based service is a sports monitoring service, the second device is a wearable sports device, the third interface is a picture of a sports track, and the The fourth interface is the detected exercise data of the user, and the exercise data includes: exercise speed, exercise distance, exercise time, and heart rate.
The method according to any one of claims 1-20, further comprising:

In the screen projection scenario, the first device determines that the screen interface of the first device is projected on the second control according to the relative position of the second control and the first control when the drag operation is released. The position in the screen of the second device.
The method of claim 22, wherein,

If the relative position is that the second control is on the left side of the first control, then the screen interface of the first device is projected on the right side of the screen of the second device;

If the relative position is that the second control is on the right side of the first control, then the screen interface of the first device is projected on the left side of the screen of the second device;

If the relative position is that the second control is in the middle of the first control, then the full screen of the screen interface of the first device is projected on the screen of the second device.
The method according to any one of claims 4, 7-11, 13-23, wherein the method further comprises:

The first device detects a third user operation acting on the second interface, the third user operation is used to associate the fourth device with the first device, and the third user operation includes selecting instructing the third control operation of the fourth device;

In response to the third user operation, the first device displays a sixth interface, the sixth interface displays the first control, the second control, and the third control are adsorbed together, the The sixth interface also displays a third scenario-based service option, the third scenario-based service option is determined according to the first device, the second device, and the fourth device, and the third scenario-based service option The service options are different from the first scenario-based service options.
The method according to any one of claims 1-24, characterized in that, after the first device instructs the second user to operate and select the operation of the first scenario-based service, the method further comprises :

the first device detects a fourth user action, the fourth user action comprising an action to separate the first control from the second control;

The first device instructs the first scenario-based service selected by the second user to stop running.
The method according to claim 25, further comprising:

The first device stops running the first scenario-based service selected by the second user operation, and/or the second device stops running the first scenario-based service selected by the second user operation.
The method according to claim 25 or 26, wherein the fourth user operation is that the user selects the second control and drags the second control to move away from the first control , until the operation of the second control is released after moving out of the specified area.
The method according to any one of claims 1-27, wherein the first device includes any one of the following: a mobile phone, a tablet computer, a portable/non-portable computer, a personal computer, and a smart TV; 2. Equipment includes any of the following: mobile phones, tablet computers, portable mobile computers, desktop personal computers, smart speakers, smart watches, smart bracelets, smart TVs, earphones, smart glasses, car machines, smart cockpits, game consoles, treadmills , spinning bikes, weight scales, body fat scales, water heaters, lamps, air conditioners, blood glucose meters, oximeters, heart rate meters, AR/VR devices, cloud hosts/cloud servers, smart wearable devices, smart home devices, printers.
The method according to any one of claims 1-28, wherein the first scenario-based service is a printing service, and the second device is a printer, and the method further includes:

The first device displays a floating printing control, the floating printing control is displayed by the first device running the first scene-based service, and the floating printing control is displayed on a user interface of the first device.
The method according to claim 29, further comprising:

The first device detects a fifth user operation on the print hover control;

The first device sends a first print file to the second device for printing based on the fifth user operation, and the first print file is displayed on the screen when the first device detects the fifth user operation. content on the first device.
The method according to claim 29 or 30, further comprising:

The first device detects a sixth user operation on the print hover control;

The first device deletes the printing suspension control based on the sixth user operation, and instructs the first scene-based service to stop running.
A device interaction method, characterized in that the method comprises:

The first device displays a first interface, the first interface includes a first control and a second control, the first control is a device option indicating the first device, and the second control is a device indicating a second device option, the second device is a device discovered by the first device;

The first device detects a first user operation, and the first user operation is used to associate the first device with the second device;

The first device sends a first binding request to the second device according to the first user operation, and receives a first consent message from the second device;

The first device displays a second interface according to the first consent message, and the second interface displays service options of a first scenario-based service, and the first scenario-based service is a combination of the first device and the Scenario-based services supported by the second device.
The method according to claim 32, wherein the first device sends a first binding request to the second device according to the first user operation, specifically comprising:

The first device displays a connection code setting box according to the first user operation, and the connection code setting box is used to set a connection code for binding the first device and the second device;

The first device receives the first connection code input in the connection code setting box, and sends the first binding request to the second device;

Wherein, the first agreement message includes the first connection code.
The method according to claim 32 or 33, wherein in the first interface, the second controls are distributed on a circle with the first control as the center and a radius of the first length, the The first control is separated from the second control.
The method according to claim 34, wherein the first interface further includes a fourth control, the fourth control is a device option indicating a fifth device, and the fifth device is the first device discovered by the first device. device, the account associated with the fifth device is the same as the account associated with the first device; the fourth control is distributed on a circle whose center is the first control and whose radius is the second length; A control is separated from the fourth control, and the first length is different from the second length.
The method according to any one of claims 32-35, wherein the account associated with the first device is different from the account associated with the second device, or the second device has no associated account.
The method according to claim 35 or 36, wherein the first interface further includes a fifth control, the fifth control is a device option indicating a sixth device, and the sixth device is the first For the device discovered by the device, the fifth control is in a separate state from the fourth control, and the method further includes:

The first device detects a seventh user operation for associating the fifth device with the sixth device;

The first device instructs the fifth device to request to bind with the sixth device according to the seventh user operation, and receives a message indicating that the sixth device agrees to bind with the fifth device The second consent message of ;

The first device displays an eighth interface according to the second consent message, and in the eighth interface, the fourth control and the fifth control are adsorbed together.
The method according to any one of claims 32-37, further comprising:

The first device detects a user operation acting on the second control, and removes the second control on the first interface.
An electronic device, characterized in that the electronic device includes: a communication device, a display device, a memory, a processor coupled to the memory, and one or more programs; the communication device is used for communication, and the display The device is used for displaying an interface, and computer-executable instructions are stored in the memory, and when the processor executes the instructions, the electronic device implements any one of claims 1-17, 19-25, 27-31 the method described.
A computer-readable storage medium, comprising instructions, characterized in that, when the instructions are run on an electronic device, the electronic device is made to perform any one of claims 1 to 17, 19 to 25, 27 to 31 the method described.
A communication system, the communication system comprising a first device and a second device, wherein the first device executes the method according to any one of claims 1-17, 19-25, 27-31.
An electronic device, characterized in that the electronic device includes: a communication device, a display device, a memory, a processor coupled to the memory, and one or more programs; the communication device is used for communication, and the display The device is used for displaying an interface, and computer-executable instructions are stored in the memory, and when the processor executes the instructions, the electronic device implements the method according to any one of claims 32 to 38.
A computer-readable storage medium, comprising instructions, wherein when the instructions are run on an electronic device, the electronic device is made to execute the method according to any one of claims 32 to 38.