CN110430323B - Multitask intelligent coordination method, device and system - Google Patents

Multitask intelligent coordination method, device and system Download PDF

Info

Publication number
CN110430323B
CN110430323B CN201910711129.6A CN201910711129A CN110430323B CN 110430323 B CN110430323 B CN 110430323B CN 201910711129 A CN201910711129 A CN 201910711129A CN 110430323 B CN110430323 B CN 110430323B
Authority
CN
China
Prior art keywords
equipment
user
main control
expansion
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910711129.6A
Other languages
Chinese (zh)
Other versions
CN110430323A (en
Inventor
李香兰
刘坤
仇卫华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics China R&D Center
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics China R&D Center
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics China R&D Center, Samsung Electronics Co Ltd filed Critical Samsung Electronics China R&D Center
Priority to CN201910711129.6A priority Critical patent/CN110430323B/en
Publication of CN110430323A publication Critical patent/CN110430323A/en
Application granted granted Critical
Publication of CN110430323B publication Critical patent/CN110430323B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G06F13/4063—Device-to-bus coupling
    • G06F13/4068—Electrical coupling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72454—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72463—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions to restrict the functionality of the device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72469—User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72484—User interfaces specially adapted for cordless or mobile telephones wherein functions are triggered by incoming communication events
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/484—Precedence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion

Abstract

The application discloses a multitask intelligent coordination method, which comprises the following steps: when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select; the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment; the main control device sends the task selected by the user to the corresponding expansion device for display, and converts the task into background operation; and coordinating the output parameters of the main control equipment and the expansion equipment according to the use condition of the main control equipment and the expansion equipment by the user and the category of the currently operated task. The application also discloses a corresponding device and system. By applying the technical scheme disclosed by the application, the intelligence of the equipment can be improved, and multiple tasks can be better cooperated.

Description

Multitask intelligent coordination method, device and system
Technical Field
The present application relates to the field of multitask technologies, and in particular, to a multitask intelligent coordination method, device and system.
Background
In recent years, with the increasing size of mobile phone screens and the increasing abundance of mobile phone applications (apps), more and more things can be processed on mobile phones. In order to perform multiple tasks on a mobile phone at the same time, a split-screen and multi-task technology is proposed in the industry.
The screen division and multitasking method is characterized in that a mobile phone screen is divided into a plurality of interfaces, 2 or more than 2 application programs or tasks are opened at different interfaces simultaneously, and users can operate the application programs or the tasks respectively without mutual influence. Such as: and watching videos when swiping a microblog or seamlessly searching for materials by using a browser when writing articles.
However, the existing split screen technology has the following problems:
1. partial versions of the mobile device support split screen mode and can perform multiple tasks simultaneously. However, the screen of the mobile phone is too small, so that the user experience of the split screen is poor.
2. After the split screens are performed, the content on each split screen needs to be manually clicked, and operations such as audio switching are performed. Also, when 2 videos are played simultaneously in the split screen mode, if one starts playing, the other stops playing, which causes a very bad user experience.
3. When a user has a plurality of devices on hand, the user wants to use a certain application program only by downloading corresponding software from an application store of the device. In the prior art, no good solution is provided for enabling a user to use an application a on other devices, provided that the user's device a has the application a installed, and the other devices need to use the application a due to insufficient space or no desire to install the application a.
Disclosure of Invention
The application provides a multitask intelligent coordination method, equipment and a system, which are used for improving the intelligence of the equipment and enabling multitask to be better cooperated.
The application discloses a multitask intelligent coordination method, which comprises the following steps:
when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment;
the main control device sends the task selected by the user to the corresponding expansion device for display, and converts the task into background operation;
and coordinating the output parameters of the main control equipment and the expansion equipment according to the use condition of the main control equipment and the expansion equipment by the user and the category of the currently operated task.
Preferably, the usage of the master control device and the expansion device by the user at least includes: the attention condition and the operation condition of the user to the master control equipment and the expansion equipment;
the categories of the currently running task at least include: the sound requirement level of the currently running application program, the security requirement level of the application program;
the output parameters of the main control device and the expansion device at least comprise: volume, screen brightness, screen resolution, screen refresh rate.
Preferably, the method further comprises:
and recording when the user manually adjusts the output parameters of the main control equipment or the expansion equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
Preferably, the screen output parameters of the main control device and the expansion device are coordinated according to the following order of conditional priority:
conditional priority 1: if the user only operates one device in the multiple devices within a set time, and other devices do not detect the control operation or gesture instruction of the user, the other devices reduce the respective screen brightness, refresh rate and/or resolution;
conditional priority 2: if the operation of the user on the equipment is not detected within the set time, if the equipment has a plurality of auxiliary cameras, the auxiliary cameras are started for tracking the track of eyeballs of the user, the equipment which the user is watching is determined, the screen brightness, the refresh rate and/or the resolution of the equipment which the user is watching is kept normal, and the screen brightness, the refresh rate and/or the resolution of the equipment which the user is not watching is reduced;
conditional priority 3: if the operation of the user on the equipment is not detected within the set time, if the equipment is not provided with a plurality of auxiliary cameras, the front main camera is started to track the track of eyeballs of the user, the equipment which the user is watching is determined, the screen brightness, refresh rate and/or resolution of the equipment which the user is watching is kept normal, and the screen brightness, refresh rate and/or resolution of the equipment which the user is not watching is reduced;
conditional priority 4: if the equipment does not have a camera or the user does not start the eyeball tracking service, determining the equipment which is currently concerned by the user through a distance sensor of the equipment, keeping the screen brightness, the refresh rate and/or the resolution of the equipment which is concerned by the user normal, and reducing the screen brightness, the refresh rate and/or the resolution of the equipment which is not concerned by the user;
conditional priority 5: and if the conditions of the condition priorities 2-4 cannot be met, the equipment starts voice monitoring and adjusts the output of the corresponding equipment according to the voice recognition result.
Preferably, the audio output parameters of the main control device and the extension device are coordinated in the following manner:
according to the level of the sound output requirement of the application program, dividing the application program installed in the main control equipment into strong audio requirement software, weak audio requirement software, no audio requirement software and dynamic audio requirement software;
and for the application programs with different sound output requirement levels, corresponding audio output parameters are adopted for audio output.
The application also discloses a multitask intelligent coordination system, including: a master device and at least one expansion device, wherein:
when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment;
the main control device sends the task selected by the user to the corresponding expansion device for display, and converts the task into background operation;
and according to the service conditions of the main control equipment and the expansion equipment by the user and the types of the tasks currently running, the main control equipment and the expansion equipment coordinate the output parameters of the main control equipment and the expansion equipment.
The application also discloses a master control device, including at least in this master control device: register the connection module, throw screen module, situation identification module and system setting module, wherein:
the registration connection module is used for providing an equipment list for the user to select and pairing with the equipment selected by the user when the fact that the user activates the split screen mode is detected; wherein, the successfully paired device becomes an expansion device;
the screen projection module is used for projecting the tasks selected by the user to corresponding expansion equipment for display and converting the tasks into background operation;
the situation identification module is used for coordinating the output parameters of the main control equipment and the expansion equipment according to the use condition of the main control equipment and the expansion equipment by a user and the category of the currently operated task;
and the system setting module is used for setting the output parameters of the main control equipment.
Preferably, the context identification module is further configured to:
and recording when the user manually adjusts the output parameters of the main control equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
Preferably, the master control device further includes a security authentication module:
the safety authentication module is used for receiving authentication information of a user from the expansion equipment, carrying out safety authentication on the authentication information and allowing the user to perform related operations on the expansion equipment after the safety authentication is passed; wherein the authentication information includes: fingerprint, facial feature information, password.
Preferably, the main control device further comprises an information processing module;
the information processing module is used for receiving operation information of a user in a task of projecting a screen to the expansion equipment from the expansion equipment and carrying out related adjustment according to the operation information.
The application also discloses an extension device, including: register connection module, display module, interface interaction module, information processing execution module and system setting module, wherein:
the registration connection module is used for pairing with the master control equipment;
the information processing execution module is used for adapting various parameters configured by the main control equipment, processing the tasks delivered by the main control equipment, displaying the tasks through the display module, collecting the operations of the users in the tasks in the expansion equipment, and sending the operations to the main control equipment;
the interface interaction module is used for interacting with a user;
and the system setting module is used for setting the output parameters of the expansion equipment.
Preferably, the expansion device further comprises a security authentication module;
the security authentication module is used for collecting authentication information of a user, sending the authentication information to the main control equipment and receiving security authentication results separated by the main control equipment; wherein the authentication information includes: fingerprint, facial feature information, password.
According to the technical scheme, the tasks selected by the user are delivered to the corresponding expansion equipment from the main control equipment to be displayed, the output parameters of the main control equipment and the output parameters of the expansion equipment are coordinated according to the use conditions of the main control equipment and the expansion equipment by the user and the types of the tasks running at present, the capacity of executing multiple tasks through intelligent coordination of multiple devices is improved, and the multiple tasks can be better cooperated. Compared with the prior art, the application can obtain the following beneficial effects:
1. the screen of the current equipment is expanded through other equipment, so that a user can obtain better multi-task use experience;
2. the resources of a plurality of devices of a user are intelligently coordinated, so that the plurality of devices can better cooperate to meet the requirements of the user;
3. by learning user habits and carrying out personalized services according to the individual habits of the user, the user experience is improved, and the intelligence of the equipment is improved.
Drawings
FIG. 1 is a schematic diagram of a multitask intelligent coordination method according to the present application;
FIG. 2 is a schematic structural diagram of a multitask intelligent coordination system according to an embodiment of the present application;
FIG. 3 is a schematic interaction flow diagram of a multitask intelligent coordination method according to an embodiment of the present application;
fig. 4 is a schematic flowchart illustrating a process of registering and connecting a master device in an embodiment of the present application;
fig. 5 is a schematic flowchart of a registration connection at an expansion device side in an embodiment of the present application;
fig. 6 is an example of a system setting module and an extended device setting interface in the first embodiment of the present application;
FIG. 7 is a diagram illustrating a second exemplary scenario of the present application;
fig. 8 is a schematic view of a third scenario in the present application;
FIG. 9 is a diagram illustrating a fourth exemplary scenario of the present application;
fig. 10 is a schematic view of a fifth scenario according to an embodiment of the present application;
fig. 11 is a schematic view of a seventh scenario according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below by referring to the accompanying drawings and examples.
In order to solve the problems existing in the prior art, the application provides a multitask intelligent coordination technical scheme, and the main idea of the scheme comprises the following steps: by projecting screens among a plurality of devices of a user, one or a plurality of tasks of the user are projected to other expansion devices except the main control device to be displayed, all the tasks still run on the main control device, and due to the fact that the display area is expanded, the use experience of the user when the main control device executes the multiple tasks is greatly optimized. Moreover, the present application proposes a new mode: and in the personal multi-device cooperation mode, when the plurality of devices are confirmed to be in the personal multi-device cooperation mode, the use conditions of the plurality of devices are comprehensively judged according to the classification of the applications, the sound requirement level of the applications, the watching of the users and other conditions, and the volume, the screen brightness, the resolution and the like of the plurality of devices are further coordinately output, so that the plurality of devices can better cooperate to meet the requirements of the users. On the basis of the scheme, the method and the device further record when the user manually adjusts the volume, the screen brightness or the resolution of the device each time, analyze the personal habits of the user according to the records, and adjust the output of the main control device and the expansion device according to the analysis result of the personal habits, so that personalized service is provided for the user.
As described above, the present invention proposes a new mode, called "personal multi-device cooperation mode", in which one device is called "master device" and is mainly used to perform multitasking for users; the other devices are called as 'expansion devices', and are mainly used for receiving screen projection of the main control device and displaying screen projection contents.
FIG. 1 is a schematic diagram of a multitask intelligent coordination method of the present invention, which comprises the following steps:
step 101: when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
step 102: the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment;
step 103: the main control device sends the task selected by the user to the corresponding expansion device for display, and converts the task into background operation;
step 104: and coordinating the output parameters of the main control equipment and the expansion equipment according to the use condition of the main control equipment and the expansion equipment by the user and the category of the currently operated task.
Wherein, the use condition of the user to the master control device and the expansion device at least comprises: the attention condition and the operation condition of the user to the master control equipment and the expansion equipment;
the categories of the currently running task at least include: the sound requirement level of the currently running application program, the security requirement level of the application program;
the output parameters of the main control device and the expansion device at least comprise: volume, screen brightness, screen resolution, screen refresh rate.
On the basis of the method shown in fig. 1, the method may further include: and recording when the user manually adjusts the output parameters of the main control equipment or the expansion equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
Corresponding to the method, the application also provides a multitask intelligent coordination system, which comprises: a master device and at least one expansion device, wherein:
when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment;
the main control device sends the task selected by the user to the corresponding expansion device for display, and converts the task into background operation;
and according to the service conditions of the main control equipment and the expansion equipment by the user and the types of the tasks currently running, the main control equipment and the expansion equipment coordinate the output parameters of the main control equipment and the expansion equipment.
In addition, the present application further provides a master control device, which at least includes: register the connection module, throw screen module, situation identification module and system setting module, wherein:
the registration connection module is used for providing an equipment list for the user to select and pairing with the equipment selected by the user when the fact that the user activates the split screen mode is detected; wherein, the successfully paired device becomes an expansion device;
the screen projection module is used for projecting the tasks selected by the user to corresponding expansion equipment for display and converting the tasks into background operation;
the situation identification module is used for coordinating the output parameters of the main control equipment and the expansion equipment according to the use condition of the main control equipment and the expansion equipment by a user and the category of the currently operated task;
and the system setting module is used for setting the output parameters of the main control equipment.
The situation recognition module in the main control module can also be used for: and recording when the user manually adjusts the output parameters of the main control equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
Preferably, the master control device may further include a security authentication module: the safety authentication module is used for receiving authentication information of a user from the expansion equipment, carrying out safety authentication on the authentication information and allowing the user to perform related operations on the expansion equipment after the safety authentication is passed; wherein the authentication information includes: fingerprint, facial feature information, password.
Preferably, the main control device may further include an information processing module; the information processing module is used for receiving operation information of a user in a task of projecting a screen to the expansion equipment from the expansion equipment and carrying out related adjustment according to the operation information.
In addition, the present application further provides an extension device, including a registration connection module, a display module, an interface interaction module, an information processing execution module, and a system setting module, wherein:
the registration connection module is used for pairing with the master control equipment;
the information processing execution module is used for adapting various parameters configured by the main control equipment, processing the tasks delivered by the main control equipment, displaying the tasks through the display module, collecting the operations of the users in the tasks in the expansion equipment, and sending the operations to the main control equipment;
the interface interaction module is used for interacting with a user;
and the system setting module is used for setting the output parameters of the expansion equipment.
Preferably, the expansion device may further include a security authentication module; the security authentication module is used for collecting authentication information of a user, sending the authentication information to the main control equipment and receiving security authentication results separated by the main control equipment; wherein the authentication information includes: fingerprint, facial feature information, password.
The technical solution of the present application is further described in detail by several preferred embodiments.
The first embodiment is as follows:
the modules of the main control device and the expansion device in the personal multi-device cooperation mode are shown in fig. 2. Wherein:
the main control device comprises: the system comprises a registration connection module, a screen projection module, a situation identification module, a safety certification module, an information processing module and a system setting module;
the expansion device comprises: the system comprises a registration connection module, a display module, an interface interaction module, an information processing/execution module, a security authentication module and a system setting module.
Referring to fig. 3, a main interaction flow of the multitask intelligent coordination method of the embodiment is described, where the interaction flow specifically includes:
when a user presses an icon of an application program for a long time or presses the icon in the application program for a long time, activating a split screen mode;
the main control equipment provides an equipment list for a user to select, and at the moment, the user can select one or more equipment to pair;
after the pairing is successful, the application program in the main control device enters background operation, and meanwhile, the application program is delivered to another device (namely, the expansion device) for display.
After the application program is delivered to the expansion device, the user can directly operate the application program on the expansion device. At the moment, whether the user is operated by one person or not needs to be judged, if the user is not operated by one person, the personal multi-device cooperation mode is not entered, otherwise, if the user is operated by one person, the personal multi-device cooperation mode is entered;
in the personal multi-device cooperation mode, resource cooperation output among multiple devices can record and analyze the operation of the user in the process, know the personalized operation habit of the user and feed back the personalized operation habit to the resource cooperation output among the multiple devices.
The following is a detailed description of the specific working principle and interaction mode of each module shown in fig. 2:
(1) registering a connection module:
the module utilizes wireless connection technology (such as Wi-Fi connection technology) to share information and data among devices in real time.
The module is simultaneously arranged at the main control equipment end and the expansion equipment end, and the multi-end equipment is always in a connection state through the module. After the equipment is connected, a safe transmission channel is opened for the data, so that the safety problem of personal operation information and even personal identity information in the transmission process is guaranteed.
After the main control device sends a pairing request to the expansion device, whether the expansion device logs in the same user account as the main control device is checked and confirmed. When the expansion device is found to be logged in the same account as the main control device, the multiple devices are considered to be used by the user personally. At this time, if the user requests connection for the first time, the expansion device prompts the user, completes pairing after detecting the confirmation of the user (for example, completes pairing when the user clicks "confirmation" in a dialog box popped up on the expansion device), and the main control device records the information of the expansion device, including the information of the device which can be identified, such as the serial number, the model number, the wireless local area network address, and the like of the device. When the multi-device connection is needed next time, the connected devices can be directly connected without going to the expansion device again for confirmation operation.
After the main control device sends a pairing request to the expansion device, if a user account number logged in by the expansion device is different from that of the main control device, the user needs to manually click on the expansion device for confirmation, and data synchronization is completed.
In this embodiment, a process of registering and connecting the master device is shown in fig. 4, and includes the following steps:
after the master control equipment is connected with the Wi-Fi, the registration connection module searches connectable equipment;
after the connectable equipment is searched, selecting target equipment for pairing and requesting data communication;
judging whether the pairing information is received or not, and if not, continuing to return to search for connectable equipment; and if the pairing information is received, carrying out data synchronization and completing connection.
In this embodiment, the connection process of the expansion device end is shown in fig. 5, and includes the following steps:
the device is connected with Wi-Fi;
when receiving the pairing request sent by other equipment, judging whether the user accepts the pairing request, if not, continuing to return to the step;
if the pairing request is accepted, whether the account logged in by the equipment is the same as the main control equipment or not is continuously judged;
if the account number logged in by the equipment is the same as that of the main control equipment, continuously judging whether the first connection is performed, if the first connection is performed, manually confirming a pairing request by a user, and then performing data synchronization to complete connection; if the connection is not the first connection, the other side equipment is directly connected, and data synchronization is carried out to complete connection;
if the account number logged in by the device is different from that of the main control device, the user is required to manually confirm the pairing request and then perform data synchronization to complete connection.
(2) Screen projection module (master control device side):
the module is applied to a personal multi-device cooperation mode, and when a user presses an app for a long time in the main control device or performs long-time pressing operation in the app, the split screen mode can be activated. Of course, the split screen mode may be activated in other possible ways, which is not limited by the invention. After activating the split screen mode, the screen projection module displays a list of extended devices connected under the same Wi-Fi, preferably, the devices in the list are arranged in the following order: the device of the user that has been connected, other devices that have been connected, and devices that have not been connected. After the user selects the device in the list, the user uses a screen-throwing operation gesture (such as a gesture sliding upwards in a screen-throwing activation mode) to pair with other devices, after authentication is completed, a screen-throwing module of the main control device enters a background to run, and meanwhile, the content needing screen division is thrown to a display module of the expansion device to be displayed.
If the user background is running the application A (such as WeChat), the user continues to use the steps to shoot the screen for a long time according to the application A on the desktop, and at the moment, the user is considered to want to use 2 different application accounts. After registration connection is carried out, the virtual machine for the screen projection module enters a background to run, and meanwhile, the application program A is displayed on the expansion equipment in a form without an account number. The user account information is not synchronized into the expansion device.
The specific implementation manner of the screen projection module may be: and starting a virtual machine process executed in the background at the main control device end, wherein the virtual machine process runs an app which needs to be executed in the screen projection by a user, and transmits the content in the virtual machine to the paired device for displaying. And the user data and the identity authentication information in the expansion equipment are sent to the running app in the virtual machine of the main control equipment, and the app executes corresponding operation, performs identity verification and the like.
The above specific implementation manner is only one implementation manner example of the present module, and in a specific application, other feasible implementation manners may be adopted, and are not limited to the above example implementation manner.
(3) A context identification module:
the module is applied to a personal multi-device cooperation mode. The following describes in detail a method for determining a collaboration mode according to comprehensive data of a user's attention to a device, a user's operation, a sound demand level of an application, a security requirement level of an application, and the like.
1) Screen brightness/refresh rate/resolution output collaboration:
conditional priority 1:
after entering the cooperative mode, the user is considered to be concentrated on a certain device of the multiple devices if the user operates the device. If the other equipment does not detect the control operation or gesture instruction of the user within the set time (for example, 5 minutes), the performances of the equipment, such as screen brightness, refresh rate, resolution and the like, are correspondingly reduced, so that the performance of the equipment is kept as it is.
When a user controls one of the devices (referred to as "controlled device" in the following table and other devices as "uncontrolled device") among the multiple devices, depending on the application program run by the device, the process may be performed, for example
The coordinated outputs shown in table 1:
Figure GDA0003320794460000131
TABLE 1
Conditional priority 2:
after entering the cooperative mode, if the user does not operate the device within a set time, and if a plurality of cameras are arranged at the front part of the device, the auxiliary camera is started to track the track of the eyeballs of the user, and the device which the user gazes at is determined. And calculating which equipment the user is paying attention to according to the eye movement, the pupil watching direction and the like shot by the 2 cameras. At this time, the coordination output shown in table 2 may be performed according to the user attention information.
Figure GDA0003320794460000132
Figure GDA0003320794460000141
TABLE 2
Conditional priority 3:
after entering the cooperative mode, if the user does not operate the device within a set time, at this time, if the device does not have a plurality of auxiliary cameras, the front main camera is started for tracking the eye gazing condition of the user, and performance cooperative output is performed on the attention device and the non-attention device according to the tracking result.
Conditional priority 4:
if the device does not have a camera or the user does not start the eyeball tracking service, the distance sensor of the device starts to function, if a user gesture operation is found within 20cm of the distance from the device, for example, a hand swings above the distance sensor of the expansion device, and the action exceeds the time required by judgment, for example, 0.3s, the user is considered to be focused on the expansion device, the application program executed by the expansion device is set as a focused application program, at the moment, if the other devices do not detect the control operation or gesture instruction of the user within the set time (for example, within 5 minutes), the device screen enters a low power consumption mode, the performance index is set as the performance index in the system configuration, and the performance of the expansion device is kept as the original state.
Conditional priority 5:
if the three conditions of the condition priorities 2-4 can not be met, the device starts the voice monitoring, and judges which device the user is concerned about by recognizing the voice information of the user. That is to say, if the user enters the collaboration mode, the front auxiliary camera, the front camera, and the distance sensor of the device cannot be enabled, the user can directly speak "i want to see the extension device", "light the main control device", and the like to the device through the voice command, and the collaboration mode is directly controlled by the voice command. Namely: and adjusting the performance index of the corresponding equipment according to the voice recognition result and then outputting the performance index.
2) Audio output collaboration:
according to the level of the sound output requirement of the application program, the application program installed in the main control equipment can be divided into strong audio requirement software, weak audio requirement software, no audio requirement software and dynamic audio requirement software. For example:
-the strong audio requirements software comprises: music software, broadcast software, video software, and the like. Such as: QQ music, dragonfly FM, Youke video, audio and video software, etc.
-the mute audio requirements software comprises: software that mainly shows text, only a small portion of audio, casual games, and the like. Such as: card games, and the like.
-the software without audio requirement comprises: character software and drawing software. Such as: notepads, drawing boards, etc.
-the dynamic audio requirements software comprises: the game system comprises an application program, social software, recording and video software and the like for large-scale hand games and with a real-time voice chat function.
After the application programs are classified according to the categories, the sound demand intensity of the application program can be re-judged according to whether the application program used by the user after entering the personal cooperation mode occupies a power amplifier of the mobile phone and the volume setting condition of the user and displayed in the setting items.
The collaboration mode may be adjusted as shown in table 3 according to different audio demand levels.
Figure GDA0003320794460000151
Figure GDA0003320794460000161
TABLE 3
Upon entering the personal collaboration mode, the system settings will be set to default values.
And then, when the user manually adjusts parameters such as volume, brightness and the like of certain equipment each time, the system records corresponding numerical values set by the current equipment and specific operation, and generates local user operation history. Through statistics of a large amount of operation histories, the volume setting average value s of the expansion equipment can be obtained from N times of setting results, the brightness setting average value is b, and the average value s and b are personalized setting values of the user, so that the system configuration can meet the requirements of the user.
(4) A security authentication module:
when a user uses an application with higher security level requirement in the expansion equipment, such as WeChat, Paibao, bank software and the like, if the user needs to perform operations such as payment and the like requiring identity verification, the expansion equipment acquires fingerprint or facial feature information or passwords and the like input by the current user, sends the information to the main control equipment for verification, and allows the payment operation to be continuously completed after the identity information is determined to be consistent with the information in the main control equipment.
(5) Display module/interface interaction module
After the devices are registered and connected, when the extension device receives the app interface information and the app data which are sent by the main control device and need to be displayed, the content transmitted by the main control device is displayed to the display module of the extension device. And after the content is displayed in the expansion equipment, the expansion equipment receives the user operation and records the user operation and the event and sends the recorded event to the main control equipment through other modules.
(6) Information processing/executing module
The module is a module for the expansion device side to execute a specific command sent by the main control device side, such as: and adapting various parameters configured by the main control equipment end, exiting the personal cooperation mode and the like. In addition, the module is also used for collecting the operation of the user in the app in the expansion device, setting and adjusting the volume of the current application by the user, judging whether the user pays attention to various composite information such as the expansion device and the like, and transmitting the collected information to the main control device for the main control device to perform relevant adjustment.
(7) A system setting module:
the system setting module at the master control device end is used for setting, so that whether the eye tracking function is performed by the system can be checked and modified, the volume of the expansion device can be modified, the sound priority level of each application can be modified, the low power consumption mode of the expansion device can be set, and the like. For example:
1) eye tracking function: the default of the item is a closing state, and when the item is in the closing state, if the user operation is not received for a long time, the system performs coordinated output on the main control equipment and the extension equipment according to the gesture and the voice command.
If the user sets, the function is started, after the cooperation mode is entered, when the user control operation is not received for a long time, the front-facing camera which starts the main control equipment and the expansion equipment tracks eyeballs of the user so as to judge the equipment which the user focuses on, and the cooperation output is carried out.
2) Volume of the extension device: the volume value of the expansion equipment obtained by learning the habit of the user can be set as the coordinated output default volume of the multiple equipment, the user can also directly adjust the volume of the multiple equipment, the equipment records each operation to form new user habit, and the system setting item is intelligently updated to the learning value. The device audio value set by the user entering the system setting module will be immediately embodied in a plurality of connected devices.
3) Application sound requirement level the requirement level of the application for sound can be viewed through the application sound requirement options of the system setting module (can be displayed in descending order of the requirement level or in different colors). The user cannot manually add items in the application, but only can determine the sound requirement intensity of the application according to the default items, whether the user occupies a power amplifier of the mobile phone when the user uses the expansion equipment to run a certain application, and the volume setting condition of the user.
4) And expanding the performance of the equipment, wherein after entering the cooperation mode, the initial value of the performance index of the expanded equipment is set by the system, and a user can enter the system to select manual adjustment. Specific indexes include screen brightness, screen resolution, screen refresh rate and the like of the expansion equipment.
Fig. 6 shows an example of the system setting module and the extended device setting interface in this embodiment.
Example two:
fig. 7 is a schematic view of a scene in an embodiment of the present application, in which a user wants to watch a yoga video as a yoga and watch a tv play while using a master control device. For this reason, the embodiment is specifically implemented as follows:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. When the main control device detects that the user presses the video app for a long time to trigger the split screen mode and slides the finger upwards, the main control device projects the content of the app to the expansion device and the expansion device plays the content.
3. And entering a personal cooperation mode, and when detecting that the user watches the video content on the main control equipment, combining the facial features and the watching condition of the user, keeping the volume of the main control equipment normal, and reducing/closing the sound of the expansion equipment.
4. The brightness of the expansion equipment is reduced, the resolution is reduced, the screen refreshing rate is changed to be a lower level, and the performance index of the main control equipment is kept normal.
5. When the fact that the user watches the video content in the expansion equipment is detected, the volume of the expansion equipment is recovered to be normal by combining the facial features and the watching condition of the user, the sound of the task of the main control equipment is reduced/closed, and the screen of the main control equipment enters a low power consumption mode.
Example three:
fig. 8 is a schematic view of a scenario of a third embodiment of the present application, in which a user uses a master control device to watch a tv series while viewing a WeChat content. For this reason, the embodiment is specifically implemented as follows:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. When the situation that the user presses the WeChat app for a long time to trigger the screen splitting mode and slides the finger upwards is detected, the main control device casts the content of the WeChat app to the expansion device and the content is played by the expansion device.
3. And entering a personal cooperation mode, and when detecting that a sound channel of equipment for controlling WeChat by a user is idle, outputting the sound channel of the equipment as a certain sound channel for playing a video, or outputting the sound channel of the equipment as a superposed sound channel for playing the video, so that the user can obtain better viewing experience.
4. When the user operation WeChat is detected, the screen of the expansion equipment is normally displayed, and the screen of the main control equipment enters a low power consumption mode.
Example four:
fig. 9 is a schematic view of a fourth scenario of an embodiment of the present application, in which a user uses a master control device to watch a tv series while viewing a WeChat content. For this reason, the embodiment is specifically implemented as follows:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. When detecting that the user presses the video app for a long time to trigger the split screen mode and slides a finger upwards, the main control device casts the content of the video app to the expansion device and the expansion device plays the content.
3. And entering a personal cooperation mode, and when detecting that a sound channel of equipment for controlling WeChat by a user is idle, outputting the sound channel of the equipment as a certain sound channel for playing a video, or outputting the sound channel of the equipment as a superposed sound channel for playing the video, so that the user can obtain better viewing experience.
4. When the condition that the user watches the video is detected, the screen of the expansion equipment is normally displayed, and the screen of the main control equipment enters a low power consumption mode.
Example five:
fig. 10 is a schematic view of a fifth scenario in an embodiment of the present application, in which a user uses a master control device to watch a tv play while viewing private content such as a WeChat/Paibao. For this reason, the embodiment is specifically implemented as follows:
1. and the user carries out registration pairing with the main control equipment by taking other equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. And when the condition that the user presses the WeChat/Payment treasure app for a long time to trigger the split screen mode and slides the finger upwards is detected, the WeChat/Payment treasure content is projected to the expansion equipment and displayed by the expansion equipment.
3. And entering a personal cooperation mode, and when detecting that a sound channel of equipment for controlling the WeChat/Payment treasure by the user is idle, outputting the sound channel of the equipment as a certain sound channel for playing a video, or outputting the sound channel of the equipment as a superposed sound channel for playing the video, so that the user obtains better viewing experience.
4. And the screen of the main control equipment enters a low power consumption mode, and the display of the expansion equipment is normal.
5. When the user operates himself, the payment or the transfer and other operations are carried out, and the payment operation can be completed through the fingerprint or the facial information.
Example six:
in this embodiment, the user uses the master control device to watch the tv play while viewing private contents such as WeChat/Paibao. The method specifically comprises the following steps:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. When the fact that the user presses the WeChat/Payment treasure app for a long time to trigger the split screen mode and slides the finger upwards is detected, the main control device casts the WeChat/Payment treasure content to the expansion device, and the expansion device displays the WeChat/Payment treasure content.
3. And entering a personal cooperation mode, and when detecting that a sound channel of equipment for controlling the WeChat/Payment treasure by the user is idle, outputting the sound channel of the equipment as a certain sound channel for playing a video, or outputting the sound channel of the equipment as a superposed sound channel for playing the video, so that the user obtains better viewing experience.
4. And the screen of the main control equipment enters a low power consumption mode, and the display of the expansion equipment is normal.
5. When the operation such as payment or transfer is performed by a non-user, the verification cannot be passed, and the payment operation fails.
Example seven:
fig. 11 is a scene schematic diagram of a fourth embodiment of the present application, in which a user wants to log in 2 different wechat accounts by using a master control device. For this reason, the embodiment is specifically implemented as follows:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. And the user opens the WeChat app in the main control equipment, puts the WeChat app into a background for running, then returns to the desktop again, and presses the WeChat app for a long time.
3. When the fact that the user presses the WeChat app for a long time to trigger the split screen mode and slides the finger upwards is detected, the main control device casts the WeChat content to the expansion device to be displayed.
4. And opening the WeChat application of the unregistered account in the expansion equipment, and normally logging in and using the expansion equipment by using other accounts.
Example seven:
in this embodiment, the user uses the main control device to view the video and refresh the web page. For this reason, the embodiment is specifically implemented as follows:
1. and the user takes out other equipment to perform registration pairing with the main control equipment, and the other equipment becomes the expansion equipment after the pairing is successful.
2. When detecting that the user presses the video app for a long time to trigger the split screen mode and slides the finger upwards, the main control device casts the video content to the expansion device for playing.
3. And entering a personal cooperation mode, and assuming that the expansion device has telephone access at the moment.
4. The telephone application is normally displayed above the screen of the expansion equipment, the video currently played by the expansion equipment is paused, and the user normally answers the call.
5. When the user hangs up the phone at the expansion equipment, the screen projection still resumes normal playing.
The invention can perform screen splitting on multiple devices in a cross-device manner, and performs coordinated output on the audio frequency, the brightness, the resolution, the screen refresh rate and the like of the multiple devices according to the attention degree of a user, the type of application, the sound requirement level of the application and the like without manual adjustment of the user. Meanwhile, recording is carried out when the user manually adjusts the related configuration of multiple tasks, and the personal habits of the user are learned, so that better personalized service experience is provided for the user.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims (11)

1. A multitask intelligent coordination method is characterized by comprising the following steps:
when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment; wherein the number of expansion devices is at least one;
the main control device sends the task selected by the user to the corresponding expansion device for displaying, the task is converted into a background to run, and the main control device and at least one expansion device enter a personal multi-device cooperation mode;
in the personal multi-device cooperation mode, the main control device comprehensively judges the use conditions of the main control device and the at least one expansion device by the user according to at least one of the classification of the application program, the sound requirement level of the application program and the safety requirement level of the application program, the attention condition and the operation condition of the user to the main control device and the at least one expansion device, and further outputs parameters to the main control device and the at least one expansion device: at least one of the volume, the screen brightness, the screen resolution and the screen refresh rate is output in harmony.
2. The method of claim 1, further comprising:
and recording when the user manually adjusts the output parameters of the main control equipment or the expansion equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
3. The method of claim 1, wherein the screen output parameters of the master device and the expansion device are coordinated in order of conditional priority as follows:
conditional priority 1: if the user only operates one device in the multiple devices within a set time, and other devices do not detect the control operation or gesture instruction of the user, the other devices reduce the respective screen brightness, refresh rate and/or resolution;
conditional priority 2: if the operation of the user on the equipment is not detected within the set time, if the equipment has a plurality of auxiliary cameras, the auxiliary cameras are started for tracking the track of eyeballs of the user, the equipment which the user is watching is determined, the screen brightness, the refresh rate and/or the resolution of the equipment which the user is watching is kept normal, and the screen brightness, the refresh rate and/or the resolution of the equipment which the user is not watching is reduced;
conditional priority 3: if the operation of the user on the equipment is not detected within the set time, if the equipment is not provided with a plurality of auxiliary cameras, the front main camera is started to track the track of eyeballs of the user, the equipment which the user is watching is determined, the screen brightness, refresh rate and/or resolution of the equipment which the user is watching is kept normal, and the screen brightness, refresh rate and/or resolution of the equipment which the user is not watching is reduced;
conditional priority 4: if the equipment does not have a camera or the user does not start the eyeball tracking service, determining the equipment which is currently concerned by the user through a distance sensor of the equipment, keeping the screen brightness, the refresh rate and/or the resolution of the equipment which is concerned by the user normal, and reducing the screen brightness, the refresh rate and/or the resolution of the equipment which is not concerned by the user;
conditional priority 5: and if the conditions of the condition priorities 2-4 cannot be met, the equipment starts voice monitoring and adjusts the output of the corresponding equipment according to the voice recognition result.
4. The method of claim 1, wherein the audio output parameters of the master device and the expansion device are coordinated as follows:
according to the level of the sound output requirement of the application program, dividing the application program installed in the main control equipment into strong audio requirement software, weak audio requirement software, no audio requirement software and dynamic audio requirement software;
and for the application programs with different sound output requirement levels, corresponding audio output parameters are adopted for audio output.
5. A multitask intelligent coordination system, comprising: a master device and at least one expansion device, wherein:
when the main control equipment detects that a user activates a split screen mode, providing an equipment list for the user to select;
the main control equipment is paired with the equipment selected by the user, and the successfully paired equipment becomes the expansion equipment;
the main control device sends the task selected by the user to the corresponding expansion device for displaying, the task is converted into a background to run, and the main control device and at least one expansion device enter a personal multi-device cooperation mode;
in the personal multi-device cooperation mode, the main control device comprehensively judges the use conditions of the main control device and the at least one expansion device by the user according to at least one of the classification of the application program, the sound requirement level of the application program and the safety requirement level of the application program, the attention condition and the operation condition of the user to the main control device and the at least one expansion device, and further outputs parameters to the main control device and the at least one expansion device: at least one of the volume, the screen brightness, the screen resolution and the screen refresh rate is output in harmony.
6. A master control device, comprising at least: register the connection module, throw screen module, situation identification module and system setting module, wherein:
the registration connection module is used for providing an equipment list for the user to select and pairing with the equipment selected by the user when the fact that the user activates the split screen mode is detected; the devices which are successfully paired become the expansion devices, and the number of the expansion devices is at least one;
the screen projection module is used for projecting the tasks selected by the user to the corresponding expansion equipment for display, converting the tasks into background operation, and enabling the main control equipment and the at least one expansion equipment to enter a personal multi-equipment cooperation mode;
the situation recognition module is configured to, in the personal multi-device cooperation mode, comprehensively judge the use conditions of the main control device and the at least one expansion device by the user according to at least one of the classification of the application program, the sound requirement level of the application program, and the security requirement level of the application program, and the attention conditions and the operation conditions of the main control device and the at least one expansion device by the user, and further output parameters of the main control device and the at least one expansion device: at least one of the volume, the screen brightness, the screen resolution and the screen refresh rate is output in a coordinated manner;
and the system setting module is used for setting the output parameters of the main control equipment.
7. The master device of claim 6, wherein the context identification module is further configured to:
and recording when the user manually adjusts the output parameters of the main control equipment each time, analyzing the personal habits of the user according to the records, and adjusting the output parameters of the main control equipment and the expansion equipment according to the analysis result.
8. The master device according to claim 6 or 7, further comprising a security authentication module:
the safety authentication module is used for receiving authentication information of a user from the expansion equipment, carrying out safety authentication on the authentication information and allowing the user to perform related operations on the expansion equipment after the safety authentication is passed; wherein the authentication information includes: fingerprint, facial feature information, password.
9. The master control device according to claim 6 or 7, further comprising an information processing module;
the information processing module is used for receiving operation information of a user in a task of projecting a screen to the expansion equipment from the expansion equipment and carrying out related adjustment according to the operation information.
10. An expansion device, comprising: register connection module, display module, interface interaction module, information processing execution module and system setting module, wherein:
the registration connection module is used for pairing with the master control equipment;
the information processing execution module is used for adapting various parameters configured by the main control device when the main control device and the expansion device enter a personal multi-device cooperation mode, processing the task delivered by the main control device, displaying the processed task through the display module, collecting the operation of the user in the task in the expansion device, and sending the operation to the main control device;
the interface interaction module is used for interacting with a user;
the system setting module is configured to receive, from the master control device, at least one of a classification of an application program, a sound requirement level of the application program, and a security requirement level of the application program, and a user's attention and operation conditions to the master control device and the at least one extension device, to comprehensively judge the user's use conditions of the master control device and the at least one extension device, and further output parameters to the master control device and the at least one extension device: and at least one of the volume, the screen brightness, the screen resolution and the screen refresh rate is subjected to coordinated output, and the output parameters of the expansion equipment are set according to the result.
11. The expansion device of claim 10, further comprising a security authentication module;
the security authentication module is used for collecting authentication information of a user, sending the authentication information to the main control equipment and receiving security authentication results separated by the main control equipment; wherein the authentication information includes: fingerprint, facial feature information, password.
CN201910711129.6A 2019-08-02 2019-08-02 Multitask intelligent coordination method, device and system Active CN110430323B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910711129.6A CN110430323B (en) 2019-08-02 2019-08-02 Multitask intelligent coordination method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910711129.6A CN110430323B (en) 2019-08-02 2019-08-02 Multitask intelligent coordination method, device and system

Publications (2)

Publication Number Publication Date
CN110430323A CN110430323A (en) 2019-11-08
CN110430323B true CN110430323B (en) 2022-02-08

Family

ID=68413853

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910711129.6A Active CN110430323B (en) 2019-08-02 2019-08-02 Multitask intelligent coordination method, device and system

Country Status (1)

Country Link
CN (1) CN110430323B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110995923B (en) * 2019-11-22 2021-08-20 维沃移动通信（杭州）有限公司 Screen projection control method and electronic equipment
CN111522619B (en) * 2020-05-03 2023-11-10 渴创技术(深圳)有限公司 Method for automatically reducing refresh frequency of extended screen based on software type and mouse pointer position
CN112118618B (en) * 2020-09-18 2024-01-26 上海闻泰电子科技有限公司 Screen refresh rate configuration method, device, computer equipment and readable storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107231489A (en) * 2017-07-06 2017-10-03 钛马信息网络技术有限公司 A kind of terminal and its throwing screen method
CN109660842A (en) * 2018-11-14 2019-04-19 华为技术有限公司 A kind of method and electronic equipment playing multi-medium data

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2843542A1 (en) * 2013-08-29 2015-03-04 Samsung Electronics Co., Ltd User terminal apparatus, method for controlling user terminal apparatus, and expanded display system
KR102480462B1 (en) * 2016-02-05 2022-12-23 삼성전자주식회사 Electronic device comprising multiple displays and method for controlling thereof
CN108021219A (en) * 2016-11-03 2018-05-11 华为技术有限公司 Control method of electronic device, device and electronic equipment
CN109413230A (en) * 2017-08-17 2019-03-01 富泰华工业（深圳）有限公司 Double-screen electronic device
CN108171171B (en) * 2017-12-29 2021-10-26 宁波大学 Self-adaptive adjusting method for mobile terminal screen
CN108513008A (en) * 2018-03-28 2018-09-07 努比亚技术有限公司 control right transfer method, mobile terminal and computer readable storage medium
CN108848268A (en) * 2018-06-27 2018-11-20 努比亚技术有限公司 Intelligent adjusting method, mobile terminal and the readable storage medium storing program for executing of screen intensity

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107231489A (en) * 2017-07-06 2017-10-03 钛马信息网络技术有限公司 A kind of terminal and its throwing screen method
CN109660842A (en) * 2018-11-14 2019-04-19 华为技术有限公司 A kind of method and electronic equipment playing multi-medium data

Also Published As

Publication number Publication date
CN110430323A (en) 2019-11-08

Similar Documents

Publication Publication Date Title
CN110430323B (en) Multitask intelligent coordination method, device and system
US10366691B2 (en) System and method for voice command context
US10477148B2 (en) Speaker anticipation
CN105957530B (en) Voice control method and device and terminal equipment
CN105204642B (en) The adjusting method and device of virtual reality interactive picture
CN104935980B (en) Interactive information processing method, client and service platform
CN111683263B (en) Live broadcast guiding method, device, equipment and computer readable storage medium
KR20110087025A (en) Video communication method and digital television thereof
CN110850593A (en) Mobile device and method for pairing electronic devices through mobile device
DE102014117344A1 (en) An apparatus and method for receiving an input and displaying in response an output on a second device different from the first device
EP1915665A2 (en) User interface and speech recognition for an electronic device
CN105637887A (en) Method in support of video impression analysis including interactive collection of computer user data
CN105260109A (en) Play speed adjusting method and terminal
CN112533068B (en) Live broadcast voting control method and device, electronic equipment and storage medium
CN114286173A (en) Display device and sound and picture parameter adjusting method
CN109089140A (en) A kind of sound control method and device
CN110444211A (en) A kind of audio recognition method and equipment
CN103269445B (en) Intelligent television system control method
CN114257824B (en) Live broadcast display method and device, storage medium and computer equipment
CN114363547A (en) Double-recording device and double-recording interaction control method
KR102460105B1 (en) Method for providing conference service and apparatus thereof
CN112243064B (en) Audio processing method and device
CN106775287A (en) Control the device and method of mobile terminal
CN110120933A (en) Air control, man-machine identification and data processing method, equipment and system
CN110275785A (en) Data processing method, device, client and server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant