CN107070981B

CN107070981B - Multi-terminal equipment cooperative control system and method

Info

Publication number: CN107070981B
Application number: CN201710054631.5A
Authority: CN
Inventors: 张立海; 申世安
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2017-01-23
Filing date: 2017-01-23
Publication date: 2020-02-04
Anticipated expiration: 2037-01-23
Also published as: CN107070981A

Abstract

The invention discloses a multi-terminal equipment cooperative control system, which comprises: the device comprises an application program layer, a control layer, a virtual device layer and a kernel driving layer. The invention also discloses a multi-terminal device cooperative control method. The invention realizes the separation of multi-terminal multi-device application service realization of upper application and bottom networking technology by introducing a multi-layer architecture and performing device management through the virtual device layer and the control layer, so that the hardware device resources in a local area network or a private cloud can achieve the cooperative sharing control of service levels.

Description

Multi-terminal equipment cooperative control system and method

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a system and a method for cooperative control of multiple terminals.

Background

With the advent of the mobile internet and the internet of everything, more and more intelligent terminals of different types can access the network. Meanwhile, many people own a plurality of terminal devices, and the terminals can be easily accessed to the Internet to enjoy network services. The development of wireless access such as Wifi and Bluetooth enables interconnection and intercommunication among multiple terminals to be more and more extensive, and people can carry out more and more services such as file transmission, screen sharing and the like in a local area network. However, these services are still mainly located at the level of connectivity and interworking, and the demand for deep cooperative control at the level of multi-terminal equipment is becoming strong, and no technology for meeting the demand exists in the prior art.

Disclosure of Invention

The invention mainly aims to provide a multi-terminal device cooperative control system and a multi-terminal device cooperative control method, and aims to solve the technical problem that the prior art cannot meet the requirements of deep cooperative control of a multi-terminal device level.

In order to achieve the above object, the present invention provides a multi-terminal device cooperative control system, including: the system comprises an application program layer, a control layer, a virtual device layer and a kernel driving layer;

the application program layer is used for responding to an operation instruction of triggering an application scene input by a user, acquiring the application scene and starting an application service according to the application scene;

the control layer is used for generating a service session corresponding to the application service when the application service is started, and sending information required by the service session to the virtual equipment layer; acquiring the service type of the service session, and determining a corresponding control type according to the service type; respectively sending control instructions to equipment corresponding to the equipment information of the service session according to the control types, and receiving and processing state feedback sent by the equipment;

the virtual device layer is used for acquiring the information required by the service session and sending a device scanning command to the kernel driver layer; receiving the equipment information fed back by the kernel driving layer, and carrying out capability matching on the equipment information according to the required requirements of the service session; taking the matching result as the equipment information of the service session, and storing the equipment information of the service session;

and the kernel driving layer is used for scanning the accessed mobile terminal when receiving the equipment scanning command and feeding back the scanned equipment information to the virtual equipment layer.

In one embodiment, the virtual device layer includes: a device discovery module and a group management module;

the device discovery module is used for acquiring the required information of the service session and sending a device scanning command to the kernel driver layer; receiving the equipment information fed back by the kernel driving layer, and carrying out capability matching on the equipment information according to the required requirements of the service session;

and the group management module is used for taking the matching result as the equipment information of the service session and storing the equipment information of the service session.

In one embodiment, the kernel driver layer is further configured to perform device scanning on an accessed mobile terminal, and send scanned device information to the device discovery module when the device information changes;

the device discovery module is further configured to receive device information sent by the kernel driver layer, and update information in the group management module according to the received device information.

In one embodiment, the control layer comprises: a service session module and a control engine;

the service session module is used for generating a service session corresponding to the application service when the application service is started;

the control engine is used for acquiring the service type of the service session, determining a corresponding control type according to the service type, and generating a control instruction according to the control type;

the service session module is further configured to send control instructions to devices corresponding to the device information of the service session, and receive state feedback sent by the devices;

and the control engine is also used for processing the state feedback.

In one embodiment, the control layer further comprises: a data synchronization module;

the data synchronization module is configured to acquire device information of the service session from the virtual device layer, and synchronize the acquired device information of the service session to the service session module.

In one embodiment, the application layer comprises: the system comprises an application scene management module and an application service module;

the application scene management module is used for responding to an operation instruction of triggering an application scene input by a user and acquiring the application scene;

and the application service module is used for starting application service according to the application scene.

In one embodiment, the application layer further comprises: a multi-terminal multi-user management module;

the multi-terminal multi-user management module is used for managing the access of the mobile terminal.

In one embodiment, the kernel driver layer includes: the device comprises a networking management module and a device driving module;

the networking management module is used for scanning the accessed mobile terminal when receiving the equipment scanning command and feeding back the scanned equipment information to the virtual equipment layer;

and the equipment driving module is used for providing driving for each equipment.

In one embodiment, the device information includes: the mobile terminal identification, the equipment identification and the equipment state.

In addition, in order to achieve the above object, the present invention further provides a multi-terminal device cooperative control method, wherein a multi-terminal device cooperative control system includes: the method comprises the following steps of:

the application program layer responds to an operation instruction of triggering an application scene input by a user, acquires the application scene and starts application service according to the application scene;

the control layer generates a service session corresponding to the application service when the application service is started; acquiring the service type of the service session, and determining a corresponding control type according to the service type;

the virtual device layer acquires the required information of the service session and sends a device scanning command to the kernel driving layer;

when receiving the device scanning command, the kernel driving layer scans the accessed mobile terminal and feeds back the scanned device information to the virtual device layer;

the virtual equipment layer receives the equipment information fed back by the kernel driving layer and performs capacity matching on the equipment information according to the required requirement of the service session; taking the matching result as the equipment information of the service session, and storing the equipment information of the service session;

and the control layer respectively sends control instructions to the equipment corresponding to the equipment information of the service session according to the control types, and receives and processes the state feedback sent by the equipment.

correspondingly, the virtual device layer obtains the information required by the service session, and sends a device scanning command to the kernel driver layer, including:

the device discovery module acquires the required information of the service session and sends a device scanning command to the kernel driver layer;

the virtual equipment layer receives the equipment information fed back by the kernel driving layer and performs capacity matching on the equipment information according to the required requirement of the service session; taking the matching result as the device information of the service session, and storing the device information of the service session, which specifically includes:

the device discovery module receives the device information fed back by the kernel driving layer and performs capability matching on the device information according to the required requirement of the service session;

and the group management module takes the matching result as the equipment information of the service session and stores the equipment information of the service session.

In an embodiment, after the kernel driver layer performs device scanning on the accessed mobile terminal and feeds back the scanned device information to the virtual device layer when receiving the device scanning command, the method further includes:

the kernel driving layer scans equipment of the accessed mobile terminal and sends the scanned equipment information to the equipment discovery module when the equipment information changes;

and the equipment discovery module receives the equipment information sent by the kernel driving layer and updates the information in the group management module according to the received equipment information.

the control layer generates a service session corresponding to the application service when the application service is started; obtaining a service type of the service session, and determining a corresponding control type according to the service type, specifically including:

the service session module generates a service session corresponding to the application service when the application service is started;

the control engine acquires the service type of the service session, determines a corresponding control type according to the service type, and generates a control instruction according to the control type;

correspondingly, the control layer respectively sends control instructions to the devices corresponding to the device information of the service session according to the control types, and receives and processes the state feedback sent by the devices, and the method specifically includes:

the service session module respectively sends control instructions to the equipment corresponding to the equipment information of the service session and receives state feedback sent by the equipment;

the control engine processes the state feedback.

before the service session module sends control instructions to the devices corresponding to the device information of the service session and receives the state feedback sent by the devices, the method further includes:

and the data synchronization module acquires the equipment information of the service session from the virtual equipment layer and synchronizes the acquired equipment information of the service session to the service session module.

the application program layer responds to an operation instruction of a user input triggering application scene, acquires the application scene, and starts application service according to the application scene, and the method specifically comprises the following steps:

the application scene management module responds to an operation instruction for triggering an application scene input by a user and acquires the application scene;

and the application service module starts application service according to the application scene.

when the kernel driver layer receives the device scanning command, the kernel driver layer performs device scanning on the accessed mobile terminal, and before feeding back the scanned device information to the virtual device layer, the method further includes:

and the multi-terminal multi-user management module manages the access of the mobile terminal.

when receiving the device scanning command, the kernel driver layer performs device scanning on the accessed mobile terminal, and feeds back the scanned device information to the virtual device layer, which specifically includes:

the equipment driving module provides driving for each equipment;

and when receiving the device scanning command, the networking management module performs device scanning on the accessed mobile terminal and feeds back the scanned device information to the virtual device layer.

The invention realizes the separation of multi-terminal multi-device application service realization of upper application and bottom networking technology by introducing a multi-layer architecture and performing device management through the virtual device layer and the control layer, so that the hardware device resources in a local area network or a private cloud can achieve the cooperative sharing control of service levels.

Drawings

Fig. 1 is a schematic diagram of a hardware structure of an alternative mobile terminal for implementing various embodiments of the present invention;

FIG. 2 is a diagram of a wireless communication system for the mobile terminal shown in FIG. 1;

FIG. 3 is a functional block diagram of a multi-terminal device cooperative control system according to a first embodiment of the present invention;

FIG. 4 is a diagram illustrating specific functional modules of the application layer in the embodiment shown in FIG. 3;

FIG. 5 is a block diagram illustrating specific functions of a control layer in the embodiment shown in FIG. 3;

FIG. 6 is a diagram illustrating specific functional modules of the virtual device layer in the embodiment shown in FIG. 3;

FIG. 7 is a data storage diagram of the group management module in the embodiment shown in FIG. 6;

FIG. 8 is a functional block diagram of a multi-terminal device cooperative control system according to a second embodiment of the present invention;

fig. 9 is a flowchart illustrating a cooperative control method for multiple terminals according to a first embodiment of the present invention;

fig. 10 is a flowchart illustrating a method for cooperative control of multiple terminals according to a second embodiment of the present invention;

fig. 11 is a flowchart illustrating a method for cooperative control of multiple terminals according to a third embodiment of the present invention;

fig. 12 is a flowchart illustrating a fourth embodiment of a method for cooperative control of multiple terminals according to the present invention;

fig. 13 is a flowchart illustrating a method for cooperative control of multiple terminals according to a fifth embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.

The mobile terminal may be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, it will be understood by those skilled in the art that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for moving purposes.

Fig. 1 is a schematic hardware configuration of a mobile terminal implementing various embodiments of the present invention.

The mobile terminal 100 may include a wireless communication unit 110, an a/V (audio/video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190, etc. Fig. 1 illustrates a mobile terminal having various components, but it is to be understood that not all illustrated components are required to be implemented. More or fewer components may alternatively be implemented. Elements of the mobile terminal will be described in detail below.

The wireless communication unit 110 typically includes one or more components that allow radio communication between the mobile terminal 100 and a wireless communication system or network. For example, the wireless communication unit may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short-range communication module 114, and a location information module 115.

The broadcast receiving module 111 receives a broadcast signal and/or broadcast associated information from an external broadcast management server via a broadcast channel. The broadcast channel may include a satellite channel and/or a terrestrial channel. The broadcast management server may be a server that generates and transmits a broadcast signal and/or broadcast associated information or a server that receives a previously generated broadcast signal and/or broadcast associated information and transmits it to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and the like. Also, the broadcast signal may further include a broadcast signal combined with a TV or radio broadcast signal. The broadcast associated information may also be provided via a mobile communication network, and in this case, the broadcast associated information may be received by the mobile communication module 112. The broadcast signal may exist in various forms, for example, it may be in a digital formAn Electronic Program Guide (EPG) of media broadcasting (DMB), an Electronic Service Guide (ESG) of digital video broadcasting-handheld (DVB-H), etc. The broadcast receiving module 111 may receive a signal broadcast by using various types of broadcasting systems. In particular, the broadcast receiving module 111 may receive a broadcast signal by using a signal such as multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), digital video broadcasting-handheld (DVB-H), forward link media (MediaFLO)^@) A digital broadcasting system of a terrestrial digital broadcasting integrated service (ISDB-T), etc. receives digital broadcasting. The broadcast receiving module 111 may be constructed to be suitable for various broadcasting systems that provide broadcast signals as well as the above-mentioned digital broadcasting systems. The broadcast signal and/or broadcast associated information received via the broadcast receiving module 111 may be stored in the memory 160 (or other type of storage medium).

The mobile communication module 112 transmits and/or receives radio signals to and/or from at least one of a base station (e.g., access point, node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received according to text and/or multimedia messages.

The wireless internet module 113 supports wireless internet access of the mobile terminal. The module may be internally or externally coupled to the terminal. The wireless internet access technology to which the module relates may include WLAN (wireless LAN) (Wi-Fi), Wibro (wireless broadband), Wimax (worldwide interoperability for microwave access), HSDPA (high speed downlink packet access), and the like.

The short-range communication module 114 is a module for supporting short-range communication. Some examples of short-range communication technologies include bluetooth^TMRadio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), zigbee^TMAnd so on.

The location information module 115 is a module for checking or acquiring location information of the mobile terminal. A typical example of the location information module is a GPS (global positioning system). According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites and applies triangulation to the calculated information, thereby accurately calculating three-dimensional current location information according to longitude, latitude, and altitude. Currently, a method for calculating position and time information uses three satellites and corrects an error of the calculated position and time information by using another satellite. In addition, the GPS module 115 can calculate speed information by continuously calculating current position information in real time.

The a/V input unit 120 is used to receive an audio or video signal. The a/V input unit 120 may include a camera 121 and a microphone 122, and the camera 121 processes image data of still pictures or video obtained by an image capturing apparatus in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 151. The image frames processed by the cameras 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and two or more cameras 121 may be provided according to the construction of the mobile terminal. The microphone 122 may receive sounds (audio data) via the microphone in a phone call mode, a recording mode, a voice recognition mode, or the like, and can process such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the mobile communication module 112 in case of a phone call mode. The microphone 122 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The user input unit 130 may generate key input data according to a command input by a user to control various operations of the mobile terminal. The user input unit 130 allows a user to input various types of information, and may include a keyboard, dome sheet, touch pad (e.g., a touch-sensitive member that detects changes in resistance, pressure, capacitance, and the like due to being touched), scroll wheel, joystick, and the like. In particular, when the touch pad is superimposed on the display unit 151 in the form of a layer, a touch screen may be formed.

The sensing unit 140 detects a current state of the mobile terminal 100 (e.g., an open or closed state of the mobile terminal 100), a position of the mobile terminal 100, presence or absence of contact (i.e., touch input) by a user with the mobile terminal 100, an orientation of the mobile terminal 100, acceleration or deceleration movement and direction of the mobile terminal 100, and the like, and generates a command or signal for controlling an operation of the mobile terminal 100. For example, when the mobile terminal 100 is implemented as a slide-type mobile phone, the sensing unit 140 may sense whether the slide-type phone is opened or closed. In addition, the sensing unit 140 can detect whether the power supply unit 190 supplies power or whether the interface unit 170 is coupled with an external device. The sensing unit 140 may include a proximity sensor 1410 as will be described below in connection with a touch screen.

The interface unit 170 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The identification module may store various information for authenticating a user using the mobile terminal 100 and may include a User Identity Module (UIM), a Subscriber Identity Module (SIM), a Universal Subscriber Identity Module (USIM), and the like. In addition, a device having an identification module (hereinafter, referred to as an "identification device") may take the form of a smart card, and thus, the identification device may be connected with the mobile terminal 100 via a port or other connection means. The interface unit 170 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal and the external device.

In addition, when the mobile terminal 100 is connected with an external cradle, the interface unit 170 may serve as a path through which power is supplied from the cradle to the mobile terminal 100 or may serve as a path through which various command signals input from the cradle are transmitted to the mobile terminal. Various command signals or power input from the cradle may be used as signals for recognizing whether the mobile terminal is accurately mounted on the cradle. The output unit 150 is configured to provide output signals (e.g., audio signals, video signals, alarm signals, vibration signals, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and the like.

The display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphical User Interface (GUI) related to a call or other communication (e.g., text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or an image and related functions, and the like.

Meanwhile, when the display unit 151 and the touch pad are overlapped with each other in the form of a layer to form a touch screen, the display unit 151 may serve as an input device and an output device. The display unit 151 may include at least one of a Liquid Crystal Display (LCD), a thin film transistor LCD (TFT-LCD), an Organic Light Emitting Diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as transparent displays, and a typical transparent display may be, for example, a TOLED (transparent organic light emitting diode) display or the like. Depending on the particular desired implementation, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown). The touch screen may be used to detect a touch input pressure as well as a touch input position and a touch input area.

The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 into an audio signal and output as sound when the mobile terminal is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output module 152 may provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 may provide an output to notify the mobile terminal 100 of the occurrence of an event. Typical events may include call reception, message reception, key signal input, touch input, and the like. In addition to audio or video output, the alarm unit 153 may provide output in different ways to notify the occurrence of an event. For example, the alarm unit 153 may provide an output in the form of vibration, and when a call, a message, or some other incoming communication (incomingmunication) is received, the alarm unit 153 may provide a tactile output (i.e., vibration) to inform the user thereof. By providing such a tactile output, the user can recognize the occurrence of various events even when the user's mobile phone is in the user's pocket. The alarm unit 153 may also provide an output notifying the occurrence of an event via the display unit 151 or the audio output module 152.

The memory 160 may store software programs and the like for processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a phonebook, messages, still images, videos, and the like) that has been or will be output. Also, the memory 160 may store data regarding various ways of vibration and audio signals output when a touch is applied to the touch screen.

The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. Also, the mobile terminal 100 may cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.

The controller 180 generally controls the overall operation of the mobile terminal. For example, the controller 180 performs control and processing related to voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 1810 for reproducing (or playing back) multimedia data, and the multimedia module 1810 may be constructed within the controller 180 or may be constructed separately from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.

The power supply unit 190 receives external power or internal power and provides appropriate power required to operate various elements and components under the control of the controller 180.

The various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, the embodiments described herein may be implemented using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, such embodiments may be implemented in the controller 180. For a software implementation, the implementation such as a process or a function may be implemented with a separate software module that allows performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in the memory 160 and executed by the controller 180.

Up to this point, mobile terminals have been described in terms of their functionality. Hereinafter, a slide-type mobile terminal among various types of mobile terminals, such as a folder-type, bar-type, swing-type, slide-type mobile terminal, and the like, will be described as an example for the sake of brevity. Accordingly, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.

The mobile terminal 100 as shown in fig. 1 may be configured to operate with communication systems such as wired and wireless communication systems and satellite-based communication systems that transmit data via frames or packets.

A communication system in which a mobile terminal according to the present invention is operable will now be described with reference to fig. 2.

Such communication systems may use different air interfaces and/or physical layers. For example, the air interface used by the communication system includes, for example, Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and Universal Mobile Telecommunications System (UMTS) (in particular, Long Term Evolution (LTE)), global system for mobile communications (GSM), and the like. By way of non-limiting example, the following description relates to a CDMA communication system, but such teachings are equally applicable to other types of systems.

Referring to fig. 2, the CDMA wireless communication system may include a plurality of mobile terminals 100, a plurality of Base Stations (BSs) 270, Base Station Controllers (BSCs) 275, and a Mobile Switching Center (MSC) 280. The MSC280 is configured to interface with a Public Switched Telephone Network (PSTN) 290. The MSC280 is also configured to interface with a BSC275, which may be coupled to the base station 270 via a backhaul. The backhaul may be constructed according to any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, frame Relay, HDSL, ADSL, or xDSL. It will be understood that a system as shown in fig. 2 may include multiple BSCs 275.

Each BS 270 may serve one or more sectors (or regions), each sector covered by a multi-directional antenna or an antenna pointing in a particular direction being radially distant from the BS 270. Alternatively, each partition may be covered by two or more antennas for diversity reception. Each BS 270 may be configured to support multiple frequency allocations, with each frequency allocation having a particular frequency spectrum (e.g., 1.25MHz,5MHz, etc.).

The intersection of partitions with frequency allocations may be referred to as a CDMA channel. The BS 270 may also be referred to as a Base Transceiver Subsystem (BTS) or other equivalent terminology. In such a case, the term "base station" may be used to generically refer to a single BSC275 and at least one BS 270. The base stations may also be referred to as "cells". Alternatively, each partition of a particular BS 270 may be referred to as a plurality of cell sites.

As shown in fig. 2, a Broadcast Transmitter (BT)295 transmits a broadcast signal to the mobile terminal 100 operating within the system. A broadcast receiving module 111 as shown in fig. 1 is provided at the mobile terminal 100 to receive a broadcast signal transmitted by the BT 295. In fig. 2, several Global Positioning System (GPS) satellites 300 are shown. The satellite 300 assists in locating at least one of the plurality of mobile terminals 100.

In fig. 2, a plurality of satellites 300 are depicted, but it is understood that useful positioning information may be obtained with any number of satellites. The GPS module 115 as shown in fig. 1 is generally configured to cooperate with satellites 300 to obtain desired positioning information. Other techniques that can track the location of the mobile terminal may be used instead of or in addition to GPS tracking techniques. In addition, at least one GPS satellite 300 may selectively or additionally process satellite DMB transmission.

As a typical operation of the wireless communication system, the BS 270 receives reverse link signals from various mobile terminals 100. The mobile terminal 100 is generally engaged in conversations, messaging, and other types of communications. Each reverse link signal received by a particular base station 270 is processed within the particular BS 270. The obtained data is forwarded to the associated BSC 275. The BSC provides call resource allocation and mobility management functions including coordination of soft handoff procedures between BSs 270. The BSCs 275 also route the received data to the MSC280, which provides additional routing services for interfacing with the PSTN 290. Similarly, the PSTN 290 interfaces with the MSC280, the MSC interfaces with the BSCs 275, and the BSCs 275 accordingly control the BS 270 to transmit forward link signals to the mobile terminal 100.

Based on the above mobile terminal hardware structure and communication system, various embodiments of the present invention are proposed.

As shown in fig. 3, a first embodiment of the present invention provides a multi-terminal device cooperative control system, where the system includes: an application layer 10, a control layer 20, a virtual device layer 30 and a kernel driver layer 40;

the application program layer 10 is configured to respond to an operation instruction for triggering an application scene input by a user, acquire the application scene, and start an application service according to the application scene;

it should be noted that the system of the present embodiment is deployed on a local area network server, or a cloud server of a private cloud, and certainly, may also be deployed on other servers with similar functions, which is not limited in this embodiment.

It can be understood that, because the system of this embodiment adopts a multi-layer architecture, and each layer in the architecture is implemented by software, the server needs to be based on an open source operating system such as Android.

To facilitate the functions of the application layer 10, referring to fig. 4, the application layer 10 may include: an application scene management module 101 and an application servitization module 102;

the application scene management module 101 is configured to respond to an operation instruction for triggering an application scene input by a user, and acquire the application scene;

understandably, the application scenarios may include: a communication scene, a multimedia scene, a file access scene, a power management scene, an input/output scene, a calculation scene and the like;

communication scene: i.e. to cooperatively control the communication functions of multiple devices. For example, the system can remotely answer calls from other equipment and transfer the audio to local equipment. A multi-party conference call or group chat service within a local area network may be established.

Multimedia scene: i.e. multimedia related application services. For example, multi-terminal media is played cooperatively, a stereo sound system and a video broadcast service can be constructed, and real-time image tracking and the like can be realized.

File access scenario: i.e. file access and storage related application services. Such as file joint access of a master device and a slave device, file transfer and file search services.

Power management scenario: i.e. service management related to the combination of power supply status of multiple devices and application services. Coordinated control of applications may be performed based on low power service and state of charge.

Input-output scenario: i.e. control of the co-operative input and output functions of multiple devices. The general input and output system can be made to be no longer limited to the local machine.

Calculating a scene: namely distributed cooperative calculation and summarization of multiple terminals in the local area network.

Of course, other extensible service scenarios may also be included, such as cooperative sensing of multiple terminals, service customization notification system, and power saving associated with standby, which is not limited in this embodiment.

The application servitization module 102 is configured to start an application service according to the application scenario.

In a specific implementation, the application servitization module 102 may serve each application scenario. The application service after serving the application scenario may provide services in multiple modes, such as a subscription and distribution mode, or a command response mode, and of course, other modes may also be used, which is not limited in this embodiment.

The application servitization module 102 needs to select roles, that is, role selection needs to be performed according to roles assumed in the application services, for example: service providers and service users, or alternatively, command requesters and command responders.

The application servitization module 102 also relates to terminal selection and device selection, and the main function thereof is to select and associate visible terminals and devices thereof related to application services.

Since not all mobile terminals can access the server, in order to control the accessed mobile terminals, in this embodiment, the application layer further includes: a multi-terminal multi-user management module 103;

the multi-terminal multi-user management module 103 is configured to manage access of the mobile terminal.

The multi-terminal multi-user management module 103 generally relates to multi-user management, multi-terminal management and rights management, and in order to manage these objects, the multi-terminal multi-user management module 103 needs to store a terminal identifier and a user identifier.

The terminal identifier is generally marked by an International Mobile Equipment Identity (IMEI), a Media Access Control (MAC) address or an IP address. Maintenance may cause synchronization problems in view of the temporal nature of the IP allocation. The MAC address and IMEI number are preferred as terminal identities.

The user identification is a globally unique email address, but of course, other unique identifications are possible.

For multi-user management, a terminal supports multiple user logins, and only one user can be enabled to be activated to operate at a time.

For multi-terminal management, one user may log on to multiple terminals. A plurality of terminals are simultaneously logged in by a user and belong to a user group. Making the synergy strongest.

Of course, there may be a case of login binding, that is, the default login status is the user and device bound, and the default login user account is set to have a general permission.

For authority management, the same user login adopts a master-slave structure, one master has multiple slaves, master-slave role switching is supported, and a master mobile terminal has super authority.

The control layer 20 is configured to generate a service session corresponding to the application service when the application service is started, and send information required by the service session to the virtual device layer; acquiring the service type of the service session, and determining a corresponding control type according to the service type; respectively sending control instructions to equipment corresponding to the equipment information of the service session according to the control types, and receiving and processing state feedback sent by the equipment;

in order to facilitate the function of the control layer 20, referring to fig. 5, the control layer 20 includes: a service session module 201 and a control engine 202;

the service session module 201 is configured to generate a service session corresponding to the application service when the application service is started;

the control engine 202 is configured to obtain a service type of the service session, determine a corresponding control type according to the service type, and generate a control instruction according to the control type;

in a specific implementation, common service types include: the subscription publishing service and the command request response service may also include other service types, which is not limited in this embodiment.

It is understood that the control types may include a master-slave control mode and a peer-to-peer distributed mode, and of course, other control types may be included, which is not limited in this embodiment.

The service session module 201 is further configured to send control instructions to the devices corresponding to the device information of the service session, and receive state feedback sent by the devices;

the control engine 202 is further configured to process the state feedback.

It should be noted that, in order to ensure the timeliness of the device information of the service session in the service session module 201, in this embodiment, the control layer further includes: a data synchronization module 203;

the data synchronization module 203 is configured to acquire the device information of the service session from the virtual device layer, and synchronize the acquired device information of the service session to the service session module 201.

The virtual device layer 30 is configured to obtain information required by the service session, and send a device scanning command to the kernel driver layer; receiving the equipment information fed back by the kernel driving layer, and carrying out capability matching on the equipment information according to the required requirements of the service session; taking the matching result as the equipment information of the service session, and storing the equipment information of the service session;

in a specific implementation, the device information includes: the device information includes an identifier of the mobile terminal (i.e. an identifier of the mobile terminal to which the device corresponding to the device information belongs), a device identifier (i.e. a device ID), and a device status.

It is understood that all devices in a mobile terminal identity may be understood as a group, and the devices in the group are group devices.

It should be noted that the virtual device layer 30 is a key layer of the entire system architecture. The method mainly aims to perform virtualization logical processing on equipment in a local area network or a private cloud, so that terminals and equipment in all local area networks are cooperatively controlled from the service perspective. And carrying out grouping management on the related terminals and devices in the local area network. And the devices in the group are allocated with a device ID and receive the unified cooperative control of the upper control framework. The advantage of this implementation is that the upper layer application does not care about the underlying communication connection technology, and considers the devices in the group as local devices for scheduling control.

To facilitate the functions of the virtual device layer 30, referring to fig. 6, the virtual device layer 30 includes: a device discovery module 301 and a group management module 302;

the device discovery module 301 is configured to obtain information required by the service session, and send a device scanning command to the kernel driver layer; receiving the equipment information fed back by the kernel driving layer, and carrying out capability matching on the equipment information according to the required requirements of the service session;

the group management module 302 is configured to use the matching result as the device information of the service session, and store the device information of the service session.

It can be understood that, the kernel driver layer 40 not only performs device scanning on the accessed mobile terminal when receiving the device scanning command, but also actively performs device scanning on the accessed mobile terminal, so that the scanned device information can be sent to the device discovery module 301 when the device information changes, and accordingly, the device discovery module 301 can update the information in the group management module according to the received device information.

Of course, in order to perform device management more securely, the device discovery module 301 may perform device authentication after performing capability matching on the device information according to the required requirement of the service session.

It should be noted that the group management module 302 may perform group management through the matrix shown in fig. 7, and of course, the device state machine corresponds to the device state and is derived from the information update of the device discovery module 301, in general, N in the figure may be plural, and in special cases, N may be 1.

The kernel driver layer 40 is configured to perform device scanning on the accessed mobile terminal when receiving the device scanning command, and feed back scanned device information to the virtual device layer.

In this embodiment, a multi-layer architecture is introduced, and device management is performed through a virtual device layer and a control layer, so that separation of multi-terminal multi-device application service implementation of upper-layer application and a bottom-layer networking technology is achieved, and hardware device resources in a local area network or a private cloud can achieve cooperative sharing control of a service level.

Further, referring to fig. 8, a second embodiment of the present invention provides a device cooperation control apparatus with multiple terminals, based on the embodiment shown in fig. 3.

In this embodiment, the kernel driver layer 40 includes: a networking management module 401 and a device driving module 402;

the networking management module 401 is configured to perform device scanning on the accessed mobile terminal when receiving the device scanning command, and feed back scanned device information to the virtual device layer 30;

in a specific implementation, the networking management module 401 may involve technologies including: wireless local area network access technology, local area network networking type selection and networking state maintenance.

For the wireless local area network access technology, wifi and bluetooth are the most commonly used, and the two technologies are the most common in hardware support, and the current intelligent terminal can provide basic support. In consideration of the bandwidth delay and the limitation factor of the number of connections, wifi can meet the requirements of most scenes.

The following three types of network types are usually selected for the lan: distributed fixed-centric network types, distributed non-fixed-centric network types, and peer-to-peer network types. The distributed fixed center network type is more suitable for matching with a master control type and a multi-slave control type, so the distributed fixed center network type can be adopted in the embodiment.

For the networking state maintenance, the functions include network establishment, naming, member state refreshing, link maintenance and the like.

The device driver module 402 is configured to provide drivers for the devices.

In order to ensure that each device can be identified, so as to implement device scanning on the accessed mobile terminal, in this embodiment, the device driving module 402 is configured to provide a drive for each device, so as to ensure that each device can be identified, and further, can scan and obtain device information of each device.

As shown in fig. 9, a first embodiment of the present invention provides a method for cooperative control of multiple terminals, where a system for cooperative control of multiple terminals includes: the method comprises the following steps of applying a program layer, a control layer, a virtual device layer and a kernel driving layer, wherein the method comprises the following steps:

s10: the application program layer responds to an operation instruction of triggering an application scene input by a user, acquires the application scene and starts application service according to the application scene;

the system of this embodiment is deployed on a local area network server, or a cloud server of a private cloud, and certainly, may also be deployed on other servers with similar functions, which is not limited in this embodiment.

S20: the control layer generates a service session corresponding to the application service when the application service is started; acquiring the service type of the service session, and determining a corresponding control type according to the service type;

s30: the virtual device layer acquires the required information of the service session and sends a device scanning command to the kernel driving layer;

s40: when receiving the device scanning command, the kernel driving layer scans the accessed mobile terminal and feeds back the scanned device information to the virtual device layer;

s50: the virtual equipment layer receives the equipment information fed back by the kernel driving layer and performs capacity matching on the equipment information according to the required requirement of the service session; taking the matching result as the equipment information of the service session, and storing the equipment information of the service session;

s60: and the control layer respectively sends control instructions to the equipment corresponding to the equipment information of the service session according to the control types, and receives and processes the state feedback sent by the equipment.

Further, referring to fig. 10, a second embodiment of the present invention provides a method for cooperative control of multiple terminals, based on the embodiment shown in fig. 9.

In this embodiment, in order to implement the functions of the application layer, the application layer may include: the system comprises an application scene management module and an application service module;

correspondingly, the step S10 may specifically include:

s101: the application scene management module responds to an operation instruction for triggering an application scene input by a user and acquires the application scene;

S102: and the application service module starts application service according to the application scene.

In a specific implementation, the application servitization module may serve each application scenario. The application service after serving the application scenario may provide services in multiple modes, such as a subscription and distribution mode, or a command response mode, and of course, other modes may also be used, which is not limited in this embodiment.

The application service module needs to select roles, that is, the roles need to be selected according to roles assumed in the application service, for example: service providers and service users, or alternatively, command requesters and command responders.

The application servitization module also relates to terminal selection and equipment selection, and the main function of the application servitization module is to select and associate visible terminals and equipment related to application services.

Since not all mobile terminals can access the server, in order to control the accessed mobile terminals, the application layer further includes: a multi-terminal multi-user management module;

prior to step S40, the method may further include:

Further, referring to fig. 11, a third embodiment of the present invention provides a method for cooperative control of multiple terminals, based on the embodiment shown in fig. 9.

In this embodiment, in order to realize the function of the control layer, the control layer includes: a service session module and a control engine;

correspondingly, step S20 specifically includes:

s201: the service session module generates a service session corresponding to the application service when the application service is started;

s202: the control engine acquires the service type of the service session, determines a corresponding control type according to the service type, and generates a control instruction according to the control type;

Correspondingly, the step S60 specifically includes:

s601: the service session module respectively sends control instructions to the equipment corresponding to the equipment information of the service session and receives state feedback sent by the equipment;

s602: the control engine processes the state feedback.

It should be noted that, in order to ensure the timeliness of the device information of the service session in the service session module 201, in this embodiment, the control layer further includes: a data synchronization module;

accordingly, before step S601, the method may further include:

s600: and the data synchronization module acquires the equipment information of the service session from the virtual equipment layer and synchronizes the acquired equipment information of the service session to the service session module.

Further, referring to fig. 12, a fourth embodiment of the present invention provides a method for device cooperative control of multiple terminals, based on the embodiment shown in fig. 9.

In this embodiment, in order to implement the function of the virtual device layer, the virtual device layer includes: a device discovery module and a group management module;

correspondingly, the step S30 specifically includes:

s301: the device discovery module acquires the required information of the service session and sends a device scanning command to the kernel driver layer;

step S50, specifically including:

s501: the device discovery module receives the device information fed back by the kernel driving layer and performs capability matching on the device information according to the required requirement of the service session;

s502: and the group management module takes the matching result as the equipment information of the service session and stores the equipment information of the service session.

The device information includes: the device information includes an identifier of the mobile terminal (i.e. an identifier of the mobile terminal to which the device corresponding to the device information belongs), a device identifier (i.e. a device ID), and a device status.

It can be understood that, the kernel driver layer may actively scan the accessed mobile terminal for the device in addition to the device scanning for the accessed mobile terminal when receiving the device scanning command, so that the scanned device information may be sent to the device discovery module when the device information changes, and accordingly, after step S40, the method may further include:

Of course, in order to perform device management more securely, in step S501, the device discovery module performs device authentication after performing capability matching on the device information according to the required requirement of the service session.

Further, referring to fig. 13, a fifth embodiment of the present invention provides a method for cooperative control of multiple terminals, based on the embodiment shown in fig. 9.

In this embodiment, the kernel driver layer includes: the device comprises a networking management module and a device driving module;

correspondingly, the step S40 specifically includes:

s401: the equipment driving module provides driving for each equipment;

in order to ensure that each device can be identified, so as to implement device scanning on the accessed mobile terminal, in this embodiment, the device driving module provides a drive for each device, so as to ensure that each device can be identified, and further, the device information of each device can be obtained through scanning.

S402: and when receiving the device scanning command, the networking management module performs device scanning on the accessed mobile terminal and feeds back the scanned device information to the virtual device layer.

In a specific implementation, the networking management module may involve techniques including: wireless local area network access technology, local area network networking type selection and networking state maintenance.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A multi-terminal device cooperative control system, comprising: the system comprises an application program layer, a control layer, a virtual device layer and a kernel driving layer;

the control layer is used for generating a service session corresponding to the application service when the application service is started, and sending information required by the service session to the virtual equipment layer; acquiring the service type of the service session, and determining a corresponding control type according to the service type; acquiring the equipment information of the service session from the virtual equipment layer, and synchronizing the acquired equipment information of the service session to the local control layer; respectively sending control instructions to equipment corresponding to the equipment information of the service session according to the control types, and receiving and processing state feedback sent by the equipment;

2. The system of claim 1, wherein the virtual device layer comprises: a device discovery module and a group management module;

3. The system of claim 2, wherein the kernel driver layer is further configured to perform device scanning on the accessed mobile terminal, and send the scanned device information to the device discovery module when the device information changes;

4. The system of claim 1, wherein the control layer comprises: a service session module and a control engine;

and the control engine is also used for processing the state feedback.

5. The system of claim 4, wherein the control layer further comprises: a data synchronization module;

6. A multi-terminal device cooperative control method is characterized in that a multi-terminal device cooperative control system comprises: the method comprises the following steps of:

the control layer acquires the equipment information of the service session from the virtual equipment layer and synchronizes the acquired equipment information of the service session to the local control layer;

7. The method of claim 6, wherein the virtual device layer comprises: a device discovery module and a group management module;

8. The method of claim 7, wherein after the kernel driver layer performs device scanning on the accessed mobile terminal and feeds back scanned device information to the virtual device layer when receiving the device scanning command, the method further comprises:

9. The method of claim 6, wherein the control layer comprises: a service session module and a control engine;

the control engine processes the state feedback.

10. The method of claim 9, wherein the control layer further comprises: a data synchronization module;