CN114731302B

CN114731302B - Information transmission method and related equipment

Info

Publication number: CN114731302B
Application number: CN202080081299.1A
Authority: CN
Inventors: 主父文刚; 唐治宇; 陈建; 姚垚; 宋党辉
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-01-23
Filing date: 2020-01-23
Publication date: 2023-06-06
Anticipated expiration: 2040-01-23
Also published as: WO2021147109A1; CN114731302A

Abstract

An information transmission method, comprising: the method comprises the steps that calling equipment obtains capability information of each service equipment in at least one service equipment based on a first command, wherein the at least one service equipment comprises first service equipment; under the condition that the calling device determines that the acquired capability information of the first service device comprises a target Artificial Intelligence (AI) function, the calling device establishes a connection channel with the first service device based on a second command, wherein the target AI function is an AI function which the calling device hopes to remotely call; the calling device remotely calls the target AI function of the first service device based on a third command; and the calling device receives result data sent by the first service device, wherein the result data comprises data generated after the first service device operates a target AI function according to the third command.

Description

Information transmission method and related equipment

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular, to an information transmission method and related devices.

Background

With the development of chip technology, the processing capability of proprietary chips with specific functions in their respective fields is becoming stronger, and with the technology maturing, the cost of developing proprietary chips is also gradually decreasing, so that manufacturers can implement, when pushing out specific services for existing main products, by adding an accessory device with a proprietary chip, and taking domestic television manufacturers as an example, manufacturers can introduce video acquisition functions for televisions by adding camera devices, and cameras may have various capabilities as accessory devices, such as video acquisition functions, basic codec capabilities, and some artificial intelligence (artificial intelligence, AI) capabilities.

One host product may be equipped with multiple accessory devices, and one accessory device may have multiple functions, and there is no effective and standardized solution for how to share the multiple different specific functions of the accessory device.

Disclosure of Invention

In view of the above technical problems, embodiments of the present application provide an information transmission method, so that data communication is performed between a control device and a calling device.

An embodiment of the present application provides an information transmission method, including:

the calling device obtains capability information of the service device, the calling device can inquire the capability information of each service device which can perform data communication with the calling device, the service device comprises first service devices, the capability information of the first service devices comprises target AI functions, the target AI functions are AI functions which the calling device hopes to remotely call, the calling device obtains the capability information of the service device based on a first command, when the calling device determines that the obtained capability information of the first service devices comprises the target AI functions, the calling device can establish a connection channel with the first service device based on a second command, the calling device remotely calls the target AI functions of the first service devices based on a third command, and receives result data fed back by the first service devices, wherein the result data comprise data generated after the first service devices operate the target AI functions according to the third command;

In the embodiment of the application, an information transmission method is provided, and a device without a target AI function, namely a calling device, can access capability information of other devices in a related network through the information transmission method provided by the embodiment of the application, and establish a connection channel with the device with the target AI function based on the information transmission method provided by the embodiment of the application, so as to realize remote calling of the target AI function. The calling device may be a sweeping robot, where the sweeping robot does not have an AI function such as a sweeping route calculation function, the corresponding target AI function is a sweeping route calculation function, and the sweeping robot may sense the AI function of other devices associated with the sweeping robot based on the information transmission method provided by the application, such as the sweeping route calculation function of the home intelligent computing center, if the sweeping robot needs to call the sweeping route calculation function, the sweeping robot obtains capability information of the home intelligent computing center based on the first command, establishes communication connection with the home intelligent computing center based on the second command on the premise that the home intelligent computing center is determined to have the target AI function, and then remotely calls the sweeping route calculation function of the home intelligent computing center based on the third command, so that the sweeping robot obtains a planned driving route.

The information transmission method provided by the embodiment of the application can use a lightweight protocol architecture, can realize flexible intercommunication of the capabilities among the devices, and realizes capability coordination among the devices of different platforms. And can expand multiple AI business for the equipment that does not possess the AI function, with low costs, the flexibility is high.

In one possible implementation manner, the method for transmitting information provided by the application can also realize dynamic update of the AI capability model of the accessory service device.

In one possible implementation manner, firmware upgrade can be further performed on the accessory service device based on the information transmission method provided by the application.

In a possible implementation manner, the information transmission method provided by the application provides a command channel and two data channels, wherein the command channel can transmit various customized control commands, and one data channel is a channel for transmitting data to the service equipment by the calling equipment and can be used for transmitting firmware upgrading data and capability upgrading data and transmitting the data of the calling equipment to the service equipment; the other channel is a channel for the service device to send data to the calling device, and can be used for returning the result data of remote capability call.

In one possible embodiment, the same calling device may have multiple different calling services that may call the same AI function or multiple different AI functions of the serving device at the same time; multiple calling devices may call the AI function of the same service device at the same time.

In a possible implementation manner, the information transmission method provided by the application is suitable for IOT devices with insufficient resources, such as IOT devices with limited memory, resources, processing capacity and the like, such as intelligent doorbell.

According to a first implementation manner of the first aspect of the embodiments of the present application, before the calling device obtains the capability information of the service device based on the first command, the calling device may receive the device information multicast by the service device through a simple service discovery protocol (simple service discovery protocol, SSDP) to obtain a service device list, where the service device list stores the device information of each service device, and the device information of the service device includes a device name, an internet protocol (internet protocol, IP) address, and a port number of the service device.

In the embodiment of the application, the calling device may receive the device information of each service device, determine the service device that the calling device may call according to the device information, and if the device name in the device information has an association relationship with the target AI function in an achievable case, if the target AI function is face recognition, the calling device may select the service device associated with the device name and the image capturing device to obtain the capability information thereof. In a second implementation manner of the first aspect of the present embodiment, according to the first implementation manner of the first aspect of the present embodiment, the first command may be a capability information acquisition command, such as a restricted application protocol (constrained application protocol, COAP), the capability information acquisition command may include an IP address, a port number, and an information acquisition command field of a service device, where the information acquisition command field is used to indicate that capability information of the service device is acquired, and a specific implementation manner of acquiring, by a calling device, the capability information of the service device based on the first command may include: the calling device sends a capability information acquisition command to each service device with a device name meeting the requirement in the service device list, the calling device receives capability information returned by the service device in response to the capability information acquisition command, the capability information indicates the AI function of the service device, the capability information can include a capability name, namely the name of the AI function supported by the service device, such as face recognition or skeleton detection, and the capability information can also include a transmission mode, output data length, and the like through a lightweight data exchange format, such as (c java script object notation, cSSON) based on c language.

In a third implementation manner of the first aspect of the present application, the second command may be a connection establishment command of the COAP, where the connection establishment command includes a service device IP address, a port number, a connection establishment command field, and a load variable, where the load variable includes a data channel type, the service device IP address, and a data channel port, the calling device establishes a connection channel with the first service device based on the second command, where the connection channel includes a first data channel and a second data channel, the first data channel is used by the calling device to send data to the first service device, the second data channel is used by the calling device to receive result data returned by the first service device, the data channel port included in the load variable is used to indicate a port of the first data channel, and the port of the second data channel may be determined according to the first data channel, for example, by adding one port to the first data port to obtain the second data channel.

In this embodiment of the present application, the connection channel established by the calling device based on the second command and the first service device may include two channels in opposite directions, and a bidirectional connection channel is formed between the calling device and the service device, so as to implement bidirectional data transmission between the calling device and the service device.

In a fourth implementation manner of the first aspect of the embodiments of the present application, the third command may be a capability call command, where the capability call command includes a caller service identification number (identity, ID), a capability call command field, and a subscription information packet, where the subscription information packet includes a called capability name, where the called capability name is a function name of a target AI function, and the calling device may include one or more caller services. The calling device may specifically send the capability calling command of each caller service to the first service device, so that the first service device may operate the corresponding target AI function according to the called capability names included in the plurality of capability calling commands, respectively.

It can be understood that when only one caller service in the calling device calls the target AI function of the first service device, the calling device sends a capability calling command of the caller service to the first service device, and when the calling device comprises two or more caller services to call the target AI function of the first service device, the calling device sends the capability calling command of each caller service to the first service device respectively, wherein the caller service and the capability calling command are in a mapping relationship and are in one-to-one correspondence. Similarly, the calling device receives the result data sent by the first service device, and specifically, the plurality of caller services of the calling device respectively receive the result data obtained by processing the target AI functions corresponding to the plurality of caller services in the first service device.

According to a fifth implementation of the first aspect of the embodiments of the present application, in a fifth implementation of the first aspect of the embodiments of the present application, the subscription information packet further includes a data source identifier, where the data source identifier is used to indicate that data to be processed by the target AI function of the first service device is from the first service device or the remote device, and it may be understood that the remote device includes the calling device.

In the embodiment of the application, after receiving the data source identifier, the service device may determine the source of the data to be processed, and if the data is from the remote device, prepare for receiving the data to be processed.

According to a fifth implementation of the first aspect of the embodiments of the present application, in a sixth implementation of the first aspect of the embodiments of the present application, when the data to be processed indicated by the data source identifier is data from the calling device, the calling device may send the data to be processed to the first service device through the first data channel.

In the embodiment of the application, when the data to be processed comes from the calling device, the calling device transmits the data to be processed to the first service device through the connection channel, so that the feasibility of the scheme is improved.

According to a seventh implementation manner of the first aspect of the present application, the calling device sends upgrade data to the first service device, the upgrade data is used for upgrading firmware in the first service device or AI functions that can be executed by the first service device, at this time, the upgrade data is used as data to be processed, the data source identifier indicates that the data source is from the calling device in the remote device, and the calling device sends the upgrade data to the service device through the first data channel.

In the embodiment of the application, the calling device can send the upgrade data to the service device, so that the service device upgrades the firmware or the executable AI function according to the upgrade data, and an implementation mode for upgrading the service device is provided.

According to the seventh implementation manner of the first aspect of the embodiments of the present application, in an eighth implementation manner of the first aspect of the embodiments of the present application, when it is determined that the service device does not have the target AI function, the calling device sends capability model data corresponding to the target AI function to the service device, where the capability model data can enable the service device to have the target AI function.

According to an eighth implementation manner of the first aspect of the embodiments of the present application, in a ninth implementation manner of the first aspect of the embodiments of the present application, a specific process of receiving, by a calling device, result data sent by a first service device may include the calling device receiving a capability data backhaul message returned by the first service device, and the calling device sending load data in the capability data backhaul message to a caller service in the calling device that remotely invokes the target AI function, where the capability data backhaul message includes: at least one caller service ID for indicating the name of the AI function that processed the load data, a function name for indicating at least one caller service in the calling device that received the load data, and load data.

In the embodiment of the application, an embodiment of respectively receiving the result data by each caller service is provided, so that the flexibility of the scheme is improved.

According to a tenth implementation manner of the first aspect of the present application, the calling device may be at least one of a television, a Set Top Box (STB), a sweeping robot, or a smart sound.

According to a tenth implementation manner of the first aspect of the embodiments of the present application, in an eleventh implementation manner of the first aspect of the embodiments of the present application, the service device may be at least one of a camera device, a smart doorbell, or a home smart computing center.

A second aspect of an embodiment of the present application provides an information transmission method, including:

the first service device responds to a first command sent by the calling device, capability information of the first service device is sent to the calling device, the capability information comprises an AI function which can be executed by the first service device, the first service device responds to a second command sent by the calling device, a connection channel is established between the first service device and the calling device and is a bidirectional channel between the first service device and the calling device in general, the first service device responds to a third command of the calling device to operate a target AI function to obtain result data, the third command comprises a function name of the target AI function, the target AI function is the function name of the AI function which the calling device hopes to remotely call, and the first service device sends the result data obtained by operating the target AI function to the calling device.

In the embodiment of the application, an information transmission method is provided, where a device with a target AI function, that is, a service device, may provide capability information to a device without the target AI function in a related network, that is, a calling device, by using the information transmission method provided in the embodiment of the application, and establish a connection channel with the calling device based on the information transmission method provided in the embodiment of the application, so as to provide remote call of the target AI function to the calling device.

Based on the second aspect of the embodiments of the present application, in a first implementation manner of the second aspect of the embodiments of the present application, the first command may be a capability information acquisition command of the COAP, where the capability information acquisition command includes an IP address, a port number, and an information acquisition command field of a service device, where the information acquisition command field is used to indicate that capability information of the first service device is acquired.

According to a second aspect of the embodiments of the present application or the first implementation of the second aspect, before the first service device sends the capability information of the first service device to the calling device in response to the first command sent by the calling device, the method further includes registering device information with the SSDP after the first service device is online, where the device information includes a device name, an IP address, and a port number, and the first service device multicasts the device information of the first service device to a network where the first service device is located through the SSDP, where the network where the first service device is located includes the calling device.

In the embodiment of the application, an implementation manner that the calling device can acquire the device information of each service device is provided, the calling device can determine the service device which can be called by the calling device according to the device information, and if the device name in the device information has an association relationship with the target AI function in the case of being capable of being realized, if the target AI function is face recognition, the calling device can select the service device associated with the device name and the camera device to acquire the capability information of the service device.

According to a third implementation manner of the second aspect of the embodiments of the present application, the second command may be a connection establishment command of the COAP, where the connection establishment command includes a first service device IP address, a port number, a connection establishment command field, and a load variable, the load variable includes a data channel type, the first service device IP address and a data channel port, a procedure that the first service device establishes a connection channel with the calling device in response to the second command sent by the calling device may specifically be that the first service device establishes a first data channel and a second data channel between the first service device and the calling device, the first data channel is used for the first service device to receive data sent by the calling device, the second data channel is used for the first service device to send the result data to the calling device, the data channel port is used for indicating a port of the first data channel, and the port of the second data channel may be determined according to the first data channel, for example, the first data port is added with a port of the second data channel.

In this embodiment of the present application, the connection channel established by the first service device in response to the second command and the calling device may include two channels in opposite directions, and a bidirectional connection channel is formed between the calling device and the service device, so as to implement bidirectional data transmission between the calling device and the service device.

According to a third implementation of the second aspect to the third implementation of the second aspect of the embodiments of the present application, the third command may be a capability call command, where the capability call command includes a caller service ID, a capability call command field, and a subscription information packet, the subscription information packet includes a called capability name, the called capability name is a function name of a target AI function, the caller device may include one or more caller services, a process that the first service device operates the target AI function in response to the third command of the caller device may be specifically that the first service device receives a capability call command of at least one caller service sent by the caller device, the first service device operates a corresponding target AI function according to the called capability name included in each capability caller command, so as to obtain result data, the first service device feeds back the result data to the service device, and the first service device sends the result data obtained by processing the target AI function corresponding to each caller service that invokes its target AI function.

According to a fifth implementation manner of the second aspect of the embodiments of the present application, the subscription information packet includes a data source identifier, where the data source identifier is used to indicate that data to be processed by the target AI function of the first service device is from the first service device or the remote device, that is, the data source identifier of the data to be processed from the first service device and the data source identifier of the remote device may be distinguished by different symbols, and it may be understood that the remote device includes the calling device.

According to a third implementation manner to a fifth implementation manner of the second aspect of the embodiments of the present application, in a sixth implementation manner of the second aspect of the embodiments of the present application, when the data to be processed indicated by the data source identifier is data from the calling device, the first service device may receive the data to be processed through the first data channel.

According to a seventh implementation manner of the second aspect of the embodiments of the present application, when the first service device needs to be upgraded, the first service device may receive upgrade data sent by the calling device, the first service device upgrades firmware or AI functions that can be executed by the first service device according to the upgrade data, at this time, the upgrade data is used as data to be processed, the data source identifier indicates that the data source is from the calling device in the remote device, and the first service device may receive the upgrade data sent by the calling device through the first data channel.

According to a seventh implementation manner of the second aspect of the embodiments of the present application, in an eighth implementation manner of the second aspect of the embodiments of the present application, if the first service device does not have the target AI function, the first service device may receive capability model data corresponding to the target AI function sent by the calling device, and the first service device performs a capability upgrade according to the capability model data, so that the first service device has the target AI function, and after the first service device completes the capability upgrade, updates capability information thereof, for example, adds the AI function corresponding to the capability model data, and feeds back the updated capability information to the calling device.

According to a seventh implementation manner of the second aspect of the embodiments of the present application, in an eighth implementation manner of the second aspect of the embodiments of the present application, a specific process that the first service device sends result data to the calling device may be that the first service device sends a capability data backhaul message to the calling device, where the capability data backhaul message includes: at least one caller service ID for indicating the name of the AI function that processed the load data, a function name for indicating at least one caller service in the calling device that received the load data, and load data.

According to a ninth implementation manner of the second aspect of the embodiments of the present application, in a tenth implementation manner of the second aspect of the embodiments of the present application, the calling device may be at least one of a television, a Set Top Box (STB), a sweeping robot, or a smart sound.

According to a tenth implementation manner of the second aspect of the embodiments of the present application, in an eleventh implementation manner of the second aspect of the embodiments of the present application, the service device may be at least one of an image capturing device, a smart doorbell, or a home smart computing center.

A third aspect of the present application provides an apparatus for information transmission, which may be used as a calling device, and the apparatus has a function of implementing the method in the first aspect and its implementation manners. The apparatus comprises a processor for receiving or transmitting data and a transmission interface for invoking software instructions stored in a memory for performing the information transmission method of the first aspect and its implementations described above.

A fourth aspect of the present application provides an apparatus for information transmission, the apparatus being operable as a calling device, the apparatus comprising:

an acquisition unit configured to acquire capability information of each of at least one service device based on a first command, the at least one service device including a first service device; the establishing unit is used for establishing a connection channel with the first service equipment based on the second command under the condition that the acquired capability information of the first service equipment comprises a target AI function, wherein the target AI function is an AI function which the calling equipment hopes to remotely call; the calling unit is used for remotely calling the target AI function of the first service equipment based on the third command; and the receiving unit is used for receiving result data sent by the first service equipment, wherein the result data comprises data generated after the first service equipment operates the target AI function according to the third command.

The elements of the calling device of the present application may be adapted to perform the functions of the methods of the first aspect and its implementations described above.

A fifth aspect of the present application provides a service device with AI-processing capability, which may be used as a service apparatus, the service device having a function of implementing the method in the second aspect and its respective implementation manners. The service device comprises a processor and a transmission interface, wherein the transmission interface is used for receiving or sending data, and the processor is used for calling software instructions stored in a memory to execute the information transmission method in the second aspect and various implementation manners thereof.

A sixth aspect of the present application provides a service apparatus having AI-processing capability, the service apparatus being operable as a service device, the service apparatus comprising:

a first transmitting unit, configured to transmit capability information to the calling device in response to a first command transmitted by the calling device, where the capability information includes AI functions that can be executed by the service device; the establishing unit is used for establishing a connection channel with the calling equipment in response to a second command sent by the calling equipment; the operation unit is used for responding to a third command of the calling equipment to operate the target AI function so as to obtain result data; the third command contains the function name of the target AI function, wherein the target AI function is the capability name of the AI function which the calling device hopes to remotely call; and the second sending unit is used for sending the result data to the calling equipment.

The units in the service device of the present application may be adapted to perform the functions of the method in the second aspect and its implementations described above.

A seventh aspect of the present application provides a computer storage medium having stored therein instructions which when executed on a computer cause the computer to perform the method of the first or second aspect and implementations thereof as described above.

An eighth aspect of the present application provides a computer program product which, when run on a computer, causes the computer to perform the method of the first or second aspect and implementations thereof as described above.

Drawings

FIG. 1 is a schematic diagram of a system framework in an embodiment of the present application;

FIG. 2 is a functional schematic diagram of a calling device and a service device in an embodiment of the present application;

fig. 3 is a schematic flow chart of an information transmission method in an embodiment of the present application;

fig. 4 is a schematic data structure diagram of result data replied by a service device in an embodiment of the present application;

fig. 5 is another flow chart of an information transmission method in the embodiment of the application;

fig. 6 is a schematic diagram of a data structure of data sent by a calling device in an embodiment of the present application;

fig. 7 is another flow chart of an information transmission method in the embodiment of the application;

FIG. 8 is a schematic diagram of a cross-platform interconnect architecture in an embodiment of the present application;

fig. 9 is a schematic diagram of a smart home architecture in an embodiment of the present application;

FIG. 10 is a schematic diagram of a calling device in an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a service device according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a calling device in an embodiment of the present application;

fig. 13 is a schematic structural diagram of a service device in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The term "and/or" means either, or both

Referring to fig. 1, a system framework of an embodiment of the present application includes:

calling devices 101 to 102, serving devices 103 to 104.

The call devices 101 to 102 may transmit information between the service devices 103 to 104 through a connection channel, which may be a data channel or a command channel, the data channel being a bidirectional channel for transmitting information between the call devices 101 to 102 and the service devices 103 to 104, and the command channel being a unidirectional channel for transmitting information between the call devices 101 to 102 to the service devices 103 to 104.

The calling devices 101 to 102 may instruct the service devices 103 to 104 to perform various operations through the connection channels, for example, the calling devices 101 to 102 may determine AI functions that the service devices 103 to 104 can perform, such as a face detection function, a bone recognition function, and the like, and select appropriate service devices 103 to 104 to perform AI function call according to the operation conditions, and control the service devices 103 to 104 to perform the target AI functions required by the calling devices 101 to 102.

Alternatively, when the service devices 103 to 104 need to perform firmware upgrade, the calling devices 101 to 102 may transmit firmware upgrade data to the service devices 103 to 104, and the service devices 103 to 104 may perform firmware upgrade according to the firmware upgrade data.

Alternatively, when the calling devices 101 to 102 need to perform the capability upgrade for the service devices 103 to 104, the calling devices 101 to 102 may transmit the capability upgrade data to the service devices 103 to 104, and the service devices 103 to 104 may perform the capability upgrade according to the capability upgrade data.

In this embodiment, only two calling devices and two service devices are described as examples, and it is understood that in this embodiment of the present application, the number of calling devices and service devices is not limited.

Referring to fig. 2, in this embodiment, the calling device and the service device may both support the HiStreaming protocol (HiStreaming protocol, HISTM), and specifically, the protocols supported by the calling device and the service device may include a limited application protocol (constrained application protocol, COAP), a simple service discovery protocol (simple service discovery protocol, SSDP), and the like, and the calling device and the service device may perform corresponding information transmission based on these commonly supported protocols.

The call of the function of the service device by the call device may be a call at the device level, for example, one call device may call functions of a plurality of service devices at the same time, and one service device may be called by a plurality of call devices at the same time.

It should be noted that, the call of the call device to the function of the service device may be a call of a service level, one or more services may be run on the call device, and multiple services may call one function of the service device at the same time, and one service may call multiple functions of the service device at the same time.

In this embodiment, the information transmission between the calling device and the service device may include different flows, which are described below:

1. function call flow:

referring to fig. 3, in this embodiment, the calling device may call the AI function provided by the service device through the connection channel, and the following details are described:

301. the HISTM Server device sends device information to the HISTM Client device;

in this embodiment, the HISTM Server device is taken as an example of a service device, and the HISTM Client device is taken as an example of a calling device.

It will be appreciated that in practical applications, the calling device and the service device may also be other types of devices, for example, the calling device may be a television, a set top box, a sweeping robot, or an intelligent sound, and the service device may be a camera device, an intelligent doorbell, or a home intelligent computing center, which is not limited herein.

After the HISTM Server device is online, registering device information in the HISTM (specifically may be SSDP), and after the registration is completed, multicasting the device information to a network where the HISTM Server device is located, wherein the HISTM Client device in the network can acquire the device information and sense that the HISTM Server device is online.

It should be noted that, the HISTM Server device may directly multicast its own device information in the network through the HISTM, or register its own device information to a certain Server in the network through the HISTM, and then multicast the device information in the network by the Server, which is not limited specifically herein.

In this embodiment, the device information of the HISTM Server device may include information for identifying the HISTM Server device, such as a device name, internet protocol (internet protocol, IP) address information, and an access Port number (Port).

When a plurality of HISTM Server devices are on line, each HISTM Server device transmits device information according to the flow.

302. The HISTM Client device stores the acquired device information into an SSDP device list;

when the HISTM Client device senses that the HISTM Server device is on line, the acquired device information of the HISTM Server device can be stored in a locally stored device list.

303. The HISTM Client device acquires capability information of the HISTM Server device;

after the HISTM Client device receives the device information of the plurality of HISTM Server devices, the HISTM Client device can poll the capability information of each HISTM Server device in the locally stored device list through the COAP.

In an optional case, before the HISTM Client device obtains the capability information of the HISTM Server device, the HISTM Client device determines the device names of the HISTM Server devices to be searched, then determines all the HISTM Server devices with the device names meeting the requirements in the device list, further, the HISTM Client device polls all the HISTM Server devices with the device names meeting the requirements through the COAP, and obtains the capability information of the HISTM Server devices with the device names meeting the requirements.

For example, if the HISTM Client device determines that the device name of the HISTM Server device to be searched is a name including a CameraVideo, the HISTM Client device may select a HISTM Server device whose device name includes a CameraVideo from the device list, for example, may be a CameraVideo0, a CameraVideo3, or the like.

After selecting all HISTM Server devices with device names meeting the requirements, the HISTM Client device polls the selected HISTM Server devices through the COAP, and acquires capability information of the HISTM Server devices.

In an optional case, after the HISTM Client device receives the capability information of one HISTM Server device with the AI function desired to be invoked, the HISTM Client device stops acquiring the capability information of other HISTM Server devices; in another possible case, the hism Client device acquires capability information of all hism Server devices having AI functions desired to be invoked, and selects an hism Server device with better capability according to a preset rule.

Specifically, the HISTM Client device may send a first command CMD1 to the HISTM Server device through the COAP, where the CMD1 includes a device name, an IP address, a port number, and an information acquisition command of the HISTM Server device, where the information acquisition command is used to indicate capability information of the HISTM Server device to be acquired.

In practical applications, the specific form of CMD1 may be as follows:

Send：coap：//192.168.1.155：50614/CameraVideo0ServiceInfo(value＝Hi_STM_Test：11)

wherein 192.168.1.155 is an IP address of a certain HISTM Server device, 50614 is a port number of the HISTM Server device, cameraVideo0 is a device name of the HISTM Server device, serviceInfo is an information acquisition command, hi_stm_test:11 are command parameters and length.

304. The HISTM Server equipment replies capability information to the HISTM Client equipment;

After receiving CMD1, HISTM Server device obtains own capability information, and feeds back capability information to HISTM Client device through first Reply message Reply 1.

In this embodiment, the Reply1 message format may be as follows:

"DevServiceType": "Server device name",

"ServiceNum": the amount of callable capabilities supported by the current device,

"capability 1 information": { "Name": "capability name 1", "transport": transmission scheme, "output-len-bytes": the output data length },

"capability 2 information": { "Name": "capability name 2", "transport": transmission scheme, "output-len-bytes": output data length …

"capability N information": { "Name": "capability name N", "transport": transmission scheme, "output-len-bytes": output data length }

The callable capability number is used to represent the capability number that the HISTM Server device can support.

The capability name is used to indicate the name of the AI function supported by the HISTM Server device, such as face recognition or skeleton detection.

The transmission mode is used to indicate what transmission mode the Reply1 message is sent to, or what transmission mode the HISTM Client device can call the capability, or what transmission mode the HISTM Server device can return the result data after the capability is called, which case depends on the requirement of the actual application environment, and the method is not limited herein.

The output data length is used to represent the length of the result data returned by the HISTM Server device after the capability is called.

In this embodiment, the capability information may be stored in a lightweight data exchange format, such as c-language (c java script object notation, CJSON), but it will be understood that the capability information may be stored in other formats, which is not limited herein.

305. The HISTM Client device determines a first HISTM Server device meeting the requirements according to the capability information;

after the HISTM Client devices receive the Reply1 message sent by each HISTM Client device, a first HISTM Server device meeting the requirements can be determined according to the capability information in the Reply1 message, the capability information of the first HISTM Server device comprises a target AI function, and the target AI function is an AI function which the HISTM Client device hopes to remotely call.

It should be noted that if the hism Client device does not find a hism Server device meeting the requirement according to the capability information in the Reply1 message currently acquired, the hism Client device will continue to wait for the next hism Server device.

306. The HISTM Client device sends a connection establishment request to the first HISTM Server device;

After the first HISTM Server device is determined by the HISTM Client device, a second command CMD2 may be sent to the first HISTM Server device by COAP to request connection establishment.

The CMD2 comprises a device name, an IP address, a port number, a connection establishment command and a load variable of the first HISTM Server device, wherein the load variable comprises a data channel type, and the IP address and the data channel port of the first HISTM Server device.

In practical applications, the specific form of CMD2 may be as follows:

Send：coap：//192.168.1.155：50614/CameraVideo0？.sys.mediaservice.setup(value＝{“p”：2，“ep”：{“addr”：“192.168.1.155”，“port”：6666}})

where 192.168.1.155 is the IP address of the first hism Server device, 50614 is the port number of the first hism Server device, it should be understood that the hism Client device sends a command, such as the above connection establishment request, etc., to the first hism Server device through the port indicated by the port number, where the camera video0 is the device name of the first hism Server device, the connection establishment command is sys.

It should be noted that, the port of the data channel is generally a port where the first hism Client device sends data to the first hism Server device, and the port where the first hism Server device sends data to the hism Client device may be confirmed according to a port included in the load variable, for example, when the port where the hism Client device sends data to the first hism Server device is 6666, the port where the first hism Server device sends data to the hism Client device may be configured to be 6666 plus 1, that is 6667.

307. The first HISTM Server device creates a data channel;

after receiving CMD2 sent by the HISTM Client device, the first HISTM Server device sends a second Reply message Reply2 to the HISTM Client device, and creates a receiving data channel and a sending data channel.

The sending data channel is used for returning result data to the HISTM Client device, and the receiving data channel is used for receiving data, such as firmware upgrading data, from the HISTM Client device.

After the data channel is established, the HISTM Client device and the first HISTM Server device realize bidirectional data intercommunication.

It should be noted that, the first HISTM Server device may select to establish the receive data channel and the transmit data channel according to actual requirements, and if operations such as firmware upgrade do not need to be performed or source data to be processed by the AI function are all from a scenario that the local end of the first HISTM Server device does not need to receive external data, the receive data channel may not be established, which is not limited in specific places.

308. The HISTM Client device sends a calling instruction to the first HISTM Server device;

in this embodiment, the HISTM Client device may send a third command CMD3 to the first HISTM Server device through COAP to request to call the target AI function of the first HISTM Server device.

The CMD3 may include a caller ID, a capability call command, and a call information message, where the call information message includes a called capability name, a data source, and a transmission mode.

Depending on the source of the data, the CMD3 can take a number of different forms:

1. the data source is a first HISTM Server device:

in this embodiment, the data source may be a first HISTM Server device, that is, the data to be processed indicating the AI function of the first HISTM Server device comes from the local end of the first HISTM Server device, or the AI function uses the data in the first HISTM Server device to process, for example, based on the video stream information collected by the AI camera in the first HISTM Server device.

At this time, the specific form of CMD3 may be as follows:

send: and (3) shape: //192.168.1.155:50614/Cameravideo0command (value= { "ClientID":1, "cmd": "subscore", "subscore": { call message })

The specific form of the call information message may be as follows:

{ "Service": "capability name", "DataSource":0, "TrasType":2}

Wherein 192.168.1.155 is the IP address of the first HISTM Server device, 50614 is the port number of the first HISTM Server device, cameraVideo0 is the device name of the first HISTM Server device, clientID is the caller identification, "cmd": "subscore" is a capability call command.

It should be noted that, the ClientID may have multiple manifestations, if it is a call at a device level, the ClientID may be an identifier of the HISTM Client device, if it is a call at a service level, the ClientID may be an identifier of a service of the HISTM Client device, and specifically, the present invention is not limited thereto.

"Service" is the name of the capability that is called, which must be consistent with the name of the capability provided by the first HISTM Server device.

"DataSource" is the data source, which in this embodiment is the first HISTM Server device.

The "TrasType" is a transmission mode, and is used to indicate what transmission mode the call information message is sent to, or is used to indicate what transmission mode the first hism Server device returns the result data after the capability is called, which specific use case depends on the requirement of the actual application environment, and is not limited herein.

2. The data source is HISTM Client equipment:

in this embodiment, the data source may be an hism Client device, that is, the first hism Server device uses data transmitted by the hism Client device to process the data.

At this time, the specific form of CMD3 may be as follows:

send: and (3) shape: //192.168.1.155:50614/Cameravideo0command (value= { "ClientID":1."cmd": "subscore", "subscore": { call message })

The specific form of the call information message may be as follows:

{ "Service": "capability name", "DataSource":1, "TrasType":2}

At this time, the parameters in CMD3 are substantially similar to those described above, and similar parts will not be repeated, and the difference is mainly "DataSource", in this embodiment, the data source is a HISTM Client device.

It should be noted that, the specific value of "DataSource" is not limited in practical application, as long as different data sources can be distinguished.

309. The first HISTM Server device processes according to the calling instruction;

after receiving the call instruction, the first HISTM Server device may perform corresponding processing according to the call instruction, for example, running a target AI function.

It should be noted that, if the CMD3 indicates that the data source is a first HISTM Server device, the first HISTM Server device uses local data to process.

If the CMD3 indicates that the data source is an HISTM Client device, the first HISTM Server device first needs to receive data to be processed from the HISTM Client device through the established receive data channel, and when a data transmission completion command sent by the HISTM Client device is received, the first HISTM Server device may process using the received data to be processed.

310. The first HISTM Server device sends result data to the HISTM Client device.

After the first HISTM Server device performs corresponding processing according to the call instruction, result data can be sent to the HISTM Client device, wherein the result data comprises data generated after the first HISTM Server device operates the target AI function according to CMD 3.

In this embodiment, the specific form of the result data may be as shown in fig. 4.

The result data can be divided into a message header and load data.

Message header information meaning:

head Length (Length of header): 8bits, which is used to represent the sum of the length of the load length field, the caller number field, the caller ID field, the capability name length field and the capability name field.

Payload Length (Payload Length field): 32bits to represent the length of the actual payload data.

ClientNum (caller number field): 8bits to represent the number of current capability calls made by the HISTM Client device.

ClientID0 (caller ID field): 8bits, which are used to represent the service ID of HISTM Client equipment calling the current capability, distinguish different service calls and report data.

Servicename Length field: the capability name length is used for representing the data transmission at this time.

ServiceName (capability name field): the serviceNameLen is 8bits used for representing the capability name of the AI function for processing the obtained load data, and the HISTM Client device performs service processing according to the capability name.

Load data portion:

payload (Payload data field): the Payload Length is 8bits, and is used for representing real load data, namely data generated after the first HISTM Server device performs corresponding processing according to a call instruction.

It should be noted that, if the call is a call at a service level, the caller ID is used to indicate the service ID of the hism Client device that receives the load data, and if the call is a call at a device level, the caller ID is used to indicate the ID of the hism Client device that receives the load data; or multiple caller IDs may also indicate multiple service IDs from multiple hism Client devices.

In this embodiment, if the service level is called, after the hism Client device receives the result data, the actual load data and the capability name are called back to each service of the hism Client device according to the header, for example, the face detection data is transferred to each face detection service application (intelligent push service, distance reminding service, etc.).

2. The equipment control flow comprises the following steps:

In this embodiment, the calling device may perform various forms of control on the service device through the connection channel, and the following descriptions respectively apply:

1. calling equipment to control service equipment to upgrade firmware:

referring to fig. 5, in this embodiment, the calling device may control the service device to perform firmware upgrade through the connection channel:

501. the HISTM Server device establishes connection with the HISTM Client device.

In this embodiment, the description is given by taking the HISTM Server device as an example of the service device, and the description is given by taking the HISTM Client device as an example of the calling device, it may be understood that in practical application, the calling device and the service device may also be other types of devices, and the specific application is not limited herein.

In this embodiment, the process of establishing connection between the hism Server device and the hism Client device may be similar to steps 301 to 307 in the embodiment shown in fig. 3, and will not be described herein.

It should be understood that in this embodiment, the connection between the hism Server device and the hism Client device may be established in other manners, for example, the connection between the hism Client device and all on-line hism Server devices is established, which is not limited herein.

No matter what manner the HISTM Server device establishes a connection with the HISTM Client device, the HISTM Server device needs to establish a receive data channel in preparation for receiving firmware upgrade data from the HISTM Client device.

502. The HISTM Client device sends a firmware upgrade start command to the HISTM Server device;

when the HISTM Client device determines that the HISTM Server device needs to perform firmware upgrade, a firmware upgrade start command can be sent to the HISTM Server device through the HISTM command channel.

Specifically, the HISTM Client device may query the current firmware version of the HISTM Server device, and query whether the updated firmware version is stored in the local or cloud, if yes, determine that the HISTM Server device needs to perform firmware upgrade, it may be understood that the HISTM Client device may also determine whether the HISTM Server device needs to perform firmware upgrade in other manners, and in particular, the method is not limited herein.

In this embodiment, the HISTM Client device may send a firmware upgrade start command to the HISTM Server device through COAP, and the specific form may be as follows:

send: and (3) shape: //192.168.1.155:50614/Cameravideo0command (value= { "ClientID":1, "cmd": "firmware upgrade", "param": "command parameter" }

Wherein 192.168.1.155 is the IP address of the HISTM Server device, 50614 is the port number of the HISTM Server device, cameraVideo0 is the device name of the HISTM Server device, clientID is the controller identification, "cmd": "firmware upgrade" is a command, and "command parameter" is a command parameter.

503. The HISTM Server equipment performs upgrade preparation;

when the HISTM Server device receives the firmware upgrade start command, upgrade preparation is required, for example, preparation of sufficient available storage space, suspension of currently ongoing work, and the like, which is not limited herein.

504. The HISTM Server device sends a reply message to the HISTM Client device;

and after the HISTM Server equipment finishes the upgrade preparation, a reply message is sent to the HISTM Client equipment.

505. The HISTM Client device sends firmware upgrade data to the HISTM Server device;

the HISTM Client device may send firmware upgrade data to the HISTM Server device via the established data channel.

In this embodiment, the data sent by the hism Client device may be as shown in fig. 6, where the data includes two parts, i.e., a header and payload data.

Message header information meaning:

head Length (Length of header): 8bits to represent the sum of the length of the payload length field and the sender ID field.

ClientID (sender ID field): 8bits, which are used to represent the ID of the HISTM Client device sending the data or the ID of the service of the HISTM Client device, and distinguish different data sources from the HISTM Server device.

Load data portion:

payload (Payload data field): the Payload Length is 8bits, which is used to represent the real load data, the data actually sent by the HISTM Client device to the HISTM Server device, and so on.

506. The HISTM Client device sends a data sending completion command to the HISTM Server device;

after the HISTM Client device finishes sending the firmware upgrade data, a data sending completion command can be sent to the HISTM Server device.

507. The HISTM Server device performs firmware upgrade.

After the HISTM Server device receives the data transmission completion command, the real load data can be used for firmware upgrading according to the message header.

2. Calling equipment to control service equipment to perform capability upgrading:

referring to fig. 7, in this embodiment, the calling device may control the service device to perform capability upgrade through the connection channel:

701. the HISTM Server device establishes connection with the HISTM Client device.

Step 701 in this embodiment is similar to step 501 in the embodiment shown in fig. 5, and will not be repeated here.

702 to 703, acquiring capability information of the HISTM Server equipment by the HISTM Client equipment;

in this embodiment, the capability information of the hism Client device may be obtained by the hism Server device, and the specific obtaining process may be similar to that of the foregoing embodiments 303 to 304 shown in fig. 3, which is not described herein again.

704. The HISTM Client device sends a capability upgrading start command to the HISTM Server device;

after the HISTM Client device acquires the capability information of the HISTM Server device, if the HISTM Server device is found not to support a certain target capability, a capability upgrading start command is sent to the HISTM Server device.

It should be noted that the target capability may be an upgrade capability of a certain capability currently supported by the HISTM Server device, or a capability that can be supported by hardware of the HISTM Server device but not actually supported by the hardware of the HISTM Server device, which is not limited herein.

In this embodiment, the HISTM Client device may send the capability upgrade start command to the HISTM Server device through COAP, and the specific form may be as follows:

send: cep: //192.168.1.155:50614/Cameravideo0command (value= { "ClientID":1, "cmd": "capability upgrade", "param": "command parameter" }

Wherein 192.168.1.155 is the IP address of the HISTM Server device, 50614 is the port number of the HISTM Server device, cameraVideo0 is the device name of the HISTM Server device, clientID is the controller identification, "cmd": "capability upgrade" is a command, and "command parameter" is a command parameter.

705. The HISTM Server equipment performs upgrade preparation;

when the HISTM Server device receives the capability upgrade start command, upgrade preparation is required, for example, preparation of sufficient available storage space, suspension of the currently ongoing related capability, etc., which is not limited herein.

706. The HISTM Server device sends a reply message to the HISTM Client device;

707. The HISTM Client device sends capability upgrading data to the HISTM Server device;

the HISTM Client device may send capability upgrade data to the HISTM Server device via the established data channel.

In this embodiment, the data sent by the hism Client device may be shown in fig. 6, which is not described herein.

708. The HISTM Client device sends a data sending completion command to the HISTM Server device;

after the HISTM Client device sends the capability upgrading data, a data sending completion command can be sent to the HISTM Server device.

709. Performing capability upgrading by HISTM Server equipment;

after the HISTM Server device receives the data transmission completion command, the real load data may be used for capability upgrading according to the header, for example, the original face recognition capability information is changed into human-shaped matting.

710. The HISTM Server device re-registers the capability information.

After the capability of the HISTM Server device is upgraded, the capability information can be registered with the HISTM again, and when the HISTM Client device acquires the capability information from the HISTM Server device again, the HISTM Server device can provide the updated capability information.

It should be noted that, the embodiments described above are described by taking SSDP and COAP as examples, and since SSDP and COAP both belong to lightweight interconnection protocols, they can operate in the internet of things device with limited resources, thereby improving the application range of the embodiments of the present application.

It is to be understood that the embodiments of the present application do not necessarily need to use SSDP and COAP, but may be implemented using other protocols in the HISTM, which is not limited herein.

In this embodiment of the present application, the hism Client device and the hism Server device may be devices of the same platform (for example, all belong to an Android platform or a Linux platform), or may be cross-platform devices (for example, one of them is an Android platform and the other is a Linux platform), and the following description is given of the cross-platform situation:

referring to fig. 8, the overall interconnection devices are divided into an hism Client device (Android device) and an hism Server device (Linux device).

The interconnection architecture can be used for the android device or the set top box device to remotely call the AI capability (face detection, face recognition and the like) of the AI Camera, so that the product form of AI business expansion is realized.

In this interconnect architecture, the HISTM Client devices can be divided into three layers: application (APP) layer, hism Service layer (hism Service layer), hism protocol layer. Each APP is an independent process, and the HISTM Service and the HISTM protocol belong to the same process. And the APP process communicates with the HISTM Service process through the Android C/S Binder, so that the call of the bottom HISTM protocol is realized. The HISTM Service layer belongs to the middle layer, and transfers the calling information of the upper APP to the protocol layer, which supports multi-instance access, and each APP process has a unique HISTM Server correspondence in the HISTM Service.

The bottom layer is an HISTM protocol layer, which comprises COAP, SSDP and other protocols, supports multi-instance calling, and can support simultaneous calling of a plurality of HISTM servers.

In this interconnect architecture, the HISTM Server device can be divided into two layers: APP layer and hism protocol layer. The APP layer and the hism protocol belong to the same process.

After APP is operated, reading a locally stored capability information configuration file, registering the capability information into an HISTM protocol, and waiting for capability call of the HISTM Client device.

Based on the above architecture, the following describes a data interaction procedure between the HISTM Client device and the HISTM Server device under the cross-platform condition:

taking AI face detection APP1 and skeleton recognition APP2 of HISTM Client equipment as examples, after the system is started, HISTM Service runs in the background, APP1 and APP2 inquire SSDP equipment lists of an HISTM protocol layer through HISTM JNI and a Binder module, and then poll equipment capacity supported by each HISTM Server equipment through a command channel.

After receiving the query information, the HISTM Server equipment returns the capability information of the local end to the HISTM protocol layer of the HISTM Client equipment, and then returns the capability information to the APP through a Binder Notify mechanism.

If the capability information returned by the HISTM Server equipment comprises the capabilities of 'face detection' and 'skeleton recognition', APP1 and APP2 respectively send a remote capability calling command to the HISTM Server equipment, and the face detection capability and the skeleton recognition capability are called.

After receiving the capability call command, the HISTM Server device starts the processing of the corresponding capability in the capability information according to the capability name, and sends the result data back to the HISTM Client device through the data channel.

After the HISTM Client device receives the data, the data is transmitted back to the APP layer through the shared memory, and the APP uses the actual data for specific service, so that the whole process is completed.

The information transmission method in the embodiment can also realize the capability of the HISTM Client device to call the HISTM Server device under the cross-platform scene, so that the application range is improved.

It should be noted that, besides being applicable to the above scenario, the hism may be widely applied to the smart home field, and the following description is given to the application of the hism in the smart home field:

referring to fig. 9, a general sweeping robot (hism Client device) establishes a network connection with a home intelligent computing center (hism Server device) through WiFi.

After the HISTM module of the common sweeping robot operates, the sweeping route calculation capability of the home intelligent calculation center is remotely called, and the sweeping route information acquired in real time is returned to the sweeping control application, so that intelligent control of the sweeping route is realized.

The method for transmitting information in the embodiment of the present application is described above, and the device in the embodiment of the present application is described below.

Referring to fig. 10, an embodiment of the present application provides a calling device 1000, where the calling device 1000 may include one or more processors 1001 and a memory 1005, where the memory 1005 stores program codes, and further, the memory 1005 may store data.

The memory 1005 may be volatile memory or nonvolatile memory or persistent memory, among others. The program code stored in the memory 1005 may include one or more modules, each of which may include a series of instruction operations in the call apparatus. Still further, the processor 1001 may be configured to communicate with the memory 1005 to execute a series of instruction operations in the memory 1005 on the recall device 1000.

The calling device 1000 may also include one or more power supplies 1002, one or more wired or wireless network interfaces 1003, one or more input/output interfaces 1004, and/or one or more operating systems, such as Windows, android, mac 0S, IOS, unix, linux, freeBSD, etc.

The processor 1001 may perform the operations performed by the calling device in the embodiments shown in fig. 3, 5 and 7, which are not described herein.

Referring to fig. 11, an embodiment of the present application provides a service device 1100, where the service device 1100 may include one or more processors 1101 and a memory 1105, where the memory 1105 stores program codes, and further, the memory 1105 may store data.

The memory 1105 may be a volatile memory or a nonvolatile memory or a persistent memory. The program code stored in the memory 1105 may include one or more modules, each of which may include a series of instruction operations in the service device. Still further, the processor 1101 may be configured to communicate with the memory 1105 and execute a series of instruction operations in the memory 1105 on the service device 1100.

The service device 1100 may also include one or more power supplies 1102, one or more wired or wireless network interfaces 1103, one or more input output interfaces 1104, and/or one or more operating systems, such as Windows, android, mac OS, IOS, unix, linux, freeBSD, etc.

The processor 1101 may perform the operations performed by the service device in the embodiments shown in fig. 3, 5 and 7, and detailed descriptions thereof are omitted herein.

Referring to fig. 12, an embodiment of the present application provides a calling device, including:

an acquiring unit 1201, configured to acquire capability information of each of at least one service device based on the first command, where the at least one service device includes a first service device;

An establishing unit 1202, configured to establish a connection channel with the first service device based on the second command, where the obtained capability information of the first service device includes a target AI function, where the target AI function is an AI function that the calling device wants to remotely call;

a calling unit 1203 configured to remotely call a target AI function of the first service device based on the third command;

and the receiving unit 1204 is configured to receive result data sent by the first service device, where the result data includes data generated after the first service device operates the target AI function according to the third command.

In this embodiment, each unit in the calling device may execute the operations executed by the calling device in the embodiments shown in fig. 3, 5, and 7, which are not described herein.

Referring to fig. 13, an embodiment of the present application provides a service device, including:

a first transmitting unit 1301 configured to transmit capability information to the calling device in response to a first command transmitted by the calling device, the capability information including an AI function executable by the service device;

an establishing unit 1302, configured to establish a connection channel with the calling device in response to the second command sent by the calling device;

an operation unit 1303, configured to operate the target AI function in response to a third command of the calling device, to obtain result data; the third command contains the function name of the target AI function, wherein the target AI function is the capability name of the AI function which the calling device hopes to remotely call;

A second sending unit 1304, configured to send the result data to a calling device.

In this embodiment, each unit in the service device may perform the operations performed by the service device in the embodiments shown in fig. 3, 5, and 7, which are not described herein in detail.

It should be appreciated that the first transmitting unit and the second transmitting unit may be physically implemented by the same transmitting unit, e.g. may both be implemented by a transceiver or a transmission interface.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be stored by a computer or data storage devices such as servers, data centers, etc. that contain an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium such as a Solid State Disk (SSD), or the like.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims

1. An information transmission method, the method comprising:

the method comprises the steps that calling equipment obtains capability information of each service equipment in at least one service equipment based on a first command, wherein the at least one service equipment comprises first service equipment;

under the condition that the calling device determines that the acquired capability information of the first service device comprises a target Artificial Intelligence (AI) function, the calling device establishes a connection channel with the first service device based on a second command, wherein the target AI function is an AI function which the calling device hopes to remotely call;

the calling device sends capability model data corresponding to the target AI function to the first service device under the condition that the first service device does not have the target AI function, wherein the capability model data can enable the first service device to have the target AI function;

the calling device remotely calls the target AI function of the first service device based on a third command;

and the calling device receives result data sent by the first service device, wherein the result data comprises data generated after the first service device operates a target AI function according to the third command.

2. The method of claim 1, wherein before the calling device obtains the capability information of each of the at least one service device based on the first command, the method further comprises:

and the calling equipment receives equipment information of the at least one service equipment multicast by a Simple Service Discovery Protocol (SSDP) to obtain a service equipment list, wherein the equipment information comprises equipment names, internet Protocol (IP) addresses and port numbers.

3. The method according to claim 1 or 2, wherein the first command is a capability information acquisition command of a limited application protocol COAP, the capability information acquisition command including an IP address, a port number, and an information acquisition command field of a service device, the information acquisition command field being used for indicating that capability information of the service device is acquired, and the calling device acquiring capability information of each of at least one service device based on the first command, specifically including:

the calling device sends the capability information acquisition command to each service device with the device name meeting the requirement in the service device list;

and the calling device receives the capability information returned by the at least one service device in response to the capability information acquisition command.

4. The method according to claim 1 or 2, wherein the second command is a connection establishment command of COAP, the connection establishment command including a service device IP address, a port number, a connection establishment command field, and a load variable, the load variable including a data channel type, the service device IP address and a data channel port, the calling device establishing a connection channel with the first service device based on the second command, specifically comprising:

the calling device establishes a first data channel and a second data channel based on the connection establishment command and the first service device, wherein the first data channel is used for sending data to the first service device by the calling device, the second data channel is used for receiving result data returned by the first service device by the calling device, and a data channel port is used for indicating a port of the first data channel.

5. The method according to claim 1 or 2, wherein the third command is a capability call command, the capability call command comprising a caller service identification number ID, a capability call command field and a subscription information message, the subscription information message comprising a called capability name, the called capability name being a function name of the target AI function, the calling device comprising a plurality of caller services, the calling device remotely calling the target AI function of the first service device based on the third command, in particular comprising:

The calling device sends a capability calling command of at least one caller service to the first service device, so that the first service device operates a corresponding target AI function according to the called capability names contained in the capability calling commands respectively;

the calling device receives the result data sent by the first service device, and specifically includes:

and the plurality of caller services of the calling device respectively receive result data obtained by processing target AI functions corresponding to the plurality of caller services in the first service device.

6. The method of claim 5, wherein the subscription information message further comprises: and the data source identifier is used for indicating that the data to be processed by the target AI function of the first service device comes from the first service device or a remote device, and the remote device comprises the calling device.

7. The method of claim 6, wherein when the data source identifier indicates that the data to be processed is from the calling device, the method further comprises: and the calling equipment sends the data to be processed to the first service equipment through a first data channel.

8. The method according to claim 4, wherein the method further comprises:

and the calling device sends upgrade data to the first service device, wherein the upgrade data is used for upgrading firmware in the first service device or AI functions which can be executed by the first service device.

9. The method according to claim 1 or 2, wherein the calling device receives result data sent by the first service device, specifically comprising:

the calling device receives a capability data return message returned by the first service device, wherein the capability data return message comprises: the system comprises at least one caller service ID, a function name and load data, wherein the function name is used for indicating an AI function name for processing the load data, and the at least one caller service ID is used for indicating at least one caller service for receiving the load data in the calling equipment;

and the calling equipment sends the load data in the capability data return message to the caller service respectively indicated by the at least one caller service ID.

10. The method of claim 1 or 2, wherein the calling device comprises at least one of a television, a set-top box, a sweeping robot, or a smart sound.

11. The method of claim 1 or 2, wherein the service device comprises at least one of a camera device, a smart doorbell, or a home smart computing center.

12. An information transmission method, the method comprising:

the method comprises the steps that a first service device responds to a first command sent by a calling device, capability information of the first service device is sent to the calling device, and the capability information comprises AI functions which can be executed by the first service device;

the first service device establishes a connection channel with the calling device in response to a second command sent by the calling device;

the first service device responds to a third command of the calling device to operate a target AI function, so that result data are obtained; the third command contains the function name of the target AI function, wherein the target AI function is the AI function which the calling device hopes to call remotely;

the first service device sends the result data to the calling device;

before the first service device runs the target AI function in response to the third command of the calling device, the method further includes:

when the first service equipment does not have the target AI function, the first service equipment receives capability model data corresponding to the target AI function, which is sent by the calling equipment;

And the first service equipment performs capability upgrading according to the capability model data so that the first service equipment has the target AI function.

13. The method of claim 12, wherein the first command is a capability information acquisition command of a limited application protocol COAP, the capability information acquisition command including an IP address, a port number, and an information acquisition command field of a service device, the information acquisition command field indicating acquisition of capability information of the first service device.

14. The method according to claim 12 or 13, wherein before the first service device sends the capability information of the first service device to the calling device in response to the first command sent by the calling device, the method further comprises:

after the first service equipment is online, registering equipment information to an SSDP, wherein the equipment information comprises an equipment name, an IP address and a port number;

and the first service equipment multicasts the equipment information of the first service equipment to the network where the first service equipment is located through the SSDP, wherein the network where the first service equipment is located comprises the calling equipment.

15. The method according to claim 12 or 13, wherein the second command is a connection establishment command of a COAP, the connection establishment command including a first service device IP address, a port number, a connection establishment command field, and a load variable, the load variable including a data channel type, the first service device IP address and a data channel port, the first service device establishing a connection channel with the calling device in response to the second command sent by the calling device, specifically including:

The first service device establishes a first data channel and a second data channel based on the connection establishment command and the calling device, wherein the first data channel is used for the first service device to receive data sent by the calling device, the second data channel is used for the first service device to send result data to the calling device, and a data channel port is used for indicating a port of the first data channel.

16. The method according to claim 12 or 13, wherein the third command is a capability call command, the capability call command including a caller service ID, a capability call command field and a subscription information message, the subscription information message including a called capability name, the called capability name being a function name of the target AI function, the calling device including a plurality of caller services, the first service device running the target AI function in response to the third command of the calling device, comprising in particular:

the first service equipment receives a capability calling command of at least one caller service sent by the calling equipment;

the first service device operates a corresponding target AI function according to the called capability names respectively contained in the at least one capability caller command;

The first service device sends the result data to the calling device, specifically including:

and the first service equipment respectively sends the target AI function processing result data corresponding to the at least one caller service.

17. The method of claim 16, wherein the subscription information message further comprises: and the data source identifier is used for indicating that the data to be processed by the target AI function of the first service device comes from the first service device or a remote device, and the remote device comprises the calling device.

18. The method of claim 17, wherein when the data source identifier indicates that the data to be processed is from the calling device, the method further comprises:

the first service device receives the data to be processed sent by the calling device through a first data channel, wherein the first data channel is a connection channel which is established between the first service device and the calling device based on the second command and is used for receiving the data sent by the calling device.

19. The method according to claim 12 or 13, characterized in that the method further comprises:

The first service equipment receives the upgrade data sent by the calling equipment;

and the first service equipment upgrades the firmware or the AI function which can be executed by the first service equipment according to the upgrade data.

20. The method according to claim 12 or 13, wherein the first service device sends result data to the calling device, in particular comprising:

the first service device sends a capability data feedback message to the calling device, wherein the capability data feedback message comprises: the system comprises at least one caller service ID, a function name and load data, wherein the function name is used for indicating an AI function name for processing the load data, and the at least one caller service ID is used for indicating at least one caller service for receiving the load data in the calling equipment.

21. The method of claim 12 or 13, wherein the calling device comprises at least one of a television, a set-top box, a sweeping robot, or a smart sound.

22. The method of claim 12 or 13, wherein the service device comprises at least one of a camera device, a smart doorbell, or a home smart computing center.

23. An apparatus for information transmission, the apparatus being a calling device, the apparatus comprising: a processor and a transmission interface for receiving or transmitting data, the processor for invoking software instructions stored in the memory to perform the steps of:

acquiring capability information of each service device of at least one service device based on a first command, wherein the at least one service device comprises a first service device;

under the condition that the acquired capability information of the first service equipment comprises a target AI function, establishing a connection channel with the first service equipment based on a second command, wherein the target AI function is an AI function which the information transmission device hopes to remotely call;

transmitting capability model data corresponding to the target AI function to the first service device under the condition that the first service device does not have the target AI function, wherein the capability model data can enable the first service device to have the target AI function;

and receiving result data sent by the first service equipment, wherein the result data comprises data generated after the first service equipment operates the target AI function according to a third command.

24. The apparatus of claim 23, wherein prior to the obtaining capability information for each of at least one service device based on the first command, the processor is further configured to:

and receiving equipment information of the at least one service equipment, which is multicast through the SSDP, to obtain a service equipment list, wherein the equipment information comprises equipment names, internet Protocol (IP) addresses and port numbers.

25. The apparatus according to claim 23 or 24, wherein the first command is a capability information acquisition command of a limited application protocol COAP, the capability information acquisition command comprising an IP address, a port number and an information acquisition command field of a service device, the information acquisition command field being used for indicating that capability information of the service device is acquired, the processor being specifically configured to:

sending the capability information acquisition command to each service device with the device name meeting the requirement in the service device list;

and receiving the capability information returned by the at least one service device in response to the capability information acquisition command.

26. The apparatus according to claim 23 or 24, wherein the second command is a connection establishment command of COAP, the connection establishment command comprising a service device IP address, a port number, a connection establishment command field, and a load variable, the load variable comprising a data channel type, the service device IP address and a data channel port, the processor being specifically configured to:

And establishing a first data channel and a second data channel between the connection establishment command and the first service equipment, wherein the first data channel is used for sending data to the first service equipment by the information transmission device, the second data channel is used for receiving result data returned by the first service equipment by the information transmission device, and a data channel port is used for indicating a port of the first data channel.

27. The apparatus of claim 23 or 24, wherein the device comprises a plurality of sensors,

the third command is a capability calling command, the capability calling command includes a caller service ID, a capability calling command field and a subscription information message, the subscription information message includes a called capability name, the called capability name is a function name of the target AI function, the calling device includes a plurality of caller services, and the processor is specifically configured to:

transmitting at least one capability calling command for calling a service to the first service device, so that the first service device operates a corresponding target AI function according to the called capability names contained in the capability calling commands respectively;

The processor is specifically configured to:

and receiving result data obtained by processing target AI functions corresponding to the caller services in the first service equipment and respectively sending the result data to the caller services.

28. The apparatus of claim 27, wherein the subscription information message further comprises: a data source identifier, where the data source identifier is used to indicate that data to be processed by the target AI function of the first service device is from the first service device or a remote device, and the remote device includes the information transmission device.

29. The apparatus of claim 28, wherein when the data source identifier indicates that the data to be processed is from the calling device, the processor is further configured to: and sending the data to be processed to the first service equipment through a first data channel.

30. The apparatus of claim 23 or 24, wherein the processor is further configured to:

and sending upgrade data to the first service equipment, wherein the upgrade data is used for upgrading firmware in the first service equipment or AI functions which can be executed by the first service equipment.

31. The apparatus according to claim 23 or 24, wherein the processor is specifically configured to:

receiving a capability data return message returned by the first service equipment, wherein the capability data return message comprises: the system comprises at least one caller service ID, a function name and load data, wherein the function name is used for indicating an AI function name for processing the load data, and the at least one caller service ID is used for indicating at least one caller service for receiving the load data;

and sending the load data in the capability data return message to caller services respectively indicated by the at least one caller service ID.

32. A service apparatus having AI processing capability, the apparatus being a service device, the apparatus comprising: a processor and a transmission interface for receiving or transmitting data, the processor for invoking software instructions stored in the memory to perform the steps of:

transmitting capability information of the service device to a calling device in response to a first command transmitted by the calling device, wherein the capability information comprises AI functions which can be executed by the service device;

Establishing a connection channel with the calling equipment in response to a second command sent by the calling equipment;

responding to a third command of the calling equipment to operate a target AI function to obtain result data; the third command contains the function name of the target AI function, wherein the target AI function is the AI function which the calling device hopes to call remotely;

sending the result data to the calling device;

the processor is further configured to:

when the service device does not have the target AI function, receiving capability model data corresponding to the target AI function, which is sent by the calling device;

and carrying out capability upgrading on the service device according to the capability model data so that the service device has the target AI function.

33. The apparatus of claim 32, wherein the first command is a capability information acquisition command of a limited application protocol COAP, the capability information acquisition command including an IP address, a port number, and an information acquisition command field of a service device, the information acquisition command field indicating acquisition of capability information of the service apparatus.

34. The apparatus according to claim 32 or 33, wherein the processor is further configured to, prior to the sending of the capability information of the service apparatus to the calling device in response to the first command sent by the calling device:

After the service device is online, registering equipment information to the SSDP, wherein the equipment information comprises an equipment name, an IP address and a port number;

and multicasting the equipment information of the service device to a network where the service device is located through the SSDP, wherein the network where the service device is located comprises the calling equipment.

35. The apparatus according to claim 32 or 33, wherein the second command is a connection establishment command of COAP, the connection establishment command comprising a service device IP address, a port number, a connection establishment command field, and a load variable, the load variable comprising a data channel type, the service device IP address and a data channel port, the processor being specifically configured to:

and based on the connection establishment command and the calling equipment, a first data channel and a second data channel are established, wherein the first data channel is used for the service device to receive data sent by the calling equipment, the second data channel is used for the service device to send result data to the calling equipment, and a data channel port is used for indicating a port of the first data channel.

36. The apparatus of claim 32 or 33, wherein the third command is a capability call command, the capability call command including a caller service ID, a capability call command field, and a subscription information message, the subscription information message including a called capability name, the called capability name being a function name of the target AI function, the calling device including a plurality of caller services, the processor being specifically configured to:

Receiving a capability calling command of at least one caller service sent by the calling equipment;

operating the corresponding target AI function according to the called capability names respectively contained in the at least one capability caller command;

the processor is specifically further configured to:

and respectively sending the result data obtained by processing the target AI function corresponding to each of the at least one caller service to the at least one caller service.

37. The apparatus of claim 36, wherein the subscription information message further comprises: and the data source identifier is used for indicating that the data to be processed by the target AI function comes from the service device or remote equipment, and the remote equipment comprises the calling equipment.

38. The apparatus of claim 37, wherein when the data source identifier indicates that the data to be processed is from the calling device, the processor is further configured to:

and receiving the data to be processed sent by the calling equipment through a first data channel, wherein the first data channel is a connection channel which is established between the service equipment and the calling equipment based on the second command and is used for receiving the data sent by the calling equipment.

39. The apparatus of claim 32 or 33, wherein the processor is further configured to:

receiving upgrade data sent by the calling equipment;

and upgrading the firmware or the AI function which can be executed by the service device according to the upgrading data.

40. The apparatus of claim 32 or 33, wherein the processor is specifically configured to:

and sending a capability data return message to the calling equipment, wherein the capability data return message comprises the following components: the system comprises at least one caller service ID, a function name and load data, wherein the function name is used for indicating an AI function name for processing the load data, and the at least one caller service ID is used for indicating at least one caller service for receiving the load data in the calling equipment.

41. A computer readable storage medium storing a computer program, characterized in that the computer program comprises instructions for implementing the method of any of the preceding claims 1 to 11 or 12 to 22.