EP4594934A1

EP4594934A1 - Split neural network computing

Info

Publication number: EP4594934A1
Application number: EP23814329.1A
Authority: EP
Inventors: Jibing Wang; Erik Richard Stauffer
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2022-12-02
Filing date: 2023-11-01
Publication date: 2025-08-06
Also published as: KR20250088557A; WO2024118286A1; JP2026500480A; CN120153377A; DE112023005029T5

Abstract

A client device (104) and one or more such networked compute devices (112) coordinate to implement a client-initiated-and-devised split neural network compute scheme in which a neural network architecture of a neural network (116) is distributed, or "split", over the client device and the one or more networked compute devices such that each device operates to perform at least one contiguous split neural network portion (122, 124) of the overall neural network operation using input data (130), with intermediate result data (136) generated therefrom then communicated to the next device in sequence, until a final result output (140) is generated by the final split neural network portion in the sequence. The device generating the final result output can then take one or more actions based on the final result output or forward the final result output to the device that initiated the split neural network compute operation for further processing.

Description

SPLIT NEURAL NETWORK COMPUTING

BACKGROUND

[0001] User equipment, wearable devices, and other client devices in cellular networks and other wireless networks are increasingly employing neural networks to perform certain operations at various protocol layers for improved performance efficiency or enhanced user experiences. For example, a deep neural network (DNN) may be employed at a client device to provide for image analysis, for transmission-environment-aware radio frequency signaling, and the like. However, client devices often are resource constrained, which often makes it impracticable to implement a sufficiently-complex neural network to provide a desired result. For example, a user device may have insufficient battery power, insufficient compute resources, or insufficient network capacity to perform a DNN operation in an allotted time frame or at an intended complexity.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The present disclosure is better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.

[0003] FIG. 1 is a diagram illustrating an example wireless system employing a split neural network compute scheme for splitting a neural network compute load between two or more devices accordance with some embodiments.

[0004] FIG. 2 is a diagram illustrating an example of a split neural network configuration using a client-first implementation in accordance with some embodiments.

[0005] FIG. 3 is a diagram illustrating another example of a split neural network configuration using an interleaved configuration in accordance with some embodiments. [0006] FIG. 4 is a diagram illustrating an example hardware configuration of a user equipment of the wireless system of FIG. 1 in accordance with some embodiments.

[0007] FIG. 5 is a diagram illustrating an example hardware configuration of a server of the wireless system of FIG. 1 in accordance with some embodiments.

[0008] FIG. 6 is a diagram illustrating a machine learning (ML) module employing a split neural network portion in accordance with some embodiments.

[0009] FIG. 7 is a flow diagram illustrating an example method for split neural network computing in a wireless system in accordance with some embodiments.

[0010] FIG. 8 is a flow diagram illustrating an example of a process for negotiating a split neural network configuration between at least two devices in accordance with some embodiments.

[0011] FIG. 9 is a flow diagram illustrating an example for performing a machine learning operation using a neural network that is split over two or more devices in accordance with some embodiments.

[0012] FIG. 10 is a ladder signaling diagram illustrating an example operation of the method of FIGs. 7-9 in accordance with some embodiments.

DETAILED DESCRIPTION

[0013] Client devices often avoid implementing complex DNNs or other neural networks due to various resource constraints. However, the networks to which they are attached often have access to servers and other compute devices that are significantly less resource constrained. To leverage this, in at least one embodiment, a client device and one or more such networked compute devices coordinate to implement a split neural network compute scheme in which the neural network architecture of a neural network is distributed, or “split”, over the client device and the one or more networked compute devices such that each device operates to perform at least one contiguous portion of the overall DNN operation (hereinafter, a “split neural network portion” or simply “split portion”), with the intermediate result data generated therefrom then communicated to the next device in sequence, until the final result output is generated by the final split neural network portion in the sequence. The device generating the final result output can then take one or more actions based on the final result output or forward the final result output to the device that initiated the split neural network compute operation for further processing.

[0014] In at least one embodiment, this split neural network configuration is initiated by the client device. Accordingly, in response to initiation of an operation that involves the use of a neural network, one of the client device or the networked compute device determines a relevant set of present local conditions regarding the client device, such as its available compute resources (e.g., processors and memory), a present battery condition, present network conditions, a present thermal condition, and the like. Moreover, in some implementations, there may be execution requirements associated with the operation, such as a latency condition indicating a maximum time period permitted for generation of a final result output, a quality-of- service (QoS) condition indicating a maximum amount of time permitted to complete the operation, or an accuracy condition indicating a minimum level of accuracy or other performance level in the final result output. In the event that the client device determines that the latency condition, QoS condition, accuracy condition, or other such execution requirements can be met given the present local conditions of the client device, the client device may choose to perform the operation entirely locally; that is, to perform the operation using an entirely-local neural network architecture for the neural network.

[0015] However, in the event that one or more specified execution requirements appear unlikely to be met with a local-only execution, in at least some embodiments, either or both of client device or a networked compute device determines a split neural network configuration in which the neural network architecture of the neural network is apportioned, or “split”, between the client device and the networked compute device (or multiple networked compute devices). The particular split between devices can be based on the present local conditions, the execution requirements, or a combination thereof. For example, with relatively low local compute resource availability, a low battery reserve, and/or a short latency requirement, the split neural network configuration may apportion more layers of the neural network architecture to the one or more networked compute devices and fewer layers to the client device, whereas with relatively high local compute resource availability, a high battery reserve condition, and a long latency condition, the split neural network configuration may apportion more layers of the neural network architecture to itself and fewer layers to the one or more networked compute devices.

[0016] In some embodiments, the initiating device then initiates a negotiation with the other device(s) intended to be included in the split neural network configuration by sending a split compute request with the proposed split neural network configuration. The other device(s) then may consider their own resource constraints and the execution requirement(s) in determining whether to accept the proposed split neural network configuration or to counter-propose with a different split neural network configuration that is more compatible with resource constraints or the execution requirement(s). When the client device and one or more networked compute resources have settled on an agreed split neural network configuration, each of the client device and the one or more networked compute resources implements its corresponding portion of the neural network architecture (that is, its corresponding “split portion”) at a corresponding machine learning (ML) module. Thereafter, the device first in the sequence of the split neural network configuration receives input data for the operation, processes the input data at its ML module, and provides the output as intermediate result data to the device that has the next split portion of the neural network architecture in sequence, and so on.

[0017] In some implementations, the split is a two-way split, with one device (e.g., a networked compute device) having the first sequence of layers or other initial portion (e.g., “first stage”) of the neural network architecture and another device (e.g., the client device) having the last sequence of layers or other final portion (e.g., “last stage”) of the neural network architecture. In other implementations, the split is an N- way split (N>2), with each of the N stages being assigned to a corresponding device. In some embodiments, each stage is assigned to a different device, such as a first stage to one networked compute device, an intermediate stage to a second networked compute device, and a final stage to the client device. In other embodiments, the stages may be interleaved between a set of M devices (M<N). For example, the first and final stages may be apportioned to the client device, and an intermediate stage apportioned to a networked compute device. [0018] With this split compute approach to implementing a neural network operation, a client device can opportunistically utilize the resources of one or more available networked compute devices to aid in the performance of a neural network operation when the client device determines that its present local conditions may prevent a local-only solution from meeting one or more conditions or goals.

[0019] For ease of description, the systems and techniques are described herein in an example context of a cellular network in which a user equipment (UE) operates as the client device and one or more servers operate together as a networked compute device. However, these references are for illustrative purposes only, and it will be appreciated that reference to a UE, server, or cellular network also applies similarly to other client devices, other networked compute devices, and other networks, respectively, unless otherwise noted.

[0020] Moreover, for ease of illustration and to facilitate understanding, the technique of the present disclosure are described in example implementations in which the UE or other client device is the initiator of the division of a neural network via a split neural network configuration, and thus the device that triggers the negotiation process. However, in other embodiments, it may be a networked compute device that instigates split of a neural network between the networked compute device and the client device (and, in some instances, between one or more additional networked compute devices as well). As such, reference and description to initiation or negotiation of a split neural network configuration by a client device will be understood to apply to initiation or negotiation of a split neural network configuration by a networked compute device instead, using the guidelines provided herein.

[0021] FIG. 1 illustrates an example wireless communications network 100 employing a split neural network compute scheme in accordance with some embodiments. In the depicted example, the wireless communication network 100 is a cellular network including a network infrastructure 102 wirelessly connected to one or more client devices, such as UE 104. The network infrastructure 102 includes a core network 106 coupled to one or more wide area networks (WANs) 108 or other packet data networks (PDNs), such as the Internet. The core network 106 further is connected to at least one base station (BS) 110. The BS 110 supports wireless communication with one or more wireless client devices, such as UE 104, via radio frequency (RF) signaling using one or more applicable RATs as specified by one or more communications protocols or standards. As such, the BS 110 operates as a wireless interface between one or more wireless devices and various networks and services provided by the network infrastructure 102, such as packet-switched (PS) data services, circuit-switched (CS) services, and the like.

[0022] The BS 110 can employ any of a variety of RATs, such as operating as a NodeB (or base transceiver station (BTS)) for a Universal Mobile Telecommunications System (UMTS) RAT (also known as “3G”), operating as an enhanced NodeB (eNodeB) for a Third Generation Partnership Project (3GPP) Long Term Evolution (LTE) RAT, operating as a 5G node B (“gNB”) for a 3GPP Fifth Generation (5G) New Radio (NR) RAT, and the like. The UE 104, in turn, represents any of a variety of client devices operable to communicate with the BS 110 via a suitable RAT, including, for example, a mobile cellular phone, a tablet computer or laptop computer, a desktop computer, a video game system, a server, a network- enabled appliance, a network-enabled automotive communications system, a network-enabled smartwatch or other wearable device, and the like. The core network 106 and/or the WAN 108 includes, or has networked access to, one or more networked compute devices 112 that provide compute resources for the core network, the WAN 108, and/or the UE 104, such as one or more servers (or a server farm), one or more workstations, and the like. In the examples described herein, the one or more networked compute devices 112 include one or more servers, and thus the networked compute device 112 is also referred to herein as “server 112” for ease of reference.

[0023] In at least one embodiment, the UE 104 is configured to implement a machine learning (ML) module (see FIG. 6) that can be configured to implement one or more neural networks (or portions thereof) so as to provide neural network/machine learning capability to one or more software applications executing at the UE 104. Examples of such neural networks include deep neural networks (DNNs), such as convolutional neural networks (CNNs), artificial neural networks (ANNs), recurrent neural networks (RNNs), general adversarial networks (GANs), and the like. For example, a software application of the UE 104 may employ a ML module that implements a CNN to provide for image detection/categorization or implements an RNN to provide for speech detection. However, depending on the complexity or compute resources required to implement the entire neural network, the UE 104 may have insufficient resources to locally perform the neural network operation in a timely manner. For example, for real-time speech detection and translation, the UE 104 may have insufficient compute resources to perform the speech detection-and- translation operation rapidly enough to facilitate a satisfactory user experience. Alternatively or additionally, performance of the neural network operation entirely at the UE 104 may impose an unreasonable or unacceptable burden on the UE 104. To illustrate, performance of a sequence of image classification operations using a DNN at the UE 104 may cause excessive battery drain on the UE 104 or may induce a thermal load at the UE 104 that is difficult to manage.

[0024] Accordingly, in at least one embodiment, the UE 104 and the network infrastructure 102 together can implement a split neural network compute scheme in which a split neural network configuration 114 is used to implement a neural network 116 that is distributed, or “split”, over the UE 104 and one or more servers 112 or other networked compute devices of the network 100 such that each device in the split neural network configuration 114 operates to perform a portion of the overall neural network operation (that is, a “split neural network portion” or “split portion”), with the intermediate result generated therefrom then communicated to the next device in sequence, until the final result output is generated by the final split portion in the sequence. The split portion allocated to a corresponding device can include any subset or division of elements of the neural network employed by the split neural network configuration 114. To illustrate, many neural networks are configured as a sequence of layers, including an input layer, one or more hidden layers, and an output layer, with each layer having one or more nodes (e.g., neurons and/or perceptrons). In such cases, a split neural network portion may comprise all of the nodes of one or more layers of the neural network, a portion of the nodes of one or more layers of the neural network, or a combination thereof. In certain neural networks, the layers/nodes are further organized, and the split of the neural network into corresponding split neural network portions can occur in accordance with such organization. For example, a GAN typically is composed of a generator and a discriminator, and the split of a GAN across multiple devices may occur in view of this composition, such as with the layers and nodes associated with the generator allocated to one or more devices and the layers and nodes associated with the discriminator allocated to one or more other devices.

[0025] In implementations, the split of the neural network is performed so that the split neural network configuration 114 is implemented as a sequence of split neural network portions that represents the overall neural network, whereby each split neural network portion receives as its input either an initial input or an output from a previous split neural network portion in the sequence, and provides its output as either an input to the next split neural network portion in the sequence or as the final result output, depending on the position of the split neural network portion in the sequence. In a two-way split, a first device may be assigned, for example, the input layer and one or more hidden layers following the input layer, and a second device may be assigned, for example, the remaining hidden layers and the output layer. Which device is assigned which split portion may depend on a variety of factors, such as comparative compute resource availability, type of neural network operation being performed, use of the output, etc. For example, for an image classification operation with the image classification result to be used by an application at the LIE 104, the server 112 may be assigned the initial split portion with the input layer and the bulk of the hidden layers, while the UE 104 may be assigned the final split portion with the remaining hidden layers and the output layer. In an N-way split (N>2), each of M devices (M>1 ) is assigned a split portion, and in the event that N>M, at least one device is assigned multiple split portions.

[0026] As described below, in some embodiments the UE 104 operates to initiate the split of a neural network for performing a neural network operation, as well as determines how the split is to be performed, either unilaterally or as a result of negotiation with the network infrastructure 102. The decision of the UE 104 to split a neural network operation between the UE 104 and one or more servers 112 (or other networked compute devices) can be based on a variety of factors, including the scope or magnitude of operation to be performed, the local resources of the UE 104 available to perform the neural network operation, one or more execution requirements (e.g., latency, QoS, accuracy thresholds), network bandwidth or other network parameters, and the like. As a general guideline for some implementations, the portion of the neural network the UE 104 would seek to offload to one or more servers 112 is inversely proportional to the local resources available to the LIE 104 and/or the strictness of the execution requirement(s), everything else being equal. Note, however, that in other embodiments, the server 112 can instead initiate the split of a neural network using similar techniques as described below.

[0027] As the UE 104 proposes/determines this particular split, the UE 104 can inform the server 112 of its allocated split neural network portion(s) using any of a variety of techniques. In some embodiments, the server 112 (and the UE 104) has access to a repository 118 that contains representations 120 of neural networks, and the UE 104 may inform the server 112 of those split neural network portion(s) allocated to the server 112, along with other relevant details, such as initial weights, the manner in which the split neural network portion is to be implemented at the server 112, etc. For example, each representation 120 may be identified by a corresponding identifier, which is then referenced by the UE 104 and server 112. The representation 120 may refer to a particular implementation of a type of neural network (e.g., a RNN with a particular number of layers, nodes, weights, etc.), and the UE 104 and server 112 utilize additional information to implement a split of this particular implementation into the decided split portions, or the representation 120 may represent a pre-determined split configuration template, such as an RNN with a particular number of layers, nodes, weights, etc., as well as a pre-determined split of these elements between a plurality of split neural network portions. Although illustrated as a separate component (e.g., a separate server in the network infrastructure 102), in some embodiments the repository 118 may be implemented, in whole or in part, at the server 112. In other embodiments, the UE 104 may provide the server 112 with information that specifies the manner in which the server 112 is to implement a corresponding split neural network portion, such as one or more data structures representing the structure of the split neural network portion, its input format, its output format, the weights or other parameters for the different nodes in the structure, and the like.

[0028] As an illustration of this split neural network compute scheme, FIG. 1 depicts the example split neural network configuration 114 for the neural network 116 in which the neural network 116 is split into two portions, split neural network portions 122, 124, with the split neural network portion 122 being allocated an input layer and the first five hidden layers of the example neural network 116 (and thus referred as the “initial split neural network portion 122) and the split neural network portion 124 being allocated the last two hidden layers and the output layer of the example neural network 116 (and thus referred to as the “final split neural network portion 124”). With this configuration, the neural network 116 is split into the sequence: initial neural network portion 122 final neural network portion 124. For this example, assume that the UE 104 negotiated this split with the server 112 based on the current local resources available to the UE 104, the execution requirement(s) for the operation to be performed (e.g., ML-assisted image classification, speech detection, etc.), and as a result of negotiation with the server 112, using the techniques described below. Further, for this implementation, the initial split neural network portion 122 is assigned to the server 112 and the final split neural network portion 124 is assigned to the UE 104. As such, the initial split neural network portion 122 is implemented at a server ML module 126 at the server 112 and the final split neural network portion 124 is implemented at a UE ML module 128 at the UE 104.

[0029] To initiate an ML-assisted operation using this example configuration, a software application of the UE 104 generates input data 130-1 for input to the neural network 116. For example, if the ML-assisted operation is image classification, the input data 130-1 can include, for example, an image captured by a camera of the UE 104. As another example, if the ML-assisted operation is speech-detection-and- translation, the input data 130-1 can include, for example, speech captured and digitized by a microphone of the UE 104. A radio frequency (RF) interface 132 of the UE 104 transmits the input data (as input data 130-2) to a corresponding RF interface 134 of the BS 110, which then transmits the input data (as input data 130-3) to the server 112. At the server 112, the input data 130-3 is provided as an input to the server ML module 126, which processes the input data 130-3 at the input layer and first five hidden layers of the neural network 116 in accordance with its allocated initial split neural network portion 122. The outputs 136 of the nodes of the fifth hidden layer thus serve as the intermediate result data 138-1 of the initial split neural network portion 122 (and thus the output of the server ML module 126). As such, the server 112 transmits the intermediate result data (as intermediate result data 138-2) to the BS 110, which in turn wirelessly transmits the intermediate result data 138-2 to the UE 104. [0030] The LIE 104 provides the intermediate result data (as intermediate result data 138-3) as an input to the UE ML module 128, which implements the final split neural network portion 124. As such, the outputs 136 represented in the intermediate result data 138-3 are provided as inputs to the corresponding nodes of the sixth hidden layer of the neural network 116, and processed by the sixth, seventh, and eighth hidden layer and output layer to generate a final result output 140, which represents the final result output by the neural network 116 given the intermediate result data 138-3 as input. The software application of the UE 104 that initiated the ML-assisted operation and/or another software application of the UE 104 then may perform one or more actions in response to the final result output 140. For example, if the ML- assisted operation is a speech-detection-and-translation operation, the final result output 140 may represent a translated representation of the input speech, and thus a software application may manipulate the UE 104 to output an audio version of the translated representation via a speaker of the UE 104 and/or to display a textual representation via a display of the UE 104.

[0031] Although FIG. 1 depicts a particular two-way split of a neural network 116 that allocates an initial portion to the server 112 and a final portion to the UE 104, the split of any given neural network can be implemented in a variety of ways, depending on needs and context. For example, the UE 104 may implement the initial portion of a split neural network configuration while a server 112 implements the final portion.

FIG. 2 illustrates an example of this, with a split neural network configuration 214 that splits a neural network 216 into an initial split portion 202 implemented at a UE ML module 228 of the UE 104 and a final split portion 204 implemented at a server ML module 226 of the server 112. With this approach, a software application at the UE 104 provides input data 230 as input to the UE ML module 228, whereby the initial layer and the first subset of hidden layers of the neural network 216 process the input data 230. The outputs 236 of the last hidden layer in the initial split portion 202 are provided as an intermediate result output 238 that is transmitted to the server 112 via the base station 110. The server 112 provides the intermediate result output 238 as an input to the first hidden layer in the final split portion 204, whereby this hidden layer, the subsequent hidden layers, and the output layer process the intermediate result output 238 to generate a final result output 240. The final result output 240 then may be transmitted back to the UE 104 for use by one or more software applications at the UE and/or the final result output 240 may be used by the server 112 or some other component of the network infrastructure 102 to perform one or more remote actions 242. For example, the neural network 116 may be used to generate augmented display information for an augmented reality (AR) display, and thus the one or more remote actions 242 may be the display of the generated augmented display information (one result of the final result output 240) at the AR display. As another example, the neural network 116 may be used to generate virtual reality (VR) content, such as VR display content orVR audio content (one example of the final result output 240), and the one or more remote actions 242 may be to display or output the generated VR content. As yet another example, the neural network 116 may be used to improve or enhance the cellular connectivity between the UE 104 and the BS 110 (e.g., by controlling some aspect of the UE’s RF antenna array based on input sensor data) and thus the one or more remote actions 242 could include, for example, modifying the operation of the RF antenna array based on the final result output 240.

[0032] Further, as described above, in some implementations the split configuration may be an N-way split (N>2), with each of M devices getting at least one split neural network configuration. FIG. 3 illustrates an example of such, in which a split neural network configuration 314 splits a neural network 316 into three split portions: an initial split portion 302 implemented at a UE ML module 328 of the UE 104, an intermediate split portion 304 implemented at a server ML module 326 of the server 112, and a final split portion 306 implemented at the UE ML module 328 of the UE 104 (or at a second instance of a UE ML module of the UE 104). In this example, a software application at the UE 104 provides input data 330 as input to the UE ML module 228 for processing by the initial split portion 302. The outputs 336-1 of the last hidden layer in the initial split portion 302 are provided as a first intermediate result output 338-1 that is transmitted to the server 112 via the base station 110. The server 112 provides the first intermediate result output 338-1 as an input to the first hidden layer in the intermediate split portion 304, whereby this hidden layer and subsequent hidden layers of the intermediate split portion 304 process the first intermediate result output 338-1 . The outputs 336-2 of the last hidden layer in the intermediate split portion 304 are provided as a second intermediate result output 338-2 that is transmitted to the UE 104 via the base station 110. The UE 104 then provides the second intermediate result output 338-2 as an input to the first hidden layer in the final split portion 306, and this hidden layer, the subsequent hidden layers, and the output layer in the final split portion 306 process this input to generate a final result output 340. The final result output 340 then may be processed by one or more software applications at the UE 104, in response to which one or more actions are taken at the UE 104.

[0033] FIG. 4 illustrates example hardware configurations for the UE 104 (as a representative client device) in accordance with some embodiments. Note that the depicted hardware configuration represents the processing components and communication components most directly related to the neural network-based processes described herein and omits certain components well-understood to be frequently implemented in such electronic devices, such as displays, peripherals, external power supplies, and the like.

[0034] In the depicted configuration, the UE 104 includes an RF interface 402 having one or more antennas 404 and one or more modems to support one or more radio access technologies (RATs), such as a Third Generation Partnership Project (3GPP) Fourth Generation Long Term Evolution (4G LTE)-compliant RAT, a 3GPP Fifth Generation New Radio (5G NR)-compliant RAT, an Institute of Electrical and Electronic Engineers (IEEE) 802.11-compliant RAT, and the like. The UE 104 further includes one or more processors 406, a set 408 of sensors, a user interface (Ul) 410, and one or more batteries 412 or other power sources. The one or more processors 406 can include, for example, one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs) or other application-specific integrated circuits (ASIC), and the like. To illustrate, the processors 406 can include an application processor (AP) utilized by the UE 104 to execute an operating system and various user-level software applications, as well as one or more processors utilized by modems or a baseband processor of the RF interface 402. The set 408 of sensor can include, for example, satellite positioning sensors such as Global Positioning System (GPS) sensors, Global Navigation Satellite System (GNSS) sensors, internal measurement unit (IMU) sensors, visual odometry sensors, accelerometers, gyroscopes, barometers, altimeters, tilt sensors or other inclinometers, ultrawideband (UWB)-based sensors, and the like. Other examples of types of sensors of the sensor set 408 can include sensors for determining a present operating status of the UE 104, such as battery level sensors, thermal sensors, screen mode sensors, and the like. The III 410 includes components for interfacing with a user, such as a display, a keyboard or touch panel, buttons, a microphone, a speaker, and the like.

[0035] The UE 104 further includes one or more computer-readable media 414 that include any of a variety of media used by electronic devices to store data and/or executable instructions, such as random-access memory (RAM), read-only memory (ROM), caches, Flash memory, solid-state drive (SSD) or other mass-storage devices, and the like. For ease of illustration and brevity, the computer-readable media 414 is referred to herein as “memory 414” in view of the frequent use of system memory or other memory to store data and instructions for execution by the processor 406, but it will be understood that reference to “memory 414” shall apply equally to other types of storage media unless otherwise noted. The one or more memories 414 of the UE 104 are used to store one or more sets of executable software instructions and associated data that manipulate the one or more processors 406 and other components of the UE 104 to perform the various functions described herein and attributed to the UE 104. The sets of executable software instructions include, for example, an operating system (OS) 416 and various drivers (not shown) and one or more user-level software applications 418 that manipulate, or otherwise interact with, the hardware of the UE 104 through the OS 416.

[0036] The sets of executable software instructions further include one or more of a neural network management module 420 and a split compute management module 422. The neural network management module 420 implements one or more ML modules (e.g., ML module 128, FIG. 1) that utilize one or more neural networks (or split portions thereof) for the UE 104, as described in detail below. The split compute management module 422 operates to monitor the statuses of local resources of the UE 104, such as battery status, available memory status, available processor resources, network status, and the like, and based on these local resource statuses (and/or other considerations), determine whether to implement a neural network as a local-only neural network (that is, entirely at the UE 104) or as a split neural network configuration for performing one or more operations using the neural network. Moreover, in the event that a split neural network configuration is to be employed, the split compute management module 422 can operate to manage the negotiation for implementation of split portions by one or more servers 112 of the network infrastructure 102, as well as coordinate with the neural network management module 420 to implement the split portion(s) allocated to the UE 104 in the negotiated split neural network configuration.

[0037] To facilitate the operations of the UE 104 as described herein, the one or more memories 414 of the UE 104 further can store data associated with these operations. This data can include, for example, one or more neural network architectural configurations 424, as well as device data (not shown) that includes, for example, user data, multimedia data, beamforming codebooks, software application configuration information, and the like. Each neural network architectural configuration 424 includes one or more data structures containing data and other information representative of a corresponding architecture and/or parameter configurations used by the neural network management module 420 to form a corresponding neural network, or split portion thereof, for the UE 104. The information included in a neural network architectural configuration 424 includes, for example, parameters that specify a fully connected layer neural network architecture, a convolutional layer neural network architecture, a recurrent neural network layer, a number of connected hidden neural network layers, an input layer architecture, an output layer architecture, a number of nodes utilized by the neural network, coefficients (e.g., weights and biases) utilized by the neural network, kernel parameters, a number of filters utilized by the neural network, strides/pooling configurations utilized by the neural network, an activation function of each neural network layer, interconnections between neural network layers, neural network layers to skip, and so forth. Accordingly, the neural network architectural configuration 424 includes any combination of neural network formation configuration elements (e.g., architecture and/or parameter configurations) that can be used to create a neural network architectural configuration (e.g., a combination of one or more neural network formation configuration elements) that defines and/or forms a DNN or other neural network. [0038] FIG. 5 illustrates example hardware configurations for a server 112 (as representative networked compute resource) in accordance with some embodiments. Note that the depicted hardware configuration represents the processing components and communication components most directly related to the neural network-based processes described herein and omits certain components well-understood to be frequently implemented in such electronic devices. Further note that although the illustrated diagram represents an implementation as a single server, the functionality, and thus the hardware components, instead may be distributed across multiple servers or other networked compute devices and may be distributed in a manner to perform the functions described herein. Thus, reference to the functionality of a single server 112 may also apply to equivalent functionality in a plurality of servers 112 or other networked compute devices, unless otherwise noted.

[0039] In the depicted configuration, the server 112 includes a network interface 502 for wired and/or wireless connections to the BS 110 and other components of the network infrastructure 102. The server 112 further includes one or more processors 506, such as, for example, one or more CPUs, GPUs, TPUs or other ASICs, and the like. The server 112 further includes one or more computer-readable media 508 that include any of a variety of media used by electronic devices to store data and/or executable instructions, such as RAM, ROM, caches, Flash memory, SSD or other mass-storage devices, and the like. As with the memory 414 of the UE 104, for ease of illustration and brevity, the computer-readable media 508 is referred to herein as “memory 508” in view of the frequent use of system memory or other memory to store data and instructions for execution by the processor 506, but it will be understood that reference to “memory 508” shall apply equally to other types of storage media unless otherwise noted.

[0040] The one or more memories 508 of the server 112 are used to store one or more sets of executable software instructions and associated data that manipulate the one or more processors 506 and other components of the server 112 to perform the various functions described herein and attributed to the server 112, individually or as a set of servers 112. The sets of executable software instructions include, for example, an OS 510 and various drivers (not shown), and various software applications. The sets of executable software instructions further include one or more of a neural network management module 512 and a split compute management module 514. As with the neural network management module 420 and split compute management module 422 of the UE 104, these modules 512, 514 operate to implement a neural network, or split portion(s) thereof, at the server 112. In particular, the neural network management module 512 implements one or more ML modules (e.g., ML module 126, FIG. 1) that utilize one or more neural networks (or split portions thereof) for the server 112, while the split compute management module 514 operates to negotiate with the UE 104 for implementation of split portions at the server 112, as well as coordinate with the neural network management module 512 to implement the split portion(s) allocated to the server 112 in the negotiated split neural network configuration.

[0041] The one or more memories 508 further store various information, such as one or more neural network architectural configurations 524 that represent trained neural network architectural configurations that can be employed at a ML module (e.g., ML module 126, FIG. 1) of the server 112. Thus, as with the neural network architectural configurations 424 of FIG. 4, each neural network architectural configuration 524 includes one or more data structures containing data and other information representative of a corresponding architecture and/or parameter configurations used by the neural network management module 512 of the server 112, to form corresponding portion(s), or the entirety of, a neural network.

[0042] FIG. 6 illustrates an example machine learning (ML) module 600 for implementing a neural network, or split portion(s) thereof, in accordance with some embodiments. As noted herein, both the UE 104 and the server 112 each implements one or more ML modules to implement one or more split neural network portions. The ML module 600 thus illustrates an example module for implementing one or more of these split neural network portions (or entire neural network).

[0043] In the depicted example, the ML module 600 implements at least a portion 602 of a deep neural network (DNN) model with groups of connected nodes (e.g., neurons and/or perceptrons) that are organized into one or more layers. The nodes between layers are configurable in a variety of ways, such as a partially connected configuration where a first subset of nodes in a first layer are connected with a second subset of nodes in a second layer, a fully-connected configuration where each node in a first layer is connected to each node in a second layer, etc. A neuron processes input data to produce a continuous output value, such as any real number between 0 and 1 . In some cases, the output value indicates how close the input data is to a desired category. A perceptron performs linear classifications on the input data, such as a binary classification. The nodes, whether neurons or perceptrons, can use a variety of algorithms to generate output information based upon adaptive learning. Using the portion 602, the ML module 600 performs a variety of different types of analysis, including single linear regression, multiple linear regression, logistic regression, stepwise regression, binary classification, multiclass classification, multivariate adaptive regression splines, locally estimated scatterplot smoothing, and so forth.

[0044] When the portion 602 includes the entire DNN model, the two or more layers include an input layer, one or more hidden layers, and an output layer. When the portion 602 includes a split portion of the DNN model, the layers included in the portion 602 depend on the position of the split portion in the sequence of split portions that together compose the DNN model. In the event that the portion 602 is an initial split portion, the portion 602 includes an initial layer and one or more adjacent hidden layers that immediately follow the initial layer. In the event that the portion 602 is a final split portion, the portion 602 includes the output layer and one or more adjacent hidden layers that immediately precede the output layer. In the event that the portion 602 is an intermediate split portion, the portion 602 includes one or more adjacent hidden layers.

[0045] For ease of illustration, the depicted portion 602 of a DNN model includes three layers 604, 606, and 608, which may be implemented as, for example, an input layer and the two adjacent hidden layers following the input layer for an initial split portion implementation, as three adjacent hidden layers for an intermediate split portion implementation, or as the output layer and the two adjacent hidden layers preceding the output layer for a final split portion implementation. However, in actual implementation, the split portion implemented by the ML module 600 typically would have a larger number of layers. Each layer has an arbitrary number of nodes, where the number of nodes between layers can be the same or different. That is, the input layer can have the same number and/or a different number of nodes as the output layer, the output layer can have the same number and/or a different number of nodes than the one or more hidden layers, and so forth. For example, node 610 corresponds to one of several nodes included in an input layer (which may be represented by layer 604), wherein the nodes perform separate, independent computations. As further described, a node receives input data and processes the input data using one or more algorithms to produce output data. Typically, the algorithms include weights and/or coefficients that change based on adaptive learning. Thus, the weights and/or coefficients reflect information learned by the neural network. Each node can, in some cases, determine whether to pass the processed input data to one or more next nodes. To illustrate, after processing input data, node 610 can determine whether to pass the processed input data to one or both of node 612 and node 614 of the adjacent hidden layer (layer 608 in this example). Alternatively or additionally, node 610 passes the processed input data to nodes based upon a layer connection architecture. This process can repeat throughout multiple layers until the portion 602 generates an output using the nodes (e.g., node 616) of the final layer (layer 608) in the portion 602. To facilitate the input of received data and the output of generated output data, the ML module 600 further can include an input interface 618 and an output interface 620. The input interface 618 operates to receive an input data 622 and distribute the individual datum contained in the input data 622 to the corresponding nodes of the first layer 604 according to some predetermined format. Similarly, the output interface 620 operates to receive the individual outputs of the nodes of the final layer 608 and provide them as output data 624 according to some predetermined format. The formats used for the input interface 618 and the output interface 620 depend on the neural network employed and the layers of the neural network implemented in the portion 602. For example, if an initial split portion, the input data 622 represents the initial data that is input for processing by the neural network, and thus input interface 618 operates to distribute the initial data to the nodes of the input layer (layer 604) accordingly. However, for an intermediate or final split portion, the input data 622 represents the outputs from an upstream hidden layer, and thus the input data 622 represents intermediate result data and the input interface 618 is configured to distribute the individual datum of the intermediate result data as inputs to the nodes of the first hidden layer (layer 604) according to the neural network architecture. Similarly, for an initial or intermediate split portion, the output data 624 is the individual outputs of the nodes of the last hidden layer (layer 608) in the ML module 600 and thus represents intermediate result data, whereas for a final split portion, the output data 624 is a final result output provided by the output layer (layer 608) of the neural network.

[0046] A neural network can also employ a variety of architectures that determine what nodes within the neural network are connected, how data is advanced and/or retained in the neural network, what weights and coefficients are used to process the input data, how the data is processed, and so forth. These various factors collectively describe a neural network architectural configuration, such as the neural network architectural configurations briefly described above. To illustrate, a recurrent neural network, such as a long short-term memory (LSTM) neural network, forms cycles between node connections to retain information from a previous portion of an input data sequence. The recurrent neural network then uses the retained information for a subsequent portion of the input data sequence. As another example, a feed-forward neural network passes information to forward connections without forming cycles to retain information. While described in the context of node connections, it is to be appreciated that a neural network architectural configuration can include a variety of parameter configurations that influence how the portion 602 or other neural network processes input data.

[0047] A neural network architectural configuration of a neural network can be characterized by various architecture and/or parameter configurations. To illustrate, consider an example in which the portion 602 implements a portion or the entirety of a CNN. Generally, a convolutional neural network corresponds to a type of DNN in which the layers process data using convolutional operations to filter the input data. Accordingly, the CNN architectural configuration can be characterized by, for example, pooling parameter(s), kernel parameter(s), weights, and/or layer parameter(s).

[0048] A pooling parameter corresponds to a parameter that specifies pooling layers within the convolutional neural network that reduce the dimensions of the input data. To illustrate, a pooling layer can combine the output of nodes at a first layer into a node input at a second layer. Alternatively or additionally, the pooling parameter specifies how and where in the layers of data processing the neural network pools data. A pooling parameter that indicates “max pooling,” for instance, configures the neural network to pool by selecting a maximum value from the grouping of data generated by the nodes of a first layer, and uses the maximum value as the input into the single node of a second layer. A pooling parameter that indicates “average pooling” configures the neural network to generate an average value from the grouping of data generated by the nodes of the first layer and uses the average value as the input to the single node of the second layer.

[0049] A kernel parameter indicates a filter size (e.g., a width and a height) to use in processing input data. Alternatively or additionally, the kernel parameter specifies a type of kernel method used in filtering and processing the input data. A support vector machine, for instance, corresponds to a kernel method that uses regression analysis to identify and/or classify data. Other types of kernel methods include Gaussian processes, canonical correlation analysis, spectral clustering methods, and so forth. Accordingly, the kernel parameter can indicate a filter size and/or a type of kernel method to apply in the neural network. Weight parameters specify weights and biases used by the algorithms within the nodes to classify input data. In some implementations, the weights and biases are learned parameter configurations, such as parameter configurations generated from training data. A layer parameter specifies layer connections and/or layer types, such as a fully-connected layer type that indicates to connect every node in a first layer (e.g., output layer 608) to every node in a second layer (e.g., hidden layer 606), a partially-connected layer type that indicates which nodes in the first layer to disconnect from the second layer, an activation layer type that indicates which filters and/or layers to activate within the neural network, and so forth. Alternatively or additionally, the layer parameter specifies types of node layers, such as a normalization layer type, a convolutional layer type, a pooling layer type, and the like.

[0050] While described in the context of pooling parameters, kernel parameters, weight parameters, and layer parameters, it will be appreciated that other parameter configurations can be used to form a DNN consistent with the guidelines provided herein. Accordingly, a neural network architectural configuration can include any suitable type of configuration parameter that can be applied to a DNN that influences how the DNN processes input data to generate output data.

[0051] Turning now to FIGs. 7-10, a method 700 for implementing a split neural network configuration for a neural network used to perform an ML operation on behalf of a software application or other process of a UE or other client device is illustrated in accordance with some embodiments. For purposes of illustration, the method 700 and accompanying examples are described in the example context of the wireless communications network 100 with the UE 104 operating as a client device and the server 112 representing one or more cooperating networked compute devices, but this method is not limited to this particular implementation context. Further, a ladder diagram 1000 of FIG. 10 is referenced below inline with the description of method 700 of FIGs. 7-9 as an example of the operation of method 700 for an example 2-way split neural network configuration to facilitate understanding. Moreover, note that the order of operations described with reference to method 700 is for illustrative purposes only, and that a different order of operations may be performed, and further that one or more operations may be omitted or one or more additional operations included in the illustrated method.

[0052] As explained above, in implementations, a software application of the UE 104 (e.g., a user-level application 418 or the OS 416 or other kernel/driver-level process) utilizes a neural network to perform a corresponding operation, the results of which are utilized by the software application or another software application to perform one or more actions. In instances where the UE 104 has sufficient resources to implement the neural network entirely locally while meeting corresponding goals (e.g., a maximum latency goal or a battery consumption goal), the UE 104 may implement the entire neural network at a ML module of the UE 104 and perform the operation using this ML module so configured. However, in the event that the UE 104 cannot implement the neural network entirely locally while meeting one or more specified goals, the UE 104 instead may seek to implement the neural network as a split neural network configuration that leverages the compute resources of one or more servers 112 of the network infrastructure 102.

[0053] Accordingly, the method 700 initiates at block 702 with a software application of the UE 104 initiating the performance of a ML operation (block 1002, FIG. 10) that utilizes an identified neural network to generate an output result on the basis of input data provided by, or referenced by, the software application. To use an example from above, this operation can be an image classification operation using a particular CNN, with the input to the CNN the image data representing the image to be classified. The initiation of the ML operation can include, for example, the software application issuing a call to an application programming interface (API) of the OS 416 for providing support to such operations. Concurrently, the split compute management module 422 monitors the statuses of local resources of the UE 104 (block 1004, FIG. 10), such as available memory, processor utilization, remaining battery capacity, current device usage, and other indications of current or near-future local resource availability. Thus, in response to initiation of the ML operation, at block 704 the split compute management module 422 determines the present conditions of the UE 104 based on these monitoring operations. Further, the split compute management module 422 identifies any execution requirements pertaining to the ML operation, such as an execution latency limit, an execution power consumption limit, a compute resource consumption limit, and the like.

[0054] At block 706, the split compute management module 422 then determines whether a local-only implementation of the identified CNN or a split configuration is better suited to the determined present conditions and any execution requirements. For example, when there are sufficient local power and compute resources available at the UE 104, the time limit for generating the final result output is relatively long, and the wireless network is bandwidth constrained, the split compute management module 422 may determine to implement the neural network entirely locally, and thus perform the ML operation solely at the UE 104. However, if there are scarce compute or power resources available to the UE 104, there is a strong wireless connection with the BS 110, and there is a relatively short time limit, the split compute management module 422 may determine to implement the neural network as a split configuration where one or more portions of the neural network are allocated to the UE 104 and one or more other portions of the neural network are allocated to the server 112 (or to multiple servers 112). This determination may be made in any of a variety of ways. For example, the determination may be implemented algorithmically (e.g., via a weighted sum equation), via a look-up table (LUT), via another, less- resource intensive trained neural network, and the like. To illustrate, the UE 104 may model the performance of a proposed split configuration to determine whether it meets a UE-side power consumption limit while also meeting a QoS (e.g., latency) requirement. If so, the UE 104 may select the proposed split configuration, and if not, the UE 104 may select another proposed split configuration for similar modeling.

[0055] In the event that the split compute management module 422 determines that a local-only implementation of the neural network is suitable or sufficient, then at block 708 the split compute management module 422 directs the neural network management module 420 to implement a ML module configured to implement the entire architectural configuration of the identified neural network, and the UE 104 performs the ML operation using this ML module.

[0056] Returning to block 706, in the event that the split compute management module 422 determines that a split configuration of the neural network is appropriate (block 1006, FIG. 10), then at block 710, the split compute management module 422 performs a negotiation process to determine a split of the neural network between the UE 104 and the server 112 (or multiple servers 112) that is acceptable to the UE 104 and the server(s) 112 given their respective resource constraints.

[0057] Turning briefly to FIG. 8, an example implementation of the negotiation process of block 710 is illustrated in accordance with some embodiments. In this example approach, the negotiation process follows the determination to implement a split configuration with the split compute management module 422 determining or identifying, at block 802, the particular neural network architecture to be implemented for performing the ML operation. This can include determining the type of neural network (e.g., CNN, RNN, ANN, GAN, etc.), as well as the parameters of the neural network type, such as the number of layers, the number of nodes for each layer, the operations performed at each node, the weights and other parameters of each node, the connections between nodes, etc. Recall further that while the following describes a negotiation process for a client-initiated split, in other embodiments it may be the server 112 that initiates the split, in which case a similar process may be implemented, but with the roles of the server 112 and the UE 104 reversed as to initiator and responder. [0058] With the neural network architecture identified, at block 804 the split compute management module 422 then determines a proposed split configuration for the neural network architecture based on the present UE conditions, execution requirements, etc. As with the determination of whether to split the neural network, this proposed split configuration can be determined or selected algorithmically, via a LUT or other selection structure, via a smaller neural network, and the like. In some embodiments, a plurality of predetermined split options are available, and the split compute management module 422 selects one of the pre-determined split options as the proposed split configuration. For example, one split option may be a two-way split that divides the hidden layers evenly between the server 112 and the UE 104, another split option may be a two-way split that apportions a larger share of the hidden layers to the server 112 than to the UE 104, and yet another split option may be a three-way split that apportions the initial and final split portions with small proportions of the hidden layers to the UE 104 and an apportions the intermediate split portion with the majority of the hidden layers to the server 112. In this example, the UE 104 can select the split option that best matches the present UE conditions while meeting the indicated execution requirements. In other embodiments, the split compute management module 422 may dynamically determine a proposed split, such as by allocating a number of layers to the portion allocated to the server 112 as inversely proportional to the available compute resources of the UE 104, to use a simple example. With a proposed split identified, the UE 104 then transmits a split request 1008 (FIG. 10) to the server 112 via the BS 110. The split request 1008 can include a description or identifier of the proposed split (e.g., a data structure identifying the relevant parameters that sufficiently describe the split portion the UE 104 is proposing for the server 112 to implement, or an identifier to a predefined split option available from the repository 118 as representations 120 of a neural network). The split request 1008 may also include information pertinent to the requested split proposal, such as the present conditions of the UE 104 and/or the execution requirements that led to the particular proposed split, and the like. Additionally or alternatively, the server 112 may obtain some or all of the current conditions of the UE 104 from the most recent Radio Resource Control (RRC) UE Capabilities Information message transmitted by the UE 104 to the BS 110 in response to an RRC UE Capabilities Enquiry message from the BS 110. [0059] At block 806, the split compute management module 514 of the server 112 (or the BS 110 or other network component acting as an intermediary for the server 112) evaluates the proposed split represented by the split request 1008 to determine whether to accept the proposed split or counter-propose with a modified proposed split. The split compute management module 514 may consider the server’s own present resource constraints and the indicated execution requirements when determining whether to accept the proposed split. For example, in the event that the server 112 is able to dedicate the resources needed to implement the split neural network portion allocated to the server 112 under the proposal and will be able to meet the indicated execution requirements with the proposed split, the split compute management module 514 may accept the proposed split and indicate this acceptance at block 808 by transmitting a split compute grant message 1010 (FIG. 10) to the UE 104 via the BS 110.

[0060] Returning to block 806, in the event that the split compute management module 514 determines that the server 112 cannot allocate the compute resources needed to implement the proposed split or cannot meet the indicated execution requirements given the proposed split, then at block 810 the split compute management module 514 may determine a split counter-proposal that proposes a modified split configuration for the neural network. For example, if the server 112 cannot dedicate sufficient resources for the original proposed split, the split compute management module 514 may counter-propose with a counter-proposal split that allocates fewer layers of the neural network architecture to the server 112. As another example, if the server 112 can allocate sufficient resources but the split compute management module 514 determines that a latency requirement or other execution requirement cannot be met given the original proposed split, the split compute management module 514 may counter-propose a modified split that makes meeting the latency requirement or other execution requirement more likely to be met. The server 112 then transmits a representation of the split counter-proposal to the UE 104. At block 812, the split compute management module 422 of the UE 104 evaluates the counter-proposal and, if acceptable, at block 814 transmits an acceptance message to the server 112, in response to which the server 112 issues the split compute grant message 1010. If the counter-proposal is not acceptable, then at block 816 the UE 104 may terminate the negotiation, and fall back to either attempting the perform the ML operation entirely locally (that is, by implementing the entire neural network at an ML module of the UE 104) or by returning a no-operation (NOP) or error message to the initiating software application to indicate the ML operation cannot be performed under present conditions. Alternatively, in some embodiments, one or more rounds of counter-counter-proposals for the split may be conducted between the UE 104 and the server 112 until a mutually-acceptable split is identified or a threshold number of counter-proposals is transmitted.

[0061] Returning to FIG. 7, with a proposed split configuration agreed to by the server 112 and the UE 104, the server 112 and UE 104 proceed to implement their respective split portion(s). In order to do so, however, the server 112 and UE 104 need the neural network architectural details (e.g., layers, nodes, connections, weights, and other parameters) for each split portion to be implemented as a corresponding ML module. In some instances, the negotiation process provides the distribution of this information. For example, when the negotiation process results in mutual agreement to employ a predetermined split neural network configuration from, for example, the repository 118, the split portion implementation details may either be expressly included in the negotiation messaging or subsequently accessible from the repository 118 by either or both of the server 112 or the UE 104 once an agreed- upon split neural network configuration is determined. Alternatively, when the split neural network configuration is determined ad-hoc by the UE 104 or a counterproposal split from the server 112 is accepted by the UE 104, the implementation details for the split portion(s) to be implemented by the server 112 and/or the UE 104 can be included in the negotiation messaging (e.g., in the split request 1008 or the grant message 1010, respectively). However, in some embodiments, the negotiation may involve a negotiation of the overall split, but without particular details, such as weights or other parameters for specific nodes or for specific layers. In such cases, as represented by block 712, the UE 104 transmits one or more configuration messages to the server 112 via the BS 110, the one or more configuration messages including the implementation details for the split portion(s) of the neural network to be implemented by the server 112 (hereinafter, “server-side split portion(s)”) in accordance with the negotiated split configuration. With the implementation details in place, at block 714 the UE 104 configures one or more of ML modules (e.g., UE ML module 128, FIG. 1) to implement the one or more split portions allocated to the UE 104 (hereinafter, the “UE-side split portions”) and at block 716 the server 112 configures one or more of ML modules (e.g., server ML module 126, FIG. 1) to implement the one or more server-side split portions. With the UE 104 and server 112 configured to implement their respective split neural network portions of the agreed-upon split neural network configuration for the neural network, at block 718 the server 112 and UE 104 operate together to perform a split-compute ML operation via the neural network distributed between the server 112 and UE 104.

[0062] FIG. 9 illustrates an example implementation of the split-compute ML operation execution process of block 718 in accordance with some embodiments. As described above, a split neural network configuration can include a sequence of two or more split neural network portions, and the initial split portion in the sequence can be allocated to either the UE 104 or a server 112, as can the final split portion in the sequence, depending on implementation. Accordingly, for the purposes of description of the example implementation of the split-compute ML operation execution process of FIG. 9 below, the one of the UE 104 or server 112 that implements the initial split portion in a given split configuration is referred to herein as the “first device” and the other of the UE 104 or server 112 that implements the next split portion following the initial split portion in the sequence is referred to as the “second device” as reference to the order of the devices participation in the sequence of split configurations. In the example of FIG. 10 illustrating this implementation, the server 112 is the first device and the UE 104 is the second device, and the split configuration is a two-way split configuration with the server 112 implementing the initial split portion and the UE 104 implementing the final split portion.

[0063] A typical neural network receives input data, processes the input data at the various layers or other structures of the neural network, and then provides a final result output as a result of this processing of the input data. Depending on the purpose of the ML operation being performed and the origin of the input data for the neural network being implemented as a split configuration by the UE 104 and the server 112, the input data may be sourced by the UE 104, by the server 112, by another component, or a combination thereof. Regardless of the source, to begin execution of the ML operation, the input data is provided to the first device implementing the initial split portion. Accordingly, at block 902, the one or more sources of input data transmit or otherwise provide the first device access to the input data to be used for the ML operation. For example, in a split configuration in which the server 112 implements the initial split portion (that is, the server 112 is the first device and the UE 104 is the second device) but the ML operation involves input data obtained by the UE 104 (e.g., an image captured by the UE 104), the UE 104 may transmit the image data to the server 112 via the BS 110. As another example, the image data may be an image sourced from a web page (e.g., the ML operation is an image match search) and thus, rather than providing the image data directly, the UE 104 instead transmits a web address or other pointer to the image and the webpage, and the server 112 obtains the image data from the webpage using the pointer.

[0064] It will be appreciated that in a cellular network or other wireless network, such as the illustrated wireless communications network 100 of FIG. 1 , the BS 110 and UE 104 may employ one or more resource allocation schemes to facilitate uplink or downlink transfer of data between the UE 104 and the BS 110 in an efficient and timely manner. Such schemes may be pertinent in instances where the UE 104 is to provide a portion or all of the input data to the server 112 for input at the initial layer of the neural network at the server 112, and particularly so when there is a maximum latency requirement imposed for the ML operation. Accordingly, subsequent to the split compute negotiation process and prior to the transmission of the input data from the UE 104 to the server 112 via the BS 110, the UE 104 may notify the BS 110 of its uplink transmission needs for timely wireless transmission of the input data from the UE 104 to the BS 110. To illustrate, referring briefly to FIG. 10, following receipt of the split compute grant message 1010, at block 1012, the UE 104 identifies the input data to be provided by the UE 104 to the server 112 and determines the uplink (UL) data requirements in order to transmit this input data and buffers the input data in one or more UL buffers. At block 1012, the UE 104 transmits an UL transmit request for the input data in the form of an UL buffer status report (BSR) message 1014 to the BS 110 to inform the BS 110 of the amount of data in its UL buffers for UL transmission. In response to the UL BSR message 1014, the BS 110 transmits an UL grant to the UE 104 in the form of, for example, an UL downlink control information (DCI) message 1016, which informs the UE 104 of the parameters the UE 104 is to use in transmitting the data in its UL buffers to the BS 110 (including the input data), such as physical layer resource allocation, power control commands, and the like. At block 1018, the UE 104 wirelessly transmits the input data as UL data to the BS 110 using the indicated parameters of the UL DCI message 1016, and the BS 110 forwards the input data on to the server 112.

[0065] Referring back to FIG. 9, with the input data provided to the first device, at block 904 the first device receives the input data and processes (block 1020, FIG. 10) the input data at the initial split portion implemented at an ML module of the first device to generate output data that represents intermediate results of the ML operation. At block 906, the intermediate results (block 1022, FIG. 10) are transmitted to the second device. At block 908, the second device receives the intermediate results and provides the received intermediate results as input data to the next split portion implemented at an ML module of the second device, which processes (block 1024, FIG. 10) the input data to generate output data. As illustrated by decision block 910, in the event that this next split portion is the final split portion in the sequence of split portions, then the output data represents the final result of the neural network and thus the final result of the ML operation, and thus either or both of the final result is transmitted to the first device at block 912 so that the first device may take one or more actions in response to the final result or the second device may take one or more actions (block 1026, FIG. 10) in response to the final result at block 914. However, in the event that this next split portion is an intermediate split portion in the sequence of split portions, then the output data represents intermediate results and thus at block 916 the intermediate results are transmitted to the first device (or to a third device if three or more devices are implemented in the split neural network configuration), whereby these intermediate results are provided as input data to the next split configuration in the sequence to generate output data, which in turn may be the final results of the neural network if this next split configuration is the final split configuration in the sequence, and processed accordingly, or if there is one or more additional split portions yet remaining in the sequence, the output data may be provided to the next device for use as input data at the following split configuration in the sequence, and so forth, until the final split configuration in the sequence outputs the final results of the neural network.

[0066] The embodiments of the present disclosure also may be better understood through consideration of the following non-limiting examples: Example 1 : A computer-implemented method, in a first device, including: splitting a neural network into a split neural network configuration for at least the first device and a second device based on a set of one or more present conditions of the first device, the split neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion for the first device and a second neural network portion for the second device; implementing the first neural network portion at the first device; transmitting a representation of the second neural network portion to the second device; and processing a first data at the first neural network portion to generate a first output.

Example 2: The method of Example 1 , wherein: the first data includes input data for the neural network; the first output includes intermediate result data; and the method further includes: transmitting the intermediate result data to the second device for processing by the second neural network portion.

Example 3: The method of Example 2, further including: receiving a final result output of the neural network from the second device; and performing at least one action at the first device responsive to the final result output.

Example 4: The method of Example 1 , further including: receiving the first data from the second device, the first data including an intermediate result data generated by the second neural network portion at the second device; and wherein the first output is a final result output of the neural network.

Example 5: The method of Example 4, further including: performing at least one action at the first device based on the final result output of the neural network.

Example 6: The method of Example 4, further including: transmitting an input data from the first device to the second device for processing by the second neural network portion at the second device.

Example 7: The method of any of Examples 1 to 6, wherein the set of one or more present conditions includes at least one of: a battery condition; a network condition; a thermal condition; a compute resource availability; or a specified quality-of-service condition.

Example 8: The method of any of Examples 1 to 7, further including: determining a latency condition for performing a neural network operation using the neural network; and determining the split neural network configuration for at least the first device and the second device further based on the latency condition.

Example 9: The method of Example 8, further including: determining whether to perform the neural network operation entirely at the client device or by using a split configuration of the neural network based on at least one of the set of one or more present conditions or the latency condition; and determining the split neural network configuration for at least the first device and the second device responsive to determining to use a split configuration of the neural network.

Example 10: The method of any of Examples 1 to 9, further including: transmitting a split compute request to the second device, the split compute request proposing the split neural network configuration; and wherein the first neural network portion is implemented at the first device in response to receiving a grant of the split compute request from the second device.

Example 11 : The method of any of Examples 1 to 9, wherein determining the split neural network configuration for at least the first device and the second device includes: transmitting a split compute request to the second device, the split compute request proposing a different split neural network configuration that is based on the set of one or more present conditions; receiving a split compute counter-proposal from the second device, the split compute request counter-proposal proposing the split neural network configuration; and wherein the first neural network portion is implemented at the first device in response to accepting the split compute counterproposal based on the set of one or more present conditions.

Example 12: The method of one of Examples 10 or 11 , wherein the split compute request further indicates a latency condition for performing a neural network operation using the split neural network configuration. Example 13. The method of any of Examples 1 to 12, wherein the representation of the second neural network portion includes at least one of: data describing a neural network architecture of the second neural network portion; or an identifier of predetermined split neural network configuration for the neural network represented in a repository accessible to the second device.

Example 14. A computer-implemented method, in a second device, including: receiving, from a first device, an indication of a split neural network configuration for at least a first device and the second device, the split neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion for the first device and a second neural network portion for the second device; implementing the second neural network portion at the second device; receiving a first data from the first device; and processing the first data at the second neural network portion to generate a first output.

Example 15: The method of Example 14, wherein: the first data includes an input data for the neural network; the first output includes an intermediate result data; and the method further includes: transmitting the intermediate result data to the first device for processing at the first neural network portion.

Example 16: The method of Example 14, wherein: the first data includes an intermediate result data generated by the first neural network portion at the first device; and the first output is a final result output of the neural network.

Example 17: The method of Example 16, further including at least one of: performing at least one action at the second device responsive to the final result output; or transmitting the final result output to the first device.

Example 18: The method of any of Examples 14 to 17, wherein: receiving the indication of a split neural network configuration includes receiving a split compute request proposing the split neural network configuration; the method further includes determining whether to grant the split compute request; and wherein the second neural network portion is implemented at the second device responsive to determining to grant the split compute request.

Example 19: The method of Example 18, further including: receiving, from the first device, an indication of a latency condition for performing a neural network operation using the neural network; and determining whether to grant the split compute request is based on the latency condition.

Example 20: The method of any of Examples 1 to 19, wherein the second neural network portion is implemented at the second device responsive to receipt of a representation of the second neural network portion from the first device, the representation of the second neural network portion including at least one of: data representing a neural network architecture of the second neural network portion; or an identifier of predetermined split neural network configuration for the neural network represented in a repository accessible to the second device.

Example 21 : The method of any of Examples 1 to 20, wherein the first device is a user equipment and the second device is one or more servers connected to a network wirelessly accessible by the user equipment.

Example 22: The method of any of Examples 1 to 21 , wherein the neural network is a deep neural network (DNN) model.

Example 23: The method of any of Examples 1 to 22, wherein the split neural network configuration apportions a first set of contiguous layers of the neural network to the first neural network portion and a second set of contiguous layers of the neural network to the second neural network portion, the second set adjacent to the first set.

Example 24: The method of any of Examples 1 to 22, wherein the split neural network configuration includes a two-way split with an initial neural network portion and a final neural network portion, wherein the initial neural network portion is allocated to one of the first device or the second device and the final neural network portion is allocated to the other of the first device or the second device. Example 25: The method of any of Examples 1 to 22, wherein the split neural network configuration includes a three-way split with an initial neural network portion, an intermediate neural network potion, and a final neural network portion, wherein the initial neural network portion and final neural network portion are allocated to one of the first device or the second device and the intermediate neural network portion is allocated to the other of the first device or the second device.

Example 26. A device, including: a network interface; at least one processor coupled to the network interface; and a memory storing executable instructions, the executable instructions configured to manipulate the at least one processor to perform the method of any of Examples 1 to 25.

Example 27: The device of Example 26, wherein the first device is a user equipment of a cellular network and the second device is one or more servers of a network wirelessly accessible to the user equipment.

Example 28: The device of either of Example 26 or 27, wherein the device is the first device.

Example 29: The device of either of Example 26 or 27, wherein the device is the second device.

[0067] In some embodiments, certain aspects of the techniques described above may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer- readable storage medium can include, for example, a magnetic or optical disk storage device, solid-state storage devices such as Flash memory, a cache, random access memory (RAM), or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer-readable storage medium may be in source code, assembly language code, object code, or another instruction format that is interpreted or otherwise executable by one or more processors.

[0068] A computer-readable storage medium may include any storage medium, or combination of storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer- readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0069] Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0070] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.

Claims

WHAT IS CLAIMED IS:

1 . A computer-implemented method, in a first device, comprising: splitting a neural network into a split neural network configuration for at least the first device and a second device based on a set of one or more present conditions of the first device, the split neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion for the first device and a second neural network portion for the second device; implementing the first neural network portion at the first device; transmitting a representation of the second neural network portion to the second device; and processing a first data at the first neural network portion to generate a first output.

2. The method of claim 1 , wherein: the first data includes input data for the neural network; the first output includes intermediate result data; and the method further comprises: transmitting the intermediate result data to the second device for processing by the second neural network portion.

3. The method of claim 2, further comprising: receiving a final result output of the neural network from the second device; and performing at least one action at the first device responsive to the final result output.

4. The method of claim 1 , further comprising: receiving the first data from the second device, the first data including an intermediate result data generated by the second neural network portion at the second device; and wherein the first output is a final result output of the neural network.

5. The method of claim 4, further comprising: performing at least one action at the first device based on the final result output of the neural network.

6. The method of claim 4, further comprising: transmitting an input data from the first device to the second device for processing by the second neural network portion at the second device.

7. The method of any of claims 1 to 6, wherein the set of one or more present conditions includes at least one of: a battery condition; a network condition; a thermal condition; a compute resource availability; or a specified quality-of-service condition.

8. The method of any of claims 1 to 7, further comprising: determining a latency condition for performing a neural network operation using the neural network; and determining the split neural network configuration for at least the first device and the second device further based on the latency condition.

9. The method of claim 8, further comprising: determining whether to perform the neural network operation entirely at the first device or by using a split configuration of the neural network based on at least one of the set of one or more present conditions or the latency condition; and determining the split neural network configuration for at least the first device and the second device responsive to determining to use a split configuration of the neural network.

10. The method of any of claims 1 to 9, further comprising: transmitting a split compute request to the second device, the split compute request proposing the split neural network configuration; and wherein the first neural network portion is implemented at the first device in response to receiving a grant of the split compute request from the second device.

11 . The method of any of claims 1 to 9, wherein determining the split neural network configuration for at least the first device and the second device includes: transmitting a split compute request to the second device, the split compute request proposing a different split neural network configuration that is based on the set of one or more present conditions; receiving a split compute counter-proposal from the second device, the split compute counter-proposal proposing the split neural network configuration; and wherein the first neural network portion is implemented at the first device in response to accepting the split compute counter-proposal based on the set of one or more present conditions.

12. The method of one of claims 10 or 11 , wherein the split compute request further indicates a latency condition for performing a neural network operation using the split neural network configuration.

13. The method of any of claims 1 to 12, wherein the representation of the second neural network portion comprises at least one of: data describing a neural network architecture of the second neural network portion; or an identifier of predetermined split neural network configuration for the neural network represented in a repository accessible to the second device.

14. A computer-implemented method, in a second device, comprising: receiving, from a first device, an indication of a split neural network configuration for at least a first device and the second device, the split neural network configuration specifying a distribution of elements of a neural network architecture of a neural network between at least a first neural network portion for the first device and a second neural network portion for the second device; implementing the second neural network portion at the second device; receiving a first data from the first device; and processing the first data at the second neural network portion to generate a first output. e method of claim 14, wherein: the first data includes an input data for the neural network; the first output includes an intermediate result data; and the method further comprises: transmitting the intermediate result data to the first device for processing at the first neural network portion. e method of claim 14, wherein: the first data includes an intermediate result data generated by the first neural network portion at the first device; and the first output is a final result output of the neural network. e method of claim 16, further comprising at least one of: performing at least one action at the second device responsive to the final result output; or transmitting the final result output to the first device. e method of any of claims 14 to 17, wherein: receiving the indication of a split neural network configuration comprises receiving a split compute request proposing the split neural network configuration; the method further comprises determining whether to grant the split compute request; and wherein the second neural network portion is implemented at the second device responsive to determining to grant the split compute request. e method of claim 18, further comprising: receiving, from the first device, an indication of a latency condition for performing a neural network operation using the neural network; and determining whether to grant the split compute request is based on the latency condition. e method of any of claims 1 to 19, wherein the second neural network portion mented at the second device responsive to receipt of a representation of the second neural network portion from the first device, the representation of the second neural network portion including at least one of: data representing a neural network architecture of the second neural network portion; or an identifier of predetermined split neural network configuration for the neural network represented in a repository accessible to the second device.

21 . The method of any of claims 1 to 20, wherein the first device is a user equipment and the second device is one or more servers connected to a network wirelessly accessible by the user equipment.

22. The method of any of claims 1 to 21 , wherein the neural network is a deep neural network (DNN) model.

23. The method of any of claims 1 to 22, wherein the split neural network configuration apportions a first set of contiguous layers of the neural network to the first neural network portion and a second set of contiguous layers of the neural network to the second neural network portion, the second set adjacent to the first set.

24. The method of any of claims 1 to 22, wherein the split neural network configuration comprises a two-way split with an initial neural network portion and a final neural network portion, wherein the initial neural network portion is allocated to one of the first device or the second device and the final neural network portion is allocated to the other of the first device or the second device.

25. The method of any of claims 1 to 22, wherein the split neural network configuration comprises a three-way split with an initial neural network portion, an intermediate neural network portion, and a final neural network portion, wherein the initial neural network portion and final neural network portion are allocated to one of the first device or the second device and the intermediate neural network portion is allocated to the other of the first device or the second device.

26. A device, comprising: a network interface; at least one processor coupled to the network interface; and a memory storing executable instructions, the executable instructions configured to manipulate the at least one processor to perform the method of any of claims 1 to 25.

27. The device of claim 26, wherein the first device is a user equipment of a cellular network and the second device is one or more servers of a network wirelessly accessible to the user equipment.

28. The device of either of claim 26 or 27, wherein the device is the first device.

29. The device of either of claim 26 or 27, wherein the device is the second device.