CN115981666A

CN115981666A - Neural network information integration method, device, system and storage medium

Info

Publication number: CN115981666A
Application number: CN202310275238.4A
Authority: CN
Inventors: 严勇猛
Original assignee: Beijing Intengine Technology Co Ltd
Current assignee: Beijing Intengine Technology Co Ltd
Priority date: 2023-03-21
Filing date: 2023-03-21
Publication date: 2023-04-18
Anticipated expiration: 2043-03-21
Also published as: CN115981666B

Abstract

The invention discloses a neural network information integration method, a device, a system and a storage medium. The invention can compile the original data of the preset neural network, output the hierarchical information of the preset neural network, the hierarchical information comprises the characteristic diagram, the network layer, the network segment, the subnet and the network of the preset neural network, extract the characteristic parameters of the neural network corresponding to the hierarchical information, assemble the characteristic parameters of the neural network, pack the hierarchical information in turn, create the header information of the IRIFF file, and write the packed data of the hierarchical information in turn after the header information, so as to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use process, and the processing efficiency of the neural network is effectively improved.

Description

Neural network information integration method, device, system and storage medium

Technical Field

The invention relates to the field of neural network processing, in particular to a neural network information integration method, device and system and a storage medium.

Background

With the development of artificial intelligence technology, neural networks are applied to more and more fields, and the production and the life of people are improved. The neural network is a neural network learning algorithm, and is a hierarchical neural network consisting of an input layer, an intermediate layer and an output layer, wherein the intermediate layer can be expanded into multiple layers. All the neurons of adjacent layers are in full connection, and all the neurons of each layer are not in connection, the network learns according to the teaching mode of a teacher, and after a pair of learning modes are provided for the network, all the neurons obtain the input response of the network to generate connection weights (Weight). And then correcting the connection weights layer by layer from the output layer through the intermediate layers in a direction of reducing the error between the desired output and the actual output, and returning to the input layer. The process is repeatedly and alternately carried out until the global error of the network tends to a given minimum value, namely the learning process is completed.

In the prior art, the neural network has many kinds, such as a Back Propagation (BP) neural network, a Radial Basis Function (RBF-Radial Basis Function) neural network, a perceptron neural network, a linear neural network, a self-organizing neural network, and a feedback neural network. However, the applicant finds that each neural network includes various information parameters, such as a network map, auxiliary information, weight data, NPU instruction codes, and other relevant information generated by a compiler, and the parameters need to be called when the computing platform uses the neural network, and memory resources in a common computing platform are extremely limited and do not support a file system, so that the parameters can be called only from an external memory, which results in low processing efficiency of the neural network.

Disclosure of Invention

The invention provides a neural network information integration method, a device, a system and a storage medium, which can integrate characteristic parameters of a neural network and directly call the characteristic parameters through a memory during subsequent use, thereby improving the processing efficiency of the neural network.

In order to achieve the above beneficial effects, the embodiment of the present invention provides the following technical solutions:

in a first aspect, please provide a neural network information integration method, the method includes:

compiling original data of a preset neural network, and outputting grading information of the preset neural network, wherein the grading information comprises a characteristic diagram, a network layer, a network segment, a subnet and a network of the preset neural network;

extracting neural network characteristic parameters corresponding to the grading information;

compiling the characteristic parameters of the neural network, and sequentially packaging the grading information;

and creating header information of the IRIFF file, and writing the packed data of the hierarchical information in sequence after the header information to obtain the IRIFF file.

In a second aspect, it provides a neural network information integrating device, including:

the device comprises a compiling unit and a control unit, wherein the compiling unit is used for compiling original data of a preset neural network and outputting grading information of the preset neural network, and the grading information comprises a characteristic diagram, a network layer, a network segment, a subnet and a network of the preset neural network;

the extraction unit is used for extracting the neural network characteristic parameters corresponding to the grading information;

the packing unit is used for compiling the neural network characteristic parameters and packing the hierarchical information in sequence;

and the creating unit is used for creating the header information of the IRIFF file and writing the packed data of the hierarchical information in sequence after the header information to obtain the IRIFF file.

In a third aspect, the present application provides a neural network information integration system, including: the device comprises a computing module and a nonvolatile memory;

the calculation module is used for compiling original data of a preset neural network, outputting grading information of the preset neural network, wherein the grading information comprises a characteristic diagram, a network layer, a network segment, a subnet and a network of the preset neural network, extracting neural network characteristic parameters corresponding to the grading information, compiling the neural network characteristic parameters, sequentially packaging the grading information, creating header information of an IRIFF file, sequentially writing the packaged data of the grading information behind the header information to obtain the IRIFF file, and storing the IRIFF file into a nonvolatile memory.

In a fourth aspect, a storage medium is provided, where the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to perform the steps of the neural network information integration method.

The embodiment provided by the application can compile the original data of the preset neural network, output the hierarchical information of the preset neural network, wherein the hierarchical information comprises a characteristic diagram, a network layer, a network segment, a subnet and a network of the preset neural network, extract the neural network characteristic parameters corresponding to the hierarchical information, assemble the neural network characteristic parameters, sequentially pack the hierarchical information, create the header information of the IRIFF file, and sequentially write the packed data of the hierarchical information into the header information to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use process, and the processing efficiency of the neural network is effectively improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow chart of a neural network information integration method according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a hardware platform according to an embodiment of the present invention;

FIG. 3 is another schematic flow chart of a neural network information integration method according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a neural network information integrating device according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a neural network information integration apparatus according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of an intelligent terminal according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein may be combined with other embodiments.

The embodiment of the present invention provides a neural network information integration method, where an execution subject of the neural network information integration method may be the neural network information integration device provided in the embodiment of the present invention, or an electronic device integrated with the neural network information integration device, where the neural network information integration device may be implemented in a hardware or software manner.

In this embodiment, the neural network information integrating device may be specifically an electronic device, and the electronic device has a storage unit and is capable of running an application program.

A neural network information integration method, the method comprising:

Referring to fig. 1, fig. 1 is a schematic flow chart of a neural network information integration method according to an embodiment of the present invention. The neural network information integration method comprises the following steps:

step 101, compiling the original data of the preset neural network, and outputting the grading information of the preset neural network.

In an embodiment, the neural network information integration method provided in the embodiment of the present application may be applied to a hardware platform as shown in fig. 2, where the hardware platform may include an external memory and a plurality of computing modules, and the external memory may include a low-speed, non-volatile memory module, a device, or an equivalent. Such as a flash, a mechanical hard disk, etc., or even a remote file, etc.

In an embodiment, the computing module includes a main control module, at least one computing core, and a Memory, where the Memory may be an SRAM (static Random-Access Memory) or a DDR SDRAM (double data Rate Synchronous Dynamic Random Access Memory), the main control module Host may be a CPU, and the at least one computing core may be a DSP (Digital Signal Processing), an NPU (neutral-network Processing unit), a GPU (graphics Processing unit), and the like.

It should be noted that the computing modules may be physically fixed, or may be dynamically combined as needed, and the memory of each computing module may be addressed independently, or may be addressed together with the memory of one or more other computing modules. In one embodiment, the above calculation core includes two types: namely, the system can automatically and continuously read a command (which can be an instruction or a configuration parameter) sequence, decode and execute the command, and is called as an active core; otherwise called the passive core. Each active core in each compute module needs to be numbered, such as CPU, NPU _0, NPU _1, etc. as shown in fig. 2. A plurality of independent computing modules may operate simultaneously, and each computing module may compute one or more neural networks, which is not further limited in this application.

In an embodiment, the preset neural network may be finely divided from small to large according to the level of the internal structure, and specifically may include a feature map, a network layer, a network segment, a subnet, and a network. Specifically, the hierarchical information of the preset neural network, that is, the characteristic diagram, the network layer, the network segment, the subnet, and the network, may be output by compiling the raw data of the preset neural network. Among them, feature map (abbreviated as fm): i.e., 3-dimensional tensor, 3 latitudes are usually represented by H, W, C, and the combined writing is HWC. The data type is not counted in latitude and is independently represented by B.

Network layer (layer): is the basic unit of the neural network after being preprocessed by the neural network compiler, also called layer for short. In contrast to what is known as a neural network layer (primitive for short): may be a part of a split original layer or a fusion of a plurality of continuous original layers. The input is a tensor, usually the eigen-map, and the output is the eigen-map. The network layer is further subdivided here into two types: one layer is a layer which can generate a command sequence after being compiled by a neural network tool chain, and the command sequence can be executed (calculated) by a certain active core and obtain an output characteristic diagram, and is called a CMD network layer; the rest is called as RAW network layer, that is, the original information of the network layer needs to be preserved, and the actual calculation mode and process are determined by Host during calculation.

Segment (segment): i.e. one or a succession of network layers that satisfy a certain condition. Two categories are also distinguished: the method can continuously run on a certain active core without switching, and is called as a CMD network segment; otherwise called RAW segment. It should be noted that, during debugging or testing, it is also possible to make each CMD network layer as a CMD network segment separately.

Subnet (subnet): namely, in each round of calculation (corresponding to an input feature map (abbreviated as fi)), one or a plurality of continuous network segments with the same frequency are calculated. That is, there is no branching, or looping, due to conditional arbitration in between.

Network (net): one or more subnets grouped together by logical relationships. Various neural networks are known which comprise only one sub-network. The calculation results of the network layers are called intermediate feature maps. The intermediate feature maps can be divided into two categories: a feature map called static (static) that needs to be saved and participates in a subsequent round of computation; the rest (i.e. not involved in the subsequent round of computation) is called the local profile.

In an embodiment, a high-level concept, such as a network group (group), may be further defined, and may be formed by a plurality of networks that are logically combined together.

Further, the compiling process may be processed by a neural network compiler, and specifically, various neural network raw files or data may be input, such as a general structural description of the neural network, details of each raw layer, a quantization mode of each raw layer, a trained weight parameter, and the like. Then, the hierarchical information of the network is output, besides, quantized weight parameter data, reference feature map files, command sequences, other member information required by the IRIFF, and the like can be output.

In the process of compiling, compiling options can be further set, including compiling scope, compiling-time optimization level and the like, such as determining whether to compile all or only part of the original layer, or whether to compile completely or only but ignore the reference feature map, and the like.

And 102, extracting the neural network characteristic parameters corresponding to the grading information.

Further, the neural network characteristic parameters corresponding to the grading information, such as the above command sequence, are extracted as the input of the subsequent assembly step, wherein the precursor of the command sequence may include the following modes: c-code, assembly code, and text format configuration parameter sequences. The assembly code may be, for example, an NPU assembly in json format, and the text format may be json.

And 103, compiling the neural network characteristic parameters, and sequentially packaging the grading information.

In an embodiment, the software for implementing the process of creating the IRIFF file, i.e., the process of inputting the neural network list (e.g., netlist. Json) and linking the option output IRIFF file, is a neural network linker, and the neural network linker requires front-end tools including, but not limited to, a neural network compiler and an active core assembler. Therefore, the neural network characteristic parameters can be compiled by the active core assembler, and specifically, the precursor of the command sequence can be compiled into a corresponding binary data sequence (i.e., a command sequence), which corresponds to the precursor of the command sequence, and therefore, the following can be classified as follows: c-code, assembly code, and parameter sequences in text format. The C code is assembled, for example, by GCC (GNU compiler collection) or LLVM (Low Level Virtual Machine), the assembly code may be assembled by a general assembler, and the parameter sequence in text format may be assembled by a special text-to-binary converter.

In an embodiment, when the hierarchical information is sequentially packed, the network information of each hierarchy may be sequentially packed according to the sequence of the feature map, the network layer, the network segment, the subnet, and the network. It should be noted that the information required by the characteristic diagram, the network layer, the network segment, and the subnet is from the original data, the compiler output, and the assembler output, and does not relate to additional network information. And the information needed by the network may come from raw data, compiler output, assembler output, additional network information, compilation options, etc. The additional network information is a logical relationship between subnets or a control parameter required to be used in the logical relationship when one network includes a plurality of subnets. Thus, when the network comprises a plurality of subnets, the packing process of the network may comprise: and extracting the logical relationship and the control parameters among the subnets to generate additional network information, acquiring original network information of the network, and packaging the original network information and the additional network information.

In an embodiment, when the network information of the characteristic diagram, the network layer, the network segment, the subnet, and the network is packaged, a container header may be created first, and data may be refilled, specifically, the output from the neural network compiler or the output from the active core assembler may be filled.

And 104, creating header information of the IRIFF file, and writing the packed data of the hierarchical information in sequence after the header information to obtain the IRIFF file.

Specifically, information such as bit width of members such as a base address and a pointer contained in the file container can be defined, the number of networks in a preset neural network is obtained, then, an IRIFF file container header is defined, a field is filled in a data portion after the header information, that is, packed data of hierarchical information is filled, and lengths of front portions of the containers are aligned as required, so that the IRIFF file container is obtained. When creating an IRIFF file, a page assignment method based on an object can be adopted for packaging.

The relevant parameter information and data of each neural network may be stored in an IRIFF file, for example, 3 blocks are sequentially from left to right (corresponding to the file, i.e., from the beginning to the end): the 4 th block, namely the reference characteristic diagram, can be added during debugging of the network detail information, the weight parameter and the command sequence, and the network detail information, the weight parameter, the command sequence and the reference characteristic diagram are all continuously stored and connected end to end.

Further, the total BNF of IRIFF is as follows:

in an embodiment, the neural network characteristic parameters in the at least one container may include network detail information, weight parameters, command sequences, and the like in all the neural networks.

In an embodiment, when the IRIFF file is manufactured, a reference feature map and an array may also be manufactured and added, where the reference feature map array may be added to the reference feature maps of all layers, or only a part of layers may also be added, which is described by taking an example of adding only a part of layers, and the input and output of the RAW layer may be counted one by one through the whole network, and then the input and output of the whole network are added, and repeated data therein is removed, and the remaining feature maps corresponding to the input and output are added to the IRIFF.

As can be seen from the above, the neural network information integration method provided in the embodiment of the present application may compile original data of a preset neural network, output hierarchical information of the preset neural network, where the hierarchical information includes a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network, extract neural network feature parameters corresponding to the hierarchical information, assemble the neural network feature parameters, sequentially pack the hierarchical information, create header information of an IRIFF file, and sequentially write the packed data of the hierarchical information behind the header information, so as to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use process, and the processing efficiency of the neural network is effectively improved.

The information integration method of the neural network described in the previous embodiment is further described in detail below by way of example.

In this embodiment, the neural network information integrating device is specifically integrated in an intelligent terminal as an example for explanation.

Referring to fig. 3, fig. 3 is another schematic flow chart of a neural network information integration method according to an embodiment of the present invention. The method flow can comprise the following steps:

step 201, compiling the original data of the preset neural network, and outputting the grading information of the preset neural network.

The hierarchical information may include a feature map, a network layer, a network segment, a subnet, and a network of a preset neural network.

Step 202, extracting the neural network characteristic parameters corresponding to the grading information.

Step 203, initializing the allocable memory address and length information, and compiling the characteristic parameters of the neural network.

And step 204, sequentially packaging the hierarchical information, and updating the memory address and the length information according to the packaged data.

The first address and length information of the memory and the external memory which can be allocated can be updated according to the relation between the networks in the network group, and if the related information of a plurality of networks exists, the steps of reading, compiling, assembling, packaging and updating can be repeatedly executed in sequence.

Step 205, creating header information of the IRIFF file, and writing the packed data of the hierarchical information in sequence after the header information to obtain the IRIFF file.

Step 206, storing the IRIFF file in the nonvolatile memory.

And step 207, copying the network characteristic parameters in the IRIFF file to a memory of the platform when the preset neural network is operated, and calling a computing module of the platform to call the network characteristic parameters in a memory access mode.

In an embodiment, the step of copying the network characteristic parameters in the IRIFF file into the memory of the platform may include: analyzing the header information of the IRIFF file to obtain the names corresponding to the characteristic diagram, the network layer, the network segment, the subnet and the network of the preset neural network, sequentially obtaining the characteristic parameters of the characteristic diagram, the network layer, the network segment, the subnet and the network of the neural network according to the IRIFF format and the names, and copying the characteristic parameters to the memory of the platform.

In an embodiment, when the memory space is small during the operation of the computing module, so that all the network characteristic parameters in the IRIFF file cannot be copied into the memory, a part of the network characteristic parameters, that is, the target parameters, may be preferentially copied according to the remaining memory values. For example, network detail information of the neural network is copied into a memory, then a flash read weight parameter or a command sequence needs to be accessed to the memory in the specific calculation process of each CMD network segment, and then a calculation core in the calculation module can start calculation.

In another embodiment, if the memory is sufficient, a preset memory area may be partitioned from the memory of the platform in advance, and then the IRIFF file is loaded into the preset memory area, and the computing module of the platform is invoked to invoke the network feature parameters in a manner of accessing the memory. For example, during debugging, the IRIFF file is directly loaded into a certain area of the memory through a back door. Then the computing module starts to operate, the process of data copying is omitted, and simulation debugging can be greatly accelerated.

It should be noted that the non-volatile memory, i.e., the external memory, is, for example, a flash, and an IRIFF file is stored therein. After the power-on, a certain main control module Host can copy the information of the neural network into the memory of the module 2 according to all, wherein the main control module Host is not necessarily a CPU of the module, and is specifically determined according to the design of a hardware platform. And then, the computing module starts to operate, and the specific computing process of each layer/each network segment only needs to access the memory without accessing the flash.

As can be seen from the above, the neural network information integration method provided in the embodiment of the present application may compile original data of a preset neural network, output hierarchical information of the preset neural network, extract neural network feature parameters corresponding to the hierarchical information, initialize distributable memory addresses and length information, assemble the neural network feature parameters, sequentially pack the hierarchical information, update the memory addresses and the length information according to the packed data, create header information of an IRIFF file, sequentially write the packed data of the hierarchical information after the header information to obtain the IRIFF file, store the IRIFF file in a nonvolatile memory, copy network feature parameters in the IRIFF file into a memory of a platform when the preset neural network is operated, and call a computing module of the platform to call the network feature parameters in a manner of accessing the memory. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file and copied into the memory of the platform, so that the characteristic parameters can be directly called through the memory in subsequent use, and the processing efficiency of the neural network is effectively improved.

The application also provides a neural network information integration system, which specifically comprises a computing module and a nonvolatile memory;

the calculation module is used for compiling original data of a preset neural network, outputting grading information of the preset neural network, wherein the grading information comprises a feature diagram, a network layer, a network segment, a subnet and a network of the preset neural network, extracting neural network feature parameters corresponding to the grading information, compiling the neural network feature parameters, sequentially packaging the grading information, creating header information of an IRIFF file, sequentially writing the packaged data of the grading information behind the header information to obtain the IRIFF file, and storing the IRIFF file into a nonvolatile memory.

In order to better implement the neural network information integration method provided by the embodiment of the invention, the embodiment of the invention also provides a device based on the neural network information integration method. The terms have the same meanings as those in the neural network information integration method, and specific implementation details can refer to the description in the method embodiment.

In this embodiment, the description will be made in terms of a neural network information integration device, which may be specifically integrated into a system composed of a plurality of intelligent terminals, each of which is an intelligent terminal having a storage unit and a video playing function and mounted with a display screen.

Referring to fig. 4, fig. 4 is a schematic structural diagram of a neural network information integration apparatus 300 according to an embodiment of the present invention. The neural network information integrating device 300 may include:

a compiling unit 301, configured to compile raw data of a preset neural network, and output hierarchical information of the preset neural network, where the hierarchical information includes a feature diagram, a network layer, a network segment, a subnet, and a network of the preset neural network;

an extracting unit 302, configured to extract a neural network characteristic parameter corresponding to the hierarchical information;

a packing unit 303, configured to assemble the neural network characteristic parameters, and pack the hierarchical information in sequence;

the creating unit 304 is configured to create header information of the IRIFF file, and sequentially write the packed data of the hierarchical information after the header information to obtain the IRIFF file.

In an embodiment, please refer to fig. 5, fig. 5 is a schematic structural diagram of a neural network information integration apparatus 300 according to an embodiment of the present invention. The neural network information integrating device 300 may further include:

an initializing unit 305, configured to initialize assignable memory addresses and length information before the compiling unit 301 compiles the raw data of the preset neural network;

an updating unit 306, configured to update the memory address and the length information according to packed data after the packing unit 303 sequentially packs the hierarchical information.

In an embodiment, when the network includes a plurality of subnets, the packing unit 303 may specifically include:

an extraction subunit 3031, configured to extract the logical relationship between the plurality of subnets and the control parameter to generate additional network information;

a packing subunit 3032, configured to obtain original network information of the network, and pack the original network information and the additional network information.

As can be seen from the above, the embodiment of the present invention may compile original data of a preset neural network, output hierarchical information of the preset neural network, where the hierarchical information includes a feature diagram, a network layer, a network segment, a subnet, and a network of the preset neural network, extract neural network feature parameters corresponding to the hierarchical information, assemble the neural network feature parameters, pack the hierarchical information in sequence, create header information of an IRIFF file, and write the packed data of the hierarchical information in sequence after the header information, so as to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use process, and the processing efficiency of the neural network is effectively improved.

An embodiment of the present invention further provides an intelligent terminal 600, as shown in fig. 6, where the intelligent terminal 600 may integrate the neural network information integration apparatus, and may further include a Radio Frequency (RF) circuit 601, a memory 602 including one or more computer-readable storage media, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a Wireless Fidelity (WiFi) module 607, a processor 608 including one or more processing cores, a power supply 609, and other components. Those skilled in the art will appreciate that the intelligent terminal 600 configuration shown in fig. 6 does not constitute a limitation of the intelligent terminal 600 and may include more or less components than those shown, or some components in combination, or a different arrangement of components. Wherein:

the RF circuit 601 may be used for receiving and transmitting signals during a message transmission or communication process, and in particular, for receiving downlink messages from a base station and then processing the received downlink messages by one or more processors 608; in addition, data relating to uplink is transmitted to the base station. In general, the RF circuit 601 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 601 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global system for Mobile communications (GSM), general Packet Radio Service (GPRS), code Division Multiple Access (CDMA), wideband Code Division Multiple Access (WCDMA), long Term Evolution (LTE), email, short Messaging Service (SMS), and the like.

The memory 602 may be used to store software programs and modules, and the processor 608 executes various functional applications and information processing by operating the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, a target data playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the smart terminal 600, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 608 and the input unit 603 access to the memory 602.

The input unit 603 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, in one particular embodiment, input unit 603 may include a touch-sensitive surface as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations by a user (e.g., operations by a user on or near the touch-sensitive surface using a finger, a stylus, or any other suitable object or attachment) thereon or nearby, and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 608, and can receive and execute commands sent by the processor 608. In addition, touch sensitive surfaces may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 603 may include other input devices in addition to the touch-sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 604 may be used to display information input by or provided to the user and various graphical user interfaces of the intelligent terminal 600, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 604 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-emitting diode (OLED), or the like. Further, the touch-sensitive surface may overlay the display panel, and when a touch operation is detected on or near the touch-sensitive surface, the touch operation is transmitted to the processor 608 to determine the type of touch event, and the processor 608 then provides a corresponding visual output on the display panel according to the type of touch event. Although in FIG. 6 the touch-sensitive surface and the display panel are two separate components to implement input and output functions, in some embodiments the touch-sensitive surface may be integrated with the display panel to implement input and output functions.

The smart terminal 600 may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or the backlight when the smart terminal 600 moves to the ear. As one of the motion sensors, the gravitational acceleration sensor may detect the magnitude of acceleration in each direction (generally, three axes), detect the magnitude and direction of gravity when the mobile phone is stationary, and may be used for applications (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like) for recognizing the posture of the mobile phone, and the intelligent terminal 600 may further be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, which are not described herein again.

Audio circuitry 606, a speaker, and a microphone may provide an audio interface between a user and the smart terminal 600. The audio circuit 606 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 606 and converted into audio data, and then the audio data is processed by the audio data output processor 608, and then the audio data is sent to another intelligent terminal 600 through the RF circuit 601, or the audio data is output to the memory 602 for further processing. The audio circuitry 606 may also include an earbud jack to provide communication of peripheral headphones with the smart terminal 600.

WiFi belongs to short-distance wireless transmission technology, and the intelligent terminal 600 can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 607, and provides wireless broadband internet access for the user. Although fig. 6 shows the WiFi module 607, it is understood that it does not belong to the essential constitution of the smart terminal 600, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 608 is a control center of the smart terminal 600, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the smart terminal 600 and processes data by operating or executing software programs and/or modules stored in the memory 602 and calling data stored in the memory 602, thereby performing overall monitoring of the mobile phone. Optionally, processor 608 may include one or more processing cores; preferably, the processor 608 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 608.

The intelligent terminal 600 also includes a power supply 609 (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor 608 via a power management system, such that the power management system may manage charging, discharging, and power consumption. The power supply 609 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power data indicators, and the like.

Although not shown, the smart terminal 600 may further include a camera, a bluetooth module, and the like, which are not described herein. Specifically, in this embodiment, the processor 608 in the intelligent terminal 600 loads the executable file corresponding to the process of one or more application programs into the memory 602 according to the following instructions, and the processor 608 runs the application programs stored in the memory 602, so as to implement various functions:

In the above embodiments, the descriptions of the embodiments have respective emphasis, and a part which is not described in detail in a certain embodiment may refer to the above detailed description of the neural network information integration method, and is not described herein again.

As can be seen from the above, the intelligent terminal 600 according to the embodiment of the present invention may compile original data of the preset neural network, output hierarchical information of the preset neural network, where the hierarchical information includes a feature map, a network layer, a network segment, a subnet, and a network of the preset neural network, extract neural network feature parameters corresponding to the hierarchical information, assemble the neural network feature parameters, sequentially pack the hierarchical information, create header information of an IRIFF file, and sequentially write the packed data of the hierarchical information after the header information, so as to obtain the IRIFF file. According to the embodiment of the application, the characteristic parameters of the neural network can be integrated into the IRIFF file, so that the characteristic parameters can be directly called through the memory in the subsequent use process, and the processing efficiency of the neural network is effectively improved.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application further provides a storage medium, on which a plurality of instructions are stored, where the instructions are suitable for being loaded by a processor to perform the steps in the neural network information integration method.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

Wherein the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disk, and the like.

Since the instructions stored in the storage medium can execute the steps in any neural network information integration method provided by the embodiment of the present invention, the beneficial effects that can be achieved by any neural network information integration method provided by the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

The neural network information integration method, device, system and storage medium provided by the embodiment of the present invention are described in detail above, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A neural network information integration method, the method comprising:

compiling the neural network characteristic parameters and sequentially packaging the grading information;

2. The neural network information integration method of claim 1, wherein before the compiling of the raw data of the preset neural network, the method further comprises:

initializing allocable memory addresses and length information;

and after the hierarchical information is sequentially packed, updating the memory address and the length information according to packed data.

3. The neural network information integration method of claim 1, wherein when the network includes a plurality of subnetworks, the packing process of the network comprises:

extracting logical relations and control parameters among the plurality of subnets to generate additional network information;

and acquiring original network information of the network, and packaging the original network information and the additional network information.

4. The neural network information integration method of claim 1, further comprising:

packing user data information;

classifying the user data information according to the data value type;

arranging according to the character string of the user data information in the classification result;

and quantizing the value according to the arrangement result, and packaging the quantized value and the original key into a custom array container and a custom structure container.

5. The neural network information integration method of claim 1, wherein after obtaining the IRIFF file, the method further comprises:

storing the IRIFF file into a non-volatile memory;

and when the preset neural network is operated, copying the network characteristic parameters in the IRIFF file into a memory of a platform, and calling a computing module of the platform to call the network characteristic parameters in a mode of accessing the memory.

6. The neural network information integration method of claim 5, wherein the copying the network characteristic parameters in the IRIFF file to a memory of a platform comprises:

analyzing the header information of the IRIFF file to obtain the corresponding names of the characteristic diagram, the network layer, the network segment, the sub-network and the network of the preset neural network;

and sequentially acquiring the characteristic diagram, the network layer, the network segment, the sub-network and the neural network characteristic parameters of the network according to the IRIFF format and the name, and copying the characteristic parameters into the memory of the platform.

7. An apparatus for integrating neural network information, comprising:

8. The neural network information integrating device of claim 7, wherein the device further comprises:

the initialization unit is used for initializing allocable memory addresses and length information before the compiling unit compiles the original data of the preset neural network;

and the updating unit is used for updating the memory address and the length information according to the packed data after the packing unit packs the hierarchical information in sequence.

9. A neural network information integration system, comprising: the system comprises a computing module and a nonvolatile memory;

10. A storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to perform the steps of the neural network information integrating method according to any one of claims 1 to 6.