CN112256626A - Data processing method and device, electronic equipment and computer readable storage medium - Google Patents
Data processing method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112256626A CN112256626A CN202011108937.2A CN202011108937A CN112256626A CN 112256626 A CN112256626 A CN 112256626A CN 202011108937 A CN202011108937 A CN 202011108937A CN 112256626 A CN112256626 A CN 112256626A
- Authority
- CN
- China
- Prior art keywords
- data
- length
- processing method
- encoding
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 238000004364 calculation method Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 21
- 230000004044 response Effects 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000000034 method Methods 0.000 abstract description 26
- 230000008569 process Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 13
- 238000013139 quantization Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012821 model calculation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241001643084 Cyrtanthus elatus virus A Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The disclosure discloses a data processing method, a data processing device, an electronic device and a computer readable storage medium. Wherein the method comprises: obtaining a plurality of first data, the plurality of first data having a first length; encoding the plurality of first data into second data, the second data having a second length; reading the second data in response to executing the first task; converting the second data into a plurality of third data, wherein the plurality of third data have a second length and the values of the plurality of third data are the same as the values of the plurality of first data. According to the embodiment of the disclosure, the data is encoded to save the storage space, and then the data is decoded to adapt to the data length required in the calculation process, so that the memory space can be saved, and the method is suitable for more calculation processes.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a data processing method, an apparatus, and a computer-readable storage medium.
Background
With the development of artificial intelligence technology, more and more lightweight neural network models are running on embedded terminal processors, such as Digital Signal Processing (DSP), (Advanced RISC Machines, ARM) and other processors. With the development of the technology, higher and higher requirements are also put forward on the processing capability of the embedded terminal, and higher requirements are also put forward on the computing power and the memory space of the neural network model.
In the prior art, a plurality of DSP processors, such as a High Fidelity (HIFI) series DSP and a CEVA series DSP in the voice industry, have a minimum supported calculation length of 16 bits, and generally adopt a 16-bit quantization model, but the cache of the DSP is generally small, and the amount of parameters that can be stored by using the 16-bit quantization model is small, and the calculation effect of the model is poor; more parameters can be used using an 8-bit quantization model, but the DSP processor supports a minimum of 16-bit data.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In order to at least partially solve the technical problem, according to an aspect of the present disclosure, the following technical solutions are provided:
a method of data processing, comprising:
obtaining a plurality of first data, the plurality of first data having a first length;
encoding the plurality of first data into second data, the second data having a second length;
reading the second data in response to executing the first task;
converting the second data into a plurality of third data, wherein the plurality of third data have a second length and the values of the plurality of third data are the same as the values of the plurality of first data.
Further, after the encoding the plurality of first data into the second data, the method further includes:
storing the second data in a cache of a processor, the processor being addressed at the second length.
Further, the encoding the plurality of first data into second data includes:
generating a plurality of data segments according to the sequence of the plurality of first data;
and arranging the plurality of data segments according to the sequence to generate the second data.
Further, the encoding the plurality of first data into second data includes:
combining the plurality of first data pairwise to obtain at least one first data group;
and encoding two first data in the at least one first data group to obtain at least one second data.
Further, the converting the second data into a plurality of third data includes:
decoding the second data to obtain a plurality of first data;
converting the plurality of first data into a plurality of third data, wherein the plurality of third data have the same value as the plurality of first data.
Further, the converting the plurality of first data into a plurality of third data includes:
complementing the upper bits of the plurality of first data with 0 to generate the plurality of third data.
Further, the decoding the second data to obtain the plurality of first data includes:
segmenting the second data to obtain a plurality of data segments;
and carrying out second calculation on the plurality of data segments to obtain a plurality of first data.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a data processing apparatus, comprising:
the device comprises a first data acquisition module, a second data acquisition module and a data processing module, wherein the first data acquisition module is used for acquiring a plurality of first data, and the first data have a first length;
an encoding module to encode the plurality of first data into second data, the second data having a second length;
a second data reading module for reading the second data in response to executing the first task;
and the data conversion module is used for converting the second data into a plurality of third data, wherein the plurality of third data have a second length, and the values of the plurality of third data are the same as the values of the plurality of first data.
Further, the data processing apparatus is further configured to:
storing the second data in a cache of a processor, the processor being addressed at the second length.
Further, the encoding module is further configured to:
generating a plurality of data segments according to the sequence of the plurality of first data;
and arranging the plurality of data segments according to the sequence to generate the second data.
Further, the encoding module is further configured to:
combining the plurality of first data pairwise to obtain at least one first data group;
and encoding two first data in the at least one first data group to obtain at least one second data.
Further, the data conversion module is further configured to:
decoding the second data to obtain a plurality of first data;
converting the plurality of first data into a plurality of third data, wherein the plurality of third data have the same value as the plurality of first data.
Further, the data conversion module is further configured to:
complementing the upper bits of the plurality of first data with 0 to generate the plurality of third data.
Further, the data conversion module is further configured to:
segmenting the second data to obtain a plurality of data segments;
and carrying out second calculation on the plurality of data segments to obtain a plurality of first data.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an electronic device, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions, such that the processor when executing implements the data processing method of any of the above.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a computer readable storage medium storing non-transitory computer readable instructions which, when executed by a computer, cause the computer to perform any of the data processing methods described above.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
a data processing terminal comprises any one of the data processing devices.
The disclosure discloses a data processing method, a data processing device, an electronic device and a computer readable storage medium. Wherein the method comprises: obtaining a plurality of first data, the plurality of first data having a first length; encoding the plurality of first data into second data, the second data having a second length; reading the second data in response to executing the first task; converting the second data into a plurality of third data, wherein the plurality of third data have a second length and the values of the plurality of third data are the same as the values of the plurality of first data. According to the embodiment of the disclosure, the data is encoded to save the storage space, and then the data is decoded to adapt to the data length required in the calculation process, so that the memory space can be saved, and the method is suitable for more calculation processes.
The foregoing is a summary of the present disclosure, and for the purposes of promoting a clear understanding of the technical means of the present disclosure, the present disclosure may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a schematic flow diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 2 is a schematic flow chart diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 3 is a schematic flow chart diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 4 is a schematic flow chart diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 5 is a schematic flow chart diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 6 is a schematic flow chart diagram of a data processing method according to one embodiment of the present disclosure;
FIG. 7 is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
The embodiment of the disclosure provides a data processing method. As shown in fig. 1, the data processing method mainly includes the following steps S101 to S104.
Step S101: obtaining a plurality of first data, the plurality of first data having a first length;
optionally, the first data is quantized data in a quantized network model, such as a quantization parameter. Illustratively, the first data is an 8-bit quantization parameter, and the length of the first data is 8 bits. The quantization is to approximate the original data to obtain data with low precision and small required storage space, for example, the original parameter of the network model is 32-bit data, and the original parameter is quantized into 8-bit parameter by the quantization to calculate, so that the parameter which can be stored in the same storage space is 4 times of the previous parameter, and the available parameter is greatly increased.
Step S102: encoding the plurality of first data into second data, the second data having a second length;
because some processors do not support the calculation of 8-bit data, the addressing length is minimum 16 bits, namely the cache (RAM) takes 16 bits as a storage unit, and even if 8-bit data is stored, 16-bit storage space is occupied. Thus, if an 8-bit quantized network model is used, if the model has N parameters, then a 16-bit processor is used, and its cache will need at least 16Nbit size to store the next N parameters, but 8Nbit after the actual size of the parameters. Thus in this step S12, the plurality of first data are encoded such that they can accommodate addressing larger than the first length.
Optionally, the step S102 includes:
step S201, generating a plurality of data segments according to the sequence of the plurality of first data;
step S202, arranging the plurality of data segments according to the sequence to generate the second data.
The plurality of first data and the plurality of data segments are in one-to-one correspondence, that is, each first data generates one data segment. Illustratively, the plurality of first data are 2 8-bit quantized data, which are represented by 16-ary notation: 0x12 and 0x 34; a first data segment is generated by 0x12 and a second data segment is generated by 0x34, and then the two data segments are arranged into the second data in the order of the first data. In one example, the first data may be directly used as the data segment, that is, 2 pieces of 8-bit quantized data are combined as the upper and lower bits of a 16-bit data, respectively, such as 0x12 and 0x34 described above, and the original values thereof may be used as the upper and lower bits of a 16-bit data, respectively: 0x 1234. Therefore, a plurality of first data with shorter length can be coded into second data with longer length to adapt to the storage space and calculation of a processor with longer length.
Optionally, the step S201 includes: and carrying out first calculation on the plurality of first data according to the sequence to obtain a plurality of data segments. Wherein each first data in the sequence may be calculated differently. If 0 is added to the first data, the first preset value is added to the second first data. Illustratively, the above-mentioned 2 8-bit quantized data: 0x12 and 0x 34; wherein 0x12 is unchanged, and as the upper data of the second data, 0x34 plus a preset value 128(0x80) is converted into 0xb4 as the lower data of the second data, resulting in 0x34b4 of the second data.
Optionally, the step S102 includes:
step S301, combining the plurality of first data in pairs to obtain at least one first data group;
step S302, encoding two first data in the at least one first data group to obtain at least one second data.
Illustratively, the plurality of first data are 16 8-bit quantized data, such as:
0x12、0x56、0xab、0x2a、0x35、0x89、0x65、0xcd；
0x34、0x78、0xcd、0x3b、0x67、0x34、0xab、0xde；
the 16 8-bit quantized data are grouped two by two to obtain (0x12,0x34), (0x56,0x78), (0xab,0xcd), (0x2a,0x3b), (0x35,0x67), (0x89,0x34), (0x65,0xab), (0xcd,0 xde).
And (3) encoding two 8-bit quantized data in each group, and calculating to obtain 8 16-bit data as described above: 0x12b4, 0x56f8, 0xab4d, 0x2abb, 0x35e7, 0x89b4, 0x652b, 0xcd5 e.
Through the steps, the plurality of first data are converted into second data with the second length so as to adapt to the addressing length of the processor, so that the data storage is more compact, and the storage space is saved.
Optionally, after the step S102, the method further includes:
storing the second data in a cache of a processor, the processor being addressed at the second length.
The parameters of the quantization model described in the above embodiment need to be transferred into the cache of the processor when performing the model calculation, and since the addressing length of the processor is longer and the length of the parameters of the quantization model is shorter, the parameters are encoded into the second length by the above step S102 and then transferred into the cache of the processor to perform the calculation task of the model.
Step S103, responding to the execution of the first task, reading the second data;
optionally, in response to the processor executing a first task, such as a model calculation task, relevant second data of the first task, such as parameters of the encoded 8-bit model, are read from a cache of the processor.
Step S104, converting the second data into a plurality of third data, wherein the plurality of third data have a second length, and the values of the plurality of third data are the same as the values of the plurality of first data.
Since the processor only supports the calculation of the data with the minimum second length, and the second data is the data after encoding and cannot be directly used for calculation, in the step S104, the second data is converted into a plurality of third data, and the third data has the same value as the first data, but the length is also the second length.
Optionally, the step S104 includes:
step S401, decoding the second data to obtain a plurality of first data;
step S402, converting the plurality of first data into a plurality of third data, wherein the plurality of third data and the plurality of first data have the same value.
In the above step, the second data is first restored into a plurality of first data by a decoding method opposite to the encoding method, and then the first data is converted into third data having a second length.
Optionally, step S401 includes:
step S501, segmenting the second data to obtain a plurality of data segments;
step S502, performing a second calculation on the plurality of data segments to obtain the plurality of first data.
As with the exemplary 16bit data 0x12b4 described above, it is first split into upper data and lower data, with the upper data unchanged to yield 0x12 and the lower data minus 128(0x80) to yield 0x 34.
Optionally, the step S402 includes: complementing the upper bits of the plurality of first data with 0 to generate the plurality of third data. As the high-order complement 0 of 0x12,0x 0012 is generated, and the high-order complement 0 of 0x34 generates 0x 0034. In this way, each third data is generated with the same value as its corresponding first data, but with a length of 16 bits, so that the third data can be calculated in the processor.
Optionally, the step S104 includes:
step S601, shifting the second data to obtain first and third data;
step S602, performing a third calculation on the second data to obtain second third data.
Taking the above example as an example, the second data is 16-bit data 0x12b 4; 0x12b4 is moved to the right by 8 bits, namely 0x0012 is obtained; the 0x12b4 was ANDed with 0x00ff to yield 0x00b4, followed by subtraction of 128(0x80) to yield 0x 0034.
Through the operation in the above-described step S104, the 16-bit encoded data in the above-described example is converted into 16-bit calculated data. As described above 0x12b4, 0x56f8, 0xab4d, 0x2abb, 0x35e7, 0x89b4, 0x652b, 0xcd5e were converted to: 0x0012, 0x0056, 0x00ab,0x 002a, 0x0035, 0x0089, 0x0065, 0x00cd,0x 0034, 0x0078, 0x00cd,0x 003b, 0x0067, 0x0034, 0x00ab,0x 00 de. Because each data has the same value as the original data, the length of each data is changed into 16 bits, and the data can be calculated by a processor with 16 bits.
Through the steps S101 and S102, a plurality of first data are encoded to obtain second data with a second length, so that the cache storing the data with the second length can store more first data; through the above-described steps S103 to S104, the length of the first data is converted into the second length, so that the processor that supports the second length at minimum can perform calculation using the first data. Thereby solving the problems of memory space waste and processor inapplicability.
The above embodiment discloses a data processing method, including: obtaining a plurality of first data, the plurality of first data having a first length; encoding the plurality of first data into second data, the second data having a second length; reading the second data in response to executing the first task; converting the second data into a plurality of third data, wherein the plurality of third data have a second length and the values of the plurality of third data are the same as the values of the plurality of first data. According to the embodiment of the disclosure, the data is encoded to save the storage space, and then the data is decoded to adapt to the data length required in the calculation process, so that the memory space can be saved, and the method is suitable for more calculation processes.
It will be appreciated by those skilled in the art that obvious modifications (e.g., combinations of the enumerated modes) or equivalents may be made to the above-described embodiments.
In the above, although the steps in the data processing method embodiment are described in the above sequence, it should be clear to those skilled in the art that the steps in the embodiment of the present disclosure are not necessarily performed in the above sequence, and may also be performed in other sequences such as reverse, parallel, and cross, and further, on the basis of the above steps, those skilled in the art may also add other steps, and these obvious modifications or equivalents should also be included in the protection scope of the present disclosure, and are not described herein again.
For convenience of description, only the relevant parts of the embodiments of the present disclosure are shown, and details of the specific techniques are not disclosed, please refer to the embodiments of the method of the present disclosure.
Example two
In order to solve the technical problems that the 16-bit quantization model in the prior art increases the computational burden of the terminal and increases the memory usage, the embodiment of the present disclosure provides a data processing apparatus 700. The apparatus 700 may perform the steps of the data processing method embodiment described in the first embodiment. As shown in fig. 7, the apparatus mainly includes: a first data acquisition module 701, an encoding module 702, a second data reading module 703 and a data conversion module 704; wherein the content of the first and second substances,
a first data obtaining module 701, configured to obtain a plurality of first data, where the plurality of first data have a first length;
an encoding module 702 configured to encode the plurality of first data into second data, the second data having a second length;
a second data reading module 703 for reading the second data in response to executing the first task;
a data conversion module 704, configured to convert the second data into a plurality of third data, where the plurality of third data have a second length, and values of the plurality of third data are the same as values of the plurality of first data.
Further, the data processing apparatus 700 is further configured to:
storing the second data in a cache of a processor, the processor being addressed at the second length.
Further, the encoding module 702 is further configured to:
generating a plurality of data segments according to the sequence of the plurality of first data;
and arranging the plurality of data segments according to the sequence to generate the second data.
Further, the encoding module 702 is further configured to:
combining the plurality of first data pairwise to obtain at least one first data group;
and encoding two first data in the at least one first data group to obtain at least one second data.
Further, the data conversion module 704 is further configured to:
decoding the second data to obtain a plurality of first data;
converting the plurality of first data into a plurality of third data, wherein the plurality of third data have the same value as the plurality of first data.
Further, the data conversion module 704 is further configured to:
complementing the upper bits of the plurality of first data with 0 to generate the plurality of third data.
Further, the data conversion module 704 is further configured to:
segmenting the second data to obtain a plurality of data segments;
and carrying out second calculation on the plurality of data segments to obtain a plurality of first data.
For detailed descriptions of the working principle, the technical effect of implementation, and the like of the embodiment of the data processing apparatus, reference may be made to the description related to the embodiment of the data processing method, and further description is omitted here.
EXAMPLE III
Referring now to FIG. 8, shown is a schematic diagram of an electronic device 800 suitable for use in implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 8, an electronic device 800 may include a processing means (e.g., central processing unit, graphics processor, etc.) 801 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage means 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the electronic apparatus 800 are also stored. The processing apparatus 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
Generally, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 807 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage 808 including, for example, magnetic tape, hard disk, etc.; and a communication device 809. The communication means 809 may allow the electronic device 800 to communicate wirelessly or by wire with other devices to exchange data. While fig. 8 illustrates an electronic device 800 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 809, or installed from the storage means 808, or installed from the ROM 802. The computer program, when executed by the processing apparatus 801, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: carrying out 8bit quantization on the network parameters of the neural network model to obtain first network parameters; coding the first network parameter to obtain a 16-bit quantized second network parameter supported by a terminal processor; the two first network parameters are coded to obtain a 16-bit quantized second network parameter; and decoding the second network parameter into a 16-bit network parameter in the calculation process of the neural network model.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (10)
1. A data processing method, comprising:
obtaining a plurality of first data, the plurality of first data having a first length;
encoding the plurality of first data into second data, the second data having a second length;
reading the second data in response to executing the first task;
converting the second data into a plurality of third data, wherein the plurality of third data have a second length and the values of the plurality of third data are the same as the values of the plurality of first data.
2. The data processing method of claim 1, wherein after said encoding the plurality of first data into second data, further comprising:
storing the second data in a cache of a processor, the processor being addressed at the second length.
3. The data processing method of claim 1, wherein said encoding the plurality of first data into second data comprises:
generating a plurality of data segments according to the sequence of the plurality of first data;
and arranging the plurality of data segments according to the sequence to generate the second data.
4. The data processing method of claim 1, wherein said encoding the plurality of first data into second data comprises:
combining the plurality of first data pairwise to obtain at least one first data group;
and encoding two first data in the at least one first data group to obtain at least one second data.
5. The data processing method of claim 1, wherein said converting the second data into a plurality of third data comprises:
decoding the second data to obtain a plurality of first data;
converting the plurality of first data into a plurality of third data, wherein the plurality of third data have the same value as the plurality of first data.
6. The data processing method of claim 5, wherein said converting the plurality of first data into a plurality of third data comprises:
complementing the upper bits of the plurality of first data with 0 to generate the plurality of third data.
7. The data processing method of claim 5, wherein said decoding the second data into the plurality of first data comprises:
segmenting the second data to obtain a plurality of data segments;
and carrying out second calculation on the plurality of data segments to obtain a plurality of first data.
8. A data processing apparatus, comprising:
the device comprises a first data acquisition module, a second data acquisition module and a data processing module, wherein the first data acquisition module is used for acquiring a plurality of first data, and the first data have a first length;
an encoding module to encode the plurality of first data into second data, the second data having a second length;
a second data reading module for reading the second data in response to executing the first task;
and the data conversion module is used for converting the second data into a plurality of third data, wherein the plurality of third data have a second length, and the values of the plurality of third data are the same as the values of the plurality of first data.
9. An electronic device, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions such that the processor when executing performs the data processing method of any of claims 1-7.
10. A computer-readable storage medium storing non-transitory computer-readable instructions which, when executed by a computer, cause the computer to perform the data processing method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011108937.2A CN112256626A (en) | 2020-10-16 | 2020-10-16 | Data processing method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011108937.2A CN112256626A (en) | 2020-10-16 | 2020-10-16 | Data processing method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112256626A true CN112256626A (en) | 2021-01-22 |
Family
ID=74244407
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011108937.2A Pending CN112256626A (en) | 2020-10-16 | 2020-10-16 | Data processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112256626A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023231858A1 (en) * | 2022-05-31 | 2023-12-07 | 维沃移动通信有限公司 | Data transmission circuit, data transmission method, and electronic device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030137994A1 (en) * | 1998-05-22 | 2003-07-24 | Takayuki Kikuchi | Data processing apparatus and method and computer readable storage medium |
| CN1744720A (en) * | 2004-08-31 | 2006-03-08 | 松下电器产业株式会社 | Variable length decoding device |
| CN106782576A (en) * | 2017-02-15 | 2017-05-31 | 合网络技术（北京）有限公司 | audio mixing method and device |
| CN109792256A (en) * | 2016-08-11 | 2019-05-21 | 瑞伯韦尔公司 | For the device and correlation technique coded and decoded to data to erasure codes |
| CN110222048A (en) * | 2019-05-06 | 2019-09-10 | 平安科技（深圳）有限公司 | Sequence generating method, device, computer equipment and storage medium |
| CN110309138A (en) * | 2018-03-01 | 2019-10-08 | 阿里巴巴集团控股有限公司 | Data merging method, the combiner based on FPGA and Database Systems |
| CN110690943A (en) * | 2019-09-26 | 2020-01-14 | 中兴通讯股份有限公司 | Data encoding method, decoding method, encoding and decoding method, device and storage medium |
| CN110830427A (en) * | 2018-08-13 | 2020-02-21 | 北京京东尚科信息技术有限公司 | Method and device for message encoding and message decoding in netty environment |
-
2020
- 2020-10-16 CN CN202011108937.2A patent/CN112256626A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030137994A1 (en) * | 1998-05-22 | 2003-07-24 | Takayuki Kikuchi | Data processing apparatus and method and computer readable storage medium |
| CN1744720A (en) * | 2004-08-31 | 2006-03-08 | 松下电器产业株式会社 | Variable length decoding device |
| CN109792256A (en) * | 2016-08-11 | 2019-05-21 | 瑞伯韦尔公司 | For the device and correlation technique coded and decoded to data to erasure codes |
| CN106782576A (en) * | 2017-02-15 | 2017-05-31 | 合网络技术（北京）有限公司 | audio mixing method and device |
| CN110309138A (en) * | 2018-03-01 | 2019-10-08 | 阿里巴巴集团控股有限公司 | Data merging method, the combiner based on FPGA and Database Systems |
| CN110830427A (en) * | 2018-08-13 | 2020-02-21 | 北京京东尚科信息技术有限公司 | Method and device for message encoding and message decoding in netty environment |
| CN110222048A (en) * | 2019-05-06 | 2019-09-10 | 平安科技（深圳）有限公司 | Sequence generating method, device, computer equipment and storage medium |
| CN110690943A (en) * | 2019-09-26 | 2020-01-14 | 中兴通讯股份有限公司 | Data encoding method, decoding method, encoding and decoding method, device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| HUA ZHANG: "An Efficient Base Conversion Using Variable Length Segmentation and Remainder Transfer", 《 IEEE SIGNAL PROCESSING LETTERS 》, 19 June 2019 (2019-06-19), pages 1227 - 1231 * |
| 韩大晗;崔慧娟;唐昆;刘大力;: "一款可编程语音处理器的设计与应用", 计算机工程, no. 12, 20 June 2007 (2007-06-20) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023231858A1 (en) * | 2022-05-31 | 2023-12-07 | 维沃移动通信有限公司 | Data transmission circuit, data transmission method, and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368519B (en) | Method, device, equipment and storage medium for editing online form | |
| CN112395253B (en) | Index file generation method, terminal device, electronic device and medium | |
| CN110545472B (en) | Video data processing method and device, electronic equipment and computer readable medium | |
| CN112818663A (en) | Processing method for language model, text generation method, text generation device and medium | |
| JP7437517B2 (en) | Methods, devices, electronic devices and computer readable media for generating predictive information | |
| WO2023273610A1 (en) | Speech recognition method and apparatus, medium, and electronic device | |
| CN112380876B (en) | Translation method, device, equipment and medium based on multilingual machine translation model | |
| CN112200713A (en) | Business data processing method, device and equipment in federated learning | |
| CN111581555A (en) | Document loading method, device, equipment and storage medium | |
| CN111309304B (en) | Method, device, medium and electronic equipment for generating IDL file | |
| CN112686031A (en) | Text feature extraction model quantification method, device, equipment and storage medium | |
| CN112256626A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN112416303A (en) | Software development kit thermal restoration method and device and electronic equipment | |
| CN111653261A (en) | Speech synthesis method, speech synthesis device, readable storage medium and electronic equipment | |
| CN109474826B (en) | Picture compression method and device, electronic equipment and storage medium | |
| CN113343259B (en) | SM 2-based joint signature realization method and device, electronic equipment and storage medium | |
| CN113971192A (en) | Data processing method and device, readable medium and electronic equipment | |
| CN111399902B (en) | Client source file processing method and device, readable medium and electronic equipment | |
| CN111143355B (en) | Data processing method and device | |
| CN112437013B (en) | Path acquisition method in network diagram and maintenance method of routing networking | |
| CN114283060B (en) | Video generation method, device, equipment and storage medium | |
| CN114816396A (en) | UI component library calling method and device, electronic equipment and computer readable medium | |
| CN112445517B (en) | Inlet file generation method, device, electronic equipment and computer readable medium | |
| CN110545108B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN111209042B (en) | Method, device, medium and electronic equipment for establishing function stack |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |