JPH06176000A - Neurocomputer - Google Patents

Abstract

PURPOSE:To effectively utilize a memory space by compressing input data in terms of hardwares based on a certain algorithm considering that the effective utilization of the memory space for preserving weighted values is required at the time of constituting a large scale neural network. CONSTITUTION:A main memory unit, a compression unit and an extension unit are added to a neurocomputer. By classifying the weighted value input data of 16 bits into four levels and allocating them to low-order bits in the descending order of an appearance frequency, the data are compressed. As for the hardwares, only a memory LSI equivalent to 10K bytes is added. By using this system, approximately 16% of a memory area can be compressed. Also, since a symbol table or the like is not required and the system can be realized by a simple circuit, an execution speed is fast and the system is extremely suitable for the realization of a real-time neurocomputer.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for accumulating synapse weight values in a digital or analog neurocomputer, and more particularly to a method for compressing weight value information.

[0002]

2. Description of the Related Art When realizing a large scale neural network, it is important to effectively use a memory area. For example, 16 terabits (16 × 10 ¹² ) are required when a 1-million (1 × 10 ⁶ ) neuron is formed by 16-bit perfect connection in the digital system. To achieve this,
If a 4-megabit memory is used, 4 million pieces are needed. Data compression methods include Entropy coding method and Huffman coding method. Ent
In the ropy coding method, the more frequently appearing data is assigned to the smaller number of bits. The Huffman coding method is a type of Entropy coding method, but a predetermined variable length code is used.

[0003]

The first drawback of the above coding method is that the size of the input data is limited by the size of the conversion table. The second drawback is that the process of decompressing data (the opposite of data compression) is complicated. Furthermore, the third drawback is that it is necessary to know the occurrence frequency distribution of the input data in advance. In the case of character input such as English text, Huff is required to easily understand the frequency distribution.
The man coding method is effective. However, in the case of data input such as weight values, the frequency distribution depends on the individual problem, its prediction is difficult, and the above coding method is inappropriate.

The present invention eliminates the above-mentioned drawbacks of the prior art,
It is to provide a data compression method capable of effectively utilizing a memory space of weight values in a large-scale neural network.

[0005]

It is known that the weight value distribution of a neural network does not depend on the learning algorithm and is distributed as shown in FIG. Here, the weight value takes a positive or negative value, where positive is excitatory coupling and negative is inhibitory coupling. It can be seen that the central part of FIG. 2 corresponds to the weight value of zero, and the value near the zero has a higher frequency.

Therefore, considering that the weight value is expressed in binary 16 bits (hereinafter expressed as 16 ₂ ), as shown in FIG.
The weight value distribution is divided into four areas. At this time, the weight value takes a value in the range of 0 to 65535 in decimal,
-32768 to 32767 to make the central part zero
Takes the value of. Therefore, the area is divided as follows.

Area Lower limit Upper limit Possible values 1 7e00 ₁₆ 81ff ₁₆ 1024 ₁₀ 2 7600 ₁₆ 7dff ₁₆ 2048 ₁₀ 3 8200 ₁₆ 89ff ₁₆ 2048 ₁₀ 4a 0000 ₁₆ 75ff ₁₆ 30207 ₁₀ 4b 8a00 ₁₆ ffff ₁₆ 30207 ₁₀ Since there are four areas, they can be expressed as follows using a binary 2-bit area code (RC).

Region Region Code (RC) 1 00 ₂ 2 01 ₂ 3 10 ₂ 4 11 ₂ Region 1 has a value between 1024 ₁₀ (1 in binary system).
Since it is 0 bit) and the area code is 00 ₂ (2 bits in the binary system), it can be expressed by 12 bits. Similarly, in regions 2 and 3, 11 bits + 2 bits = 13 bits,
In the area 4, it can be expressed by 16 bits + 2 bits = 18 bits.

FIG. 1 shows these conversion methods. The 16-bit uncompressed data passes through the comparator,
Converted to 12, 13, and 18 bits.

[0010]

Operation: For 16-bit input data (weight value),
By assigning to the area 1 the data near the zero where the input value occurs frequently, the data can be compressed to 12 bits. Further, by allocating other data to the areas 2, 3, and 4, the data can be converted into 13, 13, and 18 bits, respectively. In the area 4, on the contrary, the number of bits is increased to 18 bits.

[0011]

EXAMPLES The present invention will be described in detail below with reference to examples.

FIG. 4 is a block diagram of the compression system. This system is a conventional neurocomputer system 4
To the main memory 2 of compressed weight values, decompression hardware 3, compression hardware 5, address generator (read data) 6, address generator (write data)
7 is added.

FIG. 5 shows a circuit configuration diagram of the compression system. This circuit includes a comparator 8, a compressed weight value main memory 9, an 18-bit parallel input, and a parallel output shift register 1.
0 and buffer 11. Input data is classified as follows.

Area 1 32,356 (0111111000000000) → 33,279 (10000
00111111111) Area 2 30,208 (0111011000000000) → 32,255 (01111
10111111111) Area 3 33,280 (1000001000000000) → 35,327 (10001
00111111111) Area 4 0 (0000000000000000) → 32,255 (01111
10111111111) 35,328 (1000101000000000) → 65,535 (111111111111111
1) Therefore, by examining the first 7 bits of the uncompressed data, the areas 1 to 4 can be classified. Specifically, FIG.
Comparator 8 compares the upper 7 bits of the uncompressed data, generates a 2-bit area code, and adds these to the input data. The features of this hardware are that the processing speed is fast and the required memory is small. For example, compression,
10 Kbytes of RAM (random access memory) or ROM (read only memory) for the entire deployment system
Can be configured with. In addition, this system is a D
It is suitable for simulation of SP (digital signal processing) type neuro computer.

Next, FIG. 6 shows a circuit configuration diagram of the expansion system. This circuit includes a compressed weight main memory 12, a counter 13, a control block 14, a parallel input, a serial output shift register 15, a region code 16, a 16-bit serial input, a parallel output shift register 17, a 2048 × 16-bit RAM (region. 2) 18, buffer 19, 1024x
It is composed of a 16-bit RAM (area 1) 20 and a 2048 × 16-bit RAM (area 3) 21. The system is managed by the control block 14 and converted to uncompressed data through registers and counters. In the case of the area 4, the data is directly transferred to the neuro computer without the need for the RAM conversion table.

[0016]

According to the present invention, the memory can be compressed without sacrificing the accuracy of the synapse weight value. In the numerical simulation, it was possible to compress about 16% of the memory area. Further, this system can be added to a conventional neuro computer without special design. Moreover, the present invention does not require the creation of the symbol table that was required in conventional coding schemes. In the compression / decompression circuit, A
Since it does not require an LU (Alice Metec logic circuit),
It has a fast execution speed and does not require complicated control circuits.

[Brief description of drawings]

FIG. 1 shows a method of converting input data into compressed data.

FIG. 2 shows a distribution of weight values in a neural network.

FIG. 3 shows a classification method for converting the weight values of FIG. 1 into compressed data.

FIG. 4 shows a block diagram of a compression system.

FIG. 5 shows a circuit configuration diagram of a compression method.

FIG. 6 shows a circuit configuration diagram of a development system.

[Explanation of symbols]

1 --- Comparator, 2 ---- Main memory of compressed weight values, 3 ---- decompression hardware, 4--neuro computer system, 5 --- compression hardware, 6 --- -Address generator, read data, 7 --- Address generator, write data, 8--comparator, 9 --- Main memory of compressed weight value, 10--18bit parallel, parallel output shift register, 11 --Buffer, 12--main memory of compressed weight values, 13--counter, 14--control block, 15--parallel input, serial output shift register, 16--region code, 17-16 bit serial input , Parallel output shift register, 18-2048x16 bits random access memory (region 2), 19-buffer, 20-1024 16 bits random access memory (area 1), 21--2048x16 bits random access memory (region 3).

Claims (3)

[Claims] 1. A main memory unit for storing synapse weight values in a compressed form, an expansion unit for converting compressed data in the main memory into uncompressed data and transferring the uncompressed data to a neuro computer, and an uncompressed unit from the neuro computer. A neuro computer comprising a compression unit for converting data into compressed data and transferring the compressed data to a main memory unit. 2. The compression unit of claim 1, wherein said compression unit comprises an input receiving circuit, a circuit for comparing some or all of the inputs with a value made from a particular synaptic weight value distribution, and A neurocomputer comprising a circuit for generating a region code from a section, a circuit for forming compressed data from data after input and comparison, and a circuit for transferring the compressed data to a main memory unit. 3. The uncompressed unit according to claim 1, wherein the uncompressed unit includes an input receiving circuit, a circuit for deleting a region code from input data, a circuit for generating an extended region code from the region code, and an extended region. A neurocomputer comprising a circuit for forming uncompressed data from a code and input data and a circuit for transferring the uncompressed data to the neurocomputer.