WO2018198454A1 - 情報処理装置、および情報処理方法 - Google Patents

情報処理装置、および情報処理方法 Download PDF

Info

Publication number
WO2018198454A1
WO2018198454A1 PCT/JP2018/002404 JP2018002404W WO2018198454A1 WO 2018198454 A1 WO2018198454 A1 WO 2018198454A1 JP 2018002404 W JP2018002404 W JP 2018002404W WO 2018198454 A1 WO2018198454 A1 WO 2018198454A1
Authority
WO
WIPO (PCT)
Prior art keywords
information processing
processing apparatus
information
input values
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/002404
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
聡幸 廣井
山本　真紀子
章 中村
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to CN201880026324.9A priority Critical patent/CN110574024A/zh
Priority to EP18792207.5A priority patent/EP3617904A4/en
Priority to US16/479,044 priority patent/US11030524B2/en
Priority to JP2019505277A priority patent/JPWO2018198454A1/ja
Publication of WO2018198454A1 publication Critical patent/WO2018198454A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • This disclosure relates to an information processing apparatus and an information processing method.
  • Non-Patent Document 1 describes a technique for reducing the processing burden by binarizing a weighting coefficient.
  • Non-Patent Document 2 describes a method of converting multiplication into addition by converting an input signal into a log domain.
  • Non-Patent Document 1 since binarization using +1 or ⁇ 1 is performed, it is assumed that the quantization granularity becomes coarser as the number of dimensions of the weight coefficient increases. Moreover, although the method described in Non-Patent Document 2 has a predetermined effect in avoiding multiplication, it is assumed that there is still room for reducing the processing load.
  • the present disclosure proposes a new and improved information processing apparatus and information processing method that can further reduce the processing load related to the inner product calculation and can guarantee the quantization granularity of the weight coefficient.
  • an arithmetic unit that performs an inner product operation based on a plurality of input values and a plurality of weighting factors corresponding to the input values and calculates an output value
  • the arithmetic unit includes an N-dimensional hypersphere surface
  • an information processing apparatus that calculates the output value based on the weighting coefficient quantized based on the granularity in the vector direction.
  • a product-sum operation circuit that performs a product-sum operation based on a plurality of input values and a plurality of weighting factors respectively corresponding to the input values is provided, and the product-sum operation circuit has a small value
  • a storage circuit that holds address information of the input values corresponding to the plurality of weighting factors rearranged in order; a multiplication circuit that performs an operation based on the input values obtained from the address information and the weighting factors;
  • the processor includes a product-sum operation based on a plurality of input values and a plurality of weighting factors respectively corresponding to the input values to calculate an output value, wherein the calculating includes And calculating the output value based on the weighting coefficient quantized based on the granularity in the vector direction on the surface of the N-dimensional hypersphere.
  • FIG. 3 is an example of a functional block diagram of an information processing apparatus according to an embodiment of the present disclosure.
  • FIG. 2 is an example of a circuit block diagram of a product-sum operation circuit included in the information processing apparatus according to the embodiment.
  • FIG. It is an example of the offset notation concerning the address information which the address table concerning the embodiment holds.
  • DELTA quantization particle size
  • FIG. 3 is a diagram illustrating a hardware configuration example according to an embodiment of the present disclosure.
  • learning methods using a neural network such as deep learning have been widely studied. While a learning method using a neural network has high accuracy and a large processing load is involved in the calculation, a calculation method that effectively reduces the processing load is required.
  • FIG. 1 is a conceptual diagram for explaining an outline of basic operations in a neural network.
  • FIG. 1 shows two layers constituting the neural network, and cells c1 1 to c1 N and cell c2 1 belonging to the two layers, respectively.
  • an input signal (hereinafter also referred to as an input vector) input to the cell C2 1 is input to an input vector and a weighting factor (hereinafter also referred to as a weight vector) related to the cells c1 1 to c1 N belonging to the lower layer.
  • a weighting factor hereinafter also referred to as a weight vector
  • the input vector input to the cell c2 1 is a value obtained by adding a bias b to the inner product calculation result of the input vector and the weight vector related to the cells c1 1 to c1 N and further processing by the activation function h. It becomes.
  • FIG. 2 is a schematic diagram for explaining the inner product calculation of the input vector x and the weight vector w.
  • FIG. 3 is a diagram for explaining a weight vector w that is binary quantized in a two-dimensional space.
  • the granularity of the weight vector w can be expressed by the rotation angle ⁇ in the plane, and the granularity is 90 degrees as shown in FIG.
  • FIG. 4 is a diagram for explaining a weight vector w that is four-value quantized in a two-dimensional space.
  • the granularity of the weight vector w that is, the rotation angle ⁇ is about 15 degrees, and a finer granularity can be guaranteed as compared with the case of binary quantization.
  • FIG. 5 is a diagram for explaining variation in the granularity of the weight vector w in the three-dimensional space.
  • the (1,1,0) direction Since the length of the side is ⁇ square root ⁇ 2 times the length of the side in the (0, 0, 1) direction, it can be seen that the variation in the granularity at the time of quantization increases.
  • FIG. 6 is a diagram for explaining the variation in the granularity of the weight vector w in the N-dimensional space.
  • FIG. 6 shows a plane defined by (1, 1,..., 1, 0) and (0, 0,..., 0, 1) in the N-dimensional space.
  • the length of the side in the (1, 1,..., 1, 0) direction is ⁇ (N ⁇ 1) of the length of the side in the (0, 0,..., 0, 1) direction.
  • Can be represented by fold. For example, if N 100, 1, 1,. . . , 1, 0) direction is ⁇ 99 times ( ⁇ 10 times) the side in the (0, 0,..., 0, 1) direction.
  • an information processing apparatus and an information processing method according to an embodiment of the present disclosure perform an inner product operation using a weight vector quantized based on a granularity in a vector direction in an N-dimensional hypersphere plane. One of them.
  • An information processing apparatus and an information processing method according to an embodiment of the present disclosure can achieve both high approximation accuracy and reduction in processing load by quantizing a weight vector with a granularity that is not too fine and not too coarse. Is possible. More specifically, the information processing apparatus and the information processing method according to an embodiment of the present disclosure may perform an inner product operation using a weight vector expressed by a power.
  • the characteristics of the information processing apparatus and the information processing method according to an embodiment of the present disclosure will be described in detail.
  • FIG. 7 is an example of a functional block diagram of the information processing apparatus 10 according to the present embodiment.
  • the information processing apparatus 10 according to the present embodiment includes an input unit 110, a calculation unit 120, a storage unit 130, and an output unit 140.
  • the above configuration will be described focusing on the functions of the configuration.
  • the input unit 110 has a function of detecting various input operations by an operator. For this reason, the input unit 110 according to the present embodiment may include various devices for detecting an input operation by the operator.
  • the input unit 110 can be realized by various buttons, a keyboard, a touch panel, a mouse, a switch, and the like, for example.
  • the calculation unit 120 has a function of calculating an output value by performing an inner product calculation based on a plurality of input values and a plurality of weighting factors respectively corresponding to the input values.
  • the calculation unit 120 according to the present embodiment performs inner product calculation related to forward propagation of a neural network.
  • one of the features is that the calculation unit 120 according to the present embodiment calculates an output value based on a weighting coefficient quantized based on the granularity in the vector direction on the surface of the N-dimensional hypersphere. More specifically, the computing unit 120 according to the present embodiment may calculate an output value based on a weighting factor expressed by a power.
  • the storage unit 130 has a function of storing a program, data, and the like used in each configuration included in the information processing apparatus 10.
  • the storage unit 130 according to the present embodiment stores, for example, various parameters used for the neural network.
  • the output unit 140 has a function of outputting various information to the operator.
  • the output unit 140 according to the present embodiment may be configured to include a display device that outputs visual information.
  • the display device described above can be realized by, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an organic light emitting diode (OLED) device, or the like.
  • the functional configuration example of the information processing apparatus 10 according to the present embodiment has been described above. Note that the functional configuration example described above is merely an example, and the functional configuration example of the information processing apparatus 10 according to the present embodiment is not limited to the example.
  • the information processing apparatus 10 according to the present embodiment may further include a configuration other than that illustrated in FIG.
  • the information processing apparatus 10 may further include, for example, a communication unit that performs information communication with other information processing terminals.
  • the functional configuration of the information processing apparatus 10 according to the present embodiment can be flexibly changed in design.
  • the information processing apparatus 10 can maintain high uniformity in granularity by performing quantization using the weight vector w expressed by a power.
  • the operation unit 120 rearranges the plurality of weight vector w i in ascending order of value, also normalizing the plurality of weight vector w i in the most large value of the weight coefficient w i Is one of the features.
  • the weight vector w j is expressed by the following equations (2) to (4).
  • ⁇ in the above equation (2) is 0 ⁇ ⁇ 1, s j is s j ⁇ ⁇ 1,1 ⁇ , and n j is n j ⁇ ⁇ 0, 1, 2,. . . ⁇ . That is, the arithmetic unit 120 according to the present embodiment performs quantization using n j as an integer.
  • the inner product calculation executed by the calculation unit 120 is expressed by the following mathematical formula (5).
  • K represents a normalization constant.
  • the value of ⁇ may be finally determined within the above range in the inner product calculation even when the following formula (5) is appropriately modified.
  • the mathematical expressions shown in the present disclosure are merely examples, and can be flexibly modified.
  • the inner product calculation by the calculation unit 120 according to the present embodiment can be processed by N addition calculations and the number of multiplications in the order of ⁇ 1/2 log (N ⁇ 1) / log ⁇ . .
  • one of the features is that the weight vector w is approximated by a power expression of ⁇ , and the weight vectors w are rearranged in order of increasing values.
  • the weight vector w is quantized by converting the exponent of ⁇ into k values according to N.
  • N 100
  • k 4 (2 bits), 8 (3 bits), 16 (4 bits), etc.
  • n j ⁇ 1 ⁇ n j can take a value other than 0 only 4 times. For this reason, in the case of this example, the number of multiplications related to the inner product calculation is four, and all the rest are additions, so that the processing burden can be effectively reduced.
  • the information processing apparatus 10 includes a product-sum operation circuit having a table that holds address information of an input vector x corresponding to a plurality of weight vectors w rearranged in order of increasing values. Good.
  • FIG. 8 is an example of a circuit block diagram of the product-sum operation circuit 200 included in the information processing apparatus 10 according to the present embodiment.
  • the product-sum operation circuit includes a storage circuit that holds a table WT that holds address information of an input vector x corresponding to a weight vector w, a RAM 210, an adder circuit 220, and an accumulator 230.
  • ⁇ a first multiplication circuit 240 that performs multiplication according to ⁇
  • a second multiplication circuit 250 that performs multiplication according to a normalization constant.
  • the address table WT holds address information, code information, and multiplication instruction information of an input vector x corresponding to a plurality of weight vectors w rearranged in order of increasing values.
  • the above address information may include a null pointer as shown in FIG. In this case, 0 is added to the accumulator 230, and the value of the accumulator 230 can be simply multiplied by ⁇ .
  • the code information is information indicating a value corresponding to S j in the above formula (5).
  • the above multiplication instruction information is information for instructing the processing contents by the first multiplication circuit 240.
  • the multiplication instruction information according to the present embodiment may include, for example, information that specifies whether or not multiplication is necessary. In FIG. 8, when the multiplication instruction information is 0, the first multiplication circuit 240 does not perform multiplication, and when the multiplication instruction information is 1, the first multiplication circuit 240 multiplies ⁇ . An example of the case is shown.
  • the multiplication instruction information according to the present embodiment is not limited to the above example, and may include information specifying various processing contents.
  • the multiplication instruction information according to the present embodiment may include, for example, information that specifies the number of multiplications and a shift operation.
  • the RAM 210 outputs an input vector x corresponding to the weight vector w to the adder circuit 220 based on the address information input from the address table WT.
  • the adder circuit 220 performs addition based on the input vector x input from the RAM 210 and the value output from the first multiplier circuit 240. At this time, the adding circuit 220 according to the present embodiment performs the above addition based on the code information held in the address table WT.
  • the accumulator 230 accumulates the calculation results output from the adder circuit 220. Accumulator 230 outputs the accumulated value to first multiplication circuit 240 and second multiplication circuit 250. The accumulator 230 is appropriately input with a reset signal for resetting the accumulated value to zero.
  • the first multiplication circuit 240 multiplies the value accumulated by the accumulator 230 by ⁇ . At this time, as described above, the first multiplication circuit 240 performs the above multiplication based on the multiplication instruction information held in the address table WT. The first multiplication circuit 240 outputs the calculation result to the addition circuit 220.
  • the second multiplication circuit 250 multiplies the value output from the accumulator 230 by a normalization constant K.
  • the configuration example of the product-sum operation circuit 200 according to this embodiment has been described above. According to the product-sum operation circuit 200 according to the present embodiment, the number of multiplications in the inner product operation can be effectively reduced, and the processing load can be reduced.
  • the address table WT according to the present embodiment may include an offset indicating a relative position between addresses as shown in FIG.
  • FIG. 9 is an example of offset notation related to address information held by the address table WT according to the present embodiment.
  • the address table WT according to the present embodiment includes a section in which the value of n j ⁇ 1 ⁇ n j is continuously 0 in the above-described equation (5), that is, a section in which multiplication is not performed.
  • the addresses may be sorted in the order of addresses, and the offset between the addresses may be held as address information. According to the address table WT according to the present embodiment, it is possible to significantly reduce the amount of information related to address information and effectively reduce power consumption.
  • the address table WT according to the present embodiment can take various forms other than the forms shown in FIGS.
  • the address table WT according to the present embodiment may not clearly separate and hold the code information and the multiplication instruction information, or may employ an address compression method other than the above.
  • the address table WT according to the present embodiment can be flexibly modified according to the configuration of the neural network, the performance of the information processing apparatus 10, and the like.
  • w max in the above formula (6) represents the maximum value of w i .
  • the closer one of rounding up or rounding down may be selected.
  • w i quantized by a power expression is defined as w j by rearranging and normalizing in order of increasing values.
  • the weight vector w is expressed by the following equation (7).
  • the information processing method according to the present embodiment includes q 1 , q 2 ,. . . It is meaningful to repeat the process of creating a vector on the plane formed by the weight vector projected onto the space spanned by q j ⁇ 1 and q j and multiplying the vector by ⁇ nj ⁇ nj + 1 .
  • FIG. 10 is a diagram illustrating a processing image of the information processing method according to the present embodiment.
  • the weight vectors are q 1 , q 2 ,. . .
  • the quantization granularity ⁇ of the weight vector is expressed by the following formula (8) in the counterclockwise and clockwise directions, respectively, as shown in FIG. And (9).
  • l in the equations (8) and (9) is defined by the equation (10).
  • FIG. 11 is a diagram for explaining the quantization granularity ⁇ according to the present embodiment. In FIG. 11, the weight vector projected in the first quadrant is shown.
  • FIG. 12 is a graph showing the maximum value of the quantization granularity ⁇ according to ⁇ according to the present embodiment.
  • the quantization granularity is guaranteed in all orthogonal rotation directions in the N-dimensional space.
  • FIG. 13 is a diagram for explaining the maximum power multiplier according to this embodiment.
  • the weight vector projected on the first quadrant is shown.
  • the maximum power multiplier for guaranteeing the quantization granularity ⁇ may be obtained by adding the following formula (13) to the minimum m satisfying the following formula (12). Therefore, the number of multiplications executed by the information processing apparatus 10 according to the present embodiment can be obtained by the following mathematical formula (14).
  • the number of multiplications for the input number N is determined as shown in the graph of FIG.
  • the number of multiplications for the input number N is determined as shown in the graph of FIG. That is, in the inner product calculation by the calculation unit 120 according to the present embodiment, the number of multiplications can be determined based on the bottom value of the weight vector.
  • 14 and 15 are diagrams showing examples of the number of multiplications for the number N of inputs according to the present embodiment.
  • the number of multiplications can be significantly reduced in the inner product calculation related to the forward propagation of the neural network, and the product-sum calculation circuit 200 It is possible to effectively reduce the power consumption due to. Also, according to the information processing apparatus that implements the information processing method according to the present embodiment, it is possible to improve the quantization accuracy of the weight vector, and by using a neural network as compared with the conventional quantization method using the same number of bits. The effect of improving recognition accuracy and approximation accuracy is expected.
  • FIG. 16 is a block diagram illustrating a hardware configuration example of the information processing apparatus 10 according to an embodiment of the present disclosure.
  • the information processing apparatus 10 includes, for example, a CPU 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, and an output device 879.
  • Storage 880, drive 881, connection port 882, and communication device 883 Note that the hardware configuration shown here is an example, and some of the components may be omitted. Moreover, you may further include components other than the component shown here.
  • the CPU 871 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation or a part of each component based on various programs recorded in the ROM 872, RAM 873, storage 880, or removable recording medium 901.
  • the ROM 872 is a means for storing programs read by the CPU 871, data used for calculations, and the like.
  • the RAM 873 for example, a program read by the CPU 871, various parameters that change as appropriate when the program is executed, and the like are temporarily or permanently stored.
  • the CPU 871, the ROM 872, and the RAM 873 are connected to each other via, for example, a host bus 874 capable of high-speed data transmission.
  • the host bus 874 is connected to an external bus 876 having a relatively low data transmission speed via a bridge 875, for example.
  • the external bus 876 is connected to various components via an interface 877.
  • the input device 878 for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, or the like is used. Furthermore, as the input device 878, a remote controller (hereinafter referred to as a remote controller) capable of transmitting a control signal using infrared rays or other radio waves may be used.
  • the input device 878 includes a voice input device such as a microphone.
  • the output device 879 is a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or a headphone, a printer, a mobile phone, or a facsimile. It is a device that can be notified visually or audibly.
  • the output device 879 according to the present disclosure includes various vibration devices that can output a tactile stimulus.
  • the storage 880 is a device for storing various data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used.
  • the drive 881 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 901.
  • a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory
  • the removable recording medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, or various semiconductor storage media.
  • the removable recording medium 901 may be, for example, an IC card on which a non-contact IC chip is mounted, an electronic device, or the like.
  • connection port 882 is a port for connecting an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • the external connection device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.
  • the communication device 883 is a communication device for connecting to a network.
  • the information processing apparatus uses the weight vector quantized based on the granularity in the vector direction on the surface of the N-dimensional hypersphere, and uses the inner product of the weight vector and the corresponding input vector.
  • One of the features is to perform an operation. According to such a configuration, it is possible to further reduce the processing load related to the inner product calculation and to guarantee the quantization granularity of the weight coefficient.
  • a calculation unit that calculates an output value by performing an inner product calculation based on a plurality of input values and a plurality of weighting factors corresponding to the input values, With The calculation unit calculates the output value based on the weighting coefficient quantized based on the particle size in the vector direction on the surface of the N-dimensional hypersphere. Information processing device.
  • the calculation unit calculates the output value based on the weighting factor expressed by a power.
  • the bottom of the weight coefficient is determined to be a value greater than 0 and less than 1 in the inner product calculation by the calculation unit.
  • the information processing apparatus according to (2) is
  • the number of multiplications in the inner product calculation is determined based on the bottom value of the weighting coefficient.
  • the calculation unit performs an inner product calculation based on a plurality of the weighting factors rearranged in ascending order of values.
  • the calculation unit performs an inner product calculation based on the plurality of weighting factors normalized by the weighting factor having the largest value.
  • the arithmetic unit obtains the input value based on a table holding address information of the input value corresponding to the plurality of weighting factors rearranged in order of increasing value, and performs an inner product operation.
  • the address information includes an offset indicating a relative position between addresses.
  • the information processing apparatus according to (7). The table further holds multiplication instruction information associated with the address information.
  • the multiplication instruction information includes at least information designating necessity of multiplication, The information processing apparatus according to (9).
  • the table further holds code information associated with the address information.
  • the calculation unit performs an inner product calculation related to forward propagation of a neural network.
  • a product-sum operation circuit that performs a product-sum operation based on a plurality of input values and a plurality of weighting factors corresponding to the input values
  • the product-sum operation circuit includes a storage circuit that holds address information of the input values corresponding to the plurality of weighting factors rearranged in ascending order of values;
  • a multiplication circuit that performs an operation based on the input value acquired from the address information and the weighting factor; Comprising Information processing device.
  • a processor performs a product-sum operation based on a plurality of input values and a plurality of weighting factors respectively corresponding to the input values to calculate an output value; Including Calculating the output value based on the weighting coefficient quantized based on the granularity in the vector direction on the surface of the N-dimensional hypersphere; Further including Information processing method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Algebra (AREA)
  • Operations Research (AREA)
  • Complex Calculations (AREA)
PCT/JP2018/002404 2017-04-28 2018-01-26 情報処理装置、および情報処理方法 Ceased WO2018198454A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN201880026324.9A CN110574024A (zh) 2017-04-28 2018-01-26 信息处理设备和信息处理方法
EP18792207.5A EP3617904A4 (en) 2017-04-28 2018-01-26 INFORMATION PROCESSING DEVICE AND PROCESSING PROCESS
US16/479,044 US11030524B2 (en) 2017-04-28 2018-01-26 Information processing device and information processing method
JP2019505277A JPWO2018198454A1 (ja) 2017-04-28 2018-01-26 情報処理装置、および情報処理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-089144 2017-04-28
JP2017089144 2017-04-28

Publications (1)

Publication Number Publication Date
WO2018198454A1 true WO2018198454A1 (ja) 2018-11-01

Family

ID=63919552

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/002404 Ceased WO2018198454A1 (ja) 2017-04-28 2018-01-26 情報処理装置、および情報処理方法

Country Status (5)

Country Link
US (1) US11030524B2 (https=)
EP (1) EP3617904A4 (https=)
JP (3) JPWO2018198454A1 (https=)
CN (1) CN110574024A (https=)
WO (1) WO2018198454A1 (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019114231A (ja) * 2017-12-21 2019-07-11 富士通株式会社 ニューラルネットワークパラメータの量子化
CN114297576A (zh) * 2021-11-16 2022-04-08 平头哥(上海)半导体技术有限公司 一种加权平均计算方法和加权平均计算装置
JP7664451B2 (ja) 2023-04-28 2025-04-17 楽天グループ株式会社 計算システム、計算方法、及びプログラム

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019080232A (ja) * 2017-10-26 2019-05-23 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 勾配圧縮装置、勾配圧縮方法及びプログラム
JP7475164B2 (ja) * 2020-03-05 2024-04-26 キヤノン株式会社 情報処理装置、情報処理方法およびプログラム
KR102929805B1 (ko) 2022-12-16 2026-02-24 한국전자기술연구원 컨볼루션 연산 하드웨어에서 적응적으로 양자화 팩터를 결정하는 방법
TWI877065B (zh) * 2024-08-16 2025-03-11 英業達股份有限公司 風扇裝置

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006180093A (ja) * 2004-12-21 2006-07-06 Nec Electronics Corp キャンセラ装置及びデータ伝送システム

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5651090A (en) * 1994-05-06 1997-07-22 Nippon Telegraph And Telephone Corporation Coding method and coder for coding input signals of plural channels using vector quantization, and decoding method and decoder therefor
JP3357829B2 (ja) * 1997-12-24 2002-12-16 株式会社東芝 音声符号化／復号化方法
US6665646B1 (en) * 1998-12-11 2003-12-16 At&T Corp. Predictive balanced multiple description coder for data compression
US6356213B1 (en) * 2000-05-31 2002-03-12 Lucent Technologies Inc. System and method for prediction-based lossless encoding
WO2002037476A1 (en) * 2000-11-03 2002-05-10 Koninklijke Philips Electronics N.V. Sinusoidal model based coding of audio signals
US20030220889A1 (en) * 2002-05-21 2003-11-27 Bingxue Shi Analog accumulator for neural networks
JP4698593B2 (ja) * 2004-07-20 2011-06-08 パナソニック株式会社 音声復号化装置および音声復号化方法
US8332216B2 (en) * 2006-01-12 2012-12-11 Stmicroelectronics Asia Pacific Pte., Ltd. System and method for low power stereo perceptual audio coding using adaptive masking threshold
US8392176B2 (en) * 2006-04-10 2013-03-05 Qualcomm Incorporated Processing of excitation in audio coding and decoding
BRPI0718738B1 (pt) * 2006-12-12 2023-05-16 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Codificador, decodificador e métodos para codificação e decodificação de segmentos de dados representando uma corrente de dados de domínio de tempo
US7958071B2 (en) * 2007-04-19 2011-06-07 Hewlett-Packard Development Company, L.P. Computational nodes and computational-node networks that include dynamical-nanodevice connections
US8248660B2 (en) * 2007-12-14 2012-08-21 Qualcomm Incorporated Efficient diffusion dithering using dyadic rationals
CN101771477B (zh) * 2008-12-29 2013-06-05 深圳富泰宏精密工业有限公司 手机射频发射功率校正系统及方法
EP2259214B1 (en) * 2009-06-04 2013-02-27 Honda Research Institute Europe GmbH Implementing a neural associative memory based on non-linear learning of discrete synapses
US8990132B2 (en) * 2010-01-19 2015-03-24 James Ting-Ho Lo Artificial neural networks based on a low-order model of biological neural networks
WO2012093290A1 (en) * 2011-01-05 2012-07-12 Nokia Corporation Multi-channel encoding and/or decoding
CN103325373A (zh) * 2012-03-23 2013-09-25 杜比实验室特许公司 用于传送和接收音频信号的方法和设备

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006180093A (ja) * 2004-12-21 2006-07-06 Nec Electronics Corp キャンセラ装置及びデータ伝送システム

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
DAISUKE MIYASHITA ET AL.: "Convolutional Neural Networks using Logarithmic Data Representation", ARXIV, 3 March 2016 (2016-03-03), Retrieved from the Internet <URL:https://arxivorg/pdf/1603.01025.pdf>
MACHADO, EMERSON ET AL.: "FPGA Implementation of a Feedforward Neural Network-Based Classifier Using the xQuant Technique", PROCEEDINGS OF 2017 IEEE 8TH LATIN AMERICAN SYMPOSIUM ON CIRCUITS AND SYSTEMS, 20 February 2017 (2017-02-20), pages 1 - 4, XP033106363, ISSN: 2473-4667 *
MATTHIEU COURBARIAUX ET AL.: "BinaryConnect: Training Deep Neural Networks with binary weights during propagations", ARXIV, 11 November 2015 (2015-11-11), Retrieved from the Internet <URL:https://arxiv.org/pdf/1511.00363.pdf>
See also references of EP3617904A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019114231A (ja) * 2017-12-21 2019-07-11 富士通株式会社 ニューラルネットワークパラメータの量子化
CN114297576A (zh) * 2021-11-16 2022-04-08 平头哥(上海)半导体技术有限公司 一种加权平均计算方法和加权平均计算装置
JP7664451B2 (ja) 2023-04-28 2025-04-17 楽天グループ株式会社 計算システム、計算方法、及びプログラム

Also Published As

Publication number Publication date
US11030524B2 (en) 2021-06-08
JP2020074240A (ja) 2020-05-14
JP6958652B2 (ja) 2021-11-02
CN110574024A (zh) 2019-12-13
US20200387781A1 (en) 2020-12-10
EP3617904A4 (en) 2020-04-29
JP6690765B2 (ja) 2020-04-28
JPWO2018198454A1 (ja) 2019-06-27
EP3617904A1 (en) 2020-03-04
JP2020004433A (ja) 2020-01-09

Similar Documents

Publication Publication Date Title
JP6958652B2 (ja) 情報処理装置、および情報処理方法
JP7103289B2 (ja) 情報処理装置、および情報処理方法
US11741348B2 (en) Neural network circuit device, neural network, neural network processing method, and neural network execution program
CN107480770B (zh) 可调节量化位宽的神经网络量化与压缩的方法及装置
EP3657399A1 (en) Weight pruning and quantization method for a neural network and accelerating device therefor
WO2022006919A1 (zh) 基于激活定点拟合的卷积神经网络训练后量化方法及系统
JP2018147428A (ja) 情報処理装置、情報処理方法、およびプログラム
JP7287388B2 (ja) 情報処理装置および情報処理方法
CN112771547A (zh) 通信系统中的端到端学习
WO2019141193A1 (zh) 对视频帧数据进行处理的方法和装置
Langroudi et al. Alps: Adaptive quantization of deep neural networks with generalized posits
WO2022197615A1 (en) Techniques for adaptive generation and visualization of quantized neural networks
CN120318591A (zh) 一种图像识别与分类方法、系统、设备及介质
CN111930670B (zh) 异构智能处理量化装置、量化方法、电子设备及存储介质
US20220334802A1 (en) Information processing apparatus, information processing system, and information processing method
CN118364884A (zh) 神经网络模型的量化方法、装置和存储介质
CN115062777A (zh) 卷积神经网络的量化方法、神经网络、设备及存储介质
CN116188875B (zh) 图像分类方法、装置、电子设备、介质和产品
JP2023031367A (ja) 閾値決定プログラム及び閾値決定方法
CN114861901B (zh) 神经网络模型的运行方法、可读介质和电子设备
CN111178497A (zh) 特征向量二值化或三值化算法、装置及计算机可读存储介质
CN116976403A (zh) 一种网络训练方法、装置、计算机设备和存储介质
CN115964651A (zh) 一种变分模态分解参数确定方法、装置和存储介质
JP2021071934A (ja) 計算装置、複素数の絶対値の計算方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18792207

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019505277

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2018792207

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2018792207

Country of ref document: EP

Effective date: 20191128