EP3659073A1 - Appareil électronique et procédé de commande associé - Google Patents

Appareil électronique et procédé de commande associé

Info

Publication number
EP3659073A1
EP3659073A1 EP18866233.2A EP18866233A EP3659073A1 EP 3659073 A1 EP3659073 A1 EP 3659073A1 EP 18866233 A EP18866233 A EP 18866233A EP 3659073 A1 EP3659073 A1 EP 3659073A1
Authority
EP
European Patent Office
Prior art keywords
processing elements
zero element
zero
input
depth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP18866233.2A
Other languages
German (de)
English (en)
Other versions
EP3659073A4 (fr
Inventor
Kyung-Hoon Kim
Young-Hwan Park
Dong-Kwan Suh
Keshava PRASADNAGARAJA
Dae-Hyun Kim
Suk-Jin Kim
Han-Su CHO
Hyun-Jung Kim
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority claimed from PCT/KR2018/006509 external-priority patent/WO2019074185A1/fr
Publication of EP3659073A1 publication Critical patent/EP3659073A1/fr
Publication of EP3659073A4 publication Critical patent/EP3659073A4/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/15—Correlation function computation including computation of convolution operations
    • G06F17/153—Multidimensional correlation or convolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Definitions

  • the present disclosure relates generally to an electronic apparatus and a controlling method thereof and, more particularly, to an electronic apparatus and a control method for performing a convolution operation.
  • a touch sensing device such as a touch pad, is capable of providing an input method using its own body without a separate input device such as a mouse or a keyboard.
  • the touch sensing device is commonly applied to portable electronic devices for which a separate input device, such as a notebook, is difficult to be used.
  • artificial intelligence systems that implement human-level intelligence have been used in various fields.
  • an artificial intelligence system a machine learns, makes determinations, and becomes smarter, unlike an existing rule-based smart system.
  • Artificial intelligence systems are becoming more and more common, and existing rule-based smart systems are being replaced by these types of deep-learning-based artificial intelligence systems.
  • Machine learning e.g., deep learning
  • elementary technologies that utilize machine learning.
  • Machine learning includes an algorithm technology that classifies/learns characteristics of input data by itself.
  • Elementary technology simulates functions, such as recognition and judgment of human brain, using machine learning algorithms, such as deep learning.
  • the elementary technology includes technology fields, such as linguistic understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control.
  • Artificial intelligence technology may by applied in linguistic understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control.
  • Linguistic understanding is a technology for recognizing, applying/processing human language/characters and includes natural language processing, machine translation, dialogue system, query response, speech recognition/synthesis, etc.
  • Visual understanding is a technology for recognizing and processing objects as human vision, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image enhancement, etc.
  • Reasoning/prediction is technology for determining information, logically reasoning, and predicting information, including knowledge/probability based reasoning, optimization prediction, preference-based planning, and recommendation.
  • Knowledge representation is a technology for automating human experience information into knowledge data, including knowledge building (data generation/classification) and knowledge management (data utilization).
  • Motion control is a technology for controlling the self-driving of a vehicle and the motion of the robot, including motion control (navigation, collision, driving), and manipulation control (behavior control), etc.
  • CNN convolutional neural network
  • a CNN has a structure for learning two-dimensional data or three-dimensional data, and can be trained through a backpropagation algorithm.
  • a CNN is widely used in various application fields, such as object classification, object detection, etc.
  • Most operations of a CNN are convolution operations, and most of the convolution operations include multiplication processing between input data.
  • the target data e.g., an image
  • the kernel data that are input data may include a plurality of zeros, and as such, it is unnecessary to perform a multiplication operation in these cases.
  • the multiplication result is zero. That is, if at least one of the input data is zero, even if the multiplication operation is not performed, it can be known that the result is zero. Therefore, an operation cycle can be shortened by omitting unnecessary multiplication operations, which are expressed as processing data sparsity.
  • an aspect of the present disclosure is to provide an electronic apparatus that omits an unnecessary operation in a convolution operation process to improve an operation speed and a control method thereof.
  • Another aspect of the present disclosure is to provide an electronic apparatus that may improve speed of a convolution operation by omitting an operation of part of target data and part of kernel data according to zero included in the target data and a control method thereof.
  • an electronic apparatus for performing deep learning.
  • the electronic apparatus includes a storage configured to store target data and kernel data; and a processor configured to include a plurality of processing elements that are arranged in a matrix shape, and the processor is configured to input, to each of the plurality of processing elements, a first non-zero element from among a plurality of first elements included in the target data, and sequentially input, to each of a plurality of first processing elements included in a first row from among the plurality of processing elements, a second non-zero element from among the plurality of elements included in the kernel data, wherein each of the plurality of first processing elements is configured to perform operation between the input first non-zero element and the input second non-zero element based on depth information of the first non-zero element and depth information of the second non-zero element.
  • a method for controlling an electronic apparatus to perform deep learning. The method includes inputting, to each of the plurality of processing elements, a first non-zero element from among a plurality of first elements included in the target data; sequentially inputting, to each of a plurality of first processing elements included in a first row from among the plurality of processing elements, a second non-zero element from among the plurality of elements included in the kernel data; and performing operation between the input first non-zero element and the input second non-zero element based on depth information of the first non-zero element and depth information of the second non-zero element.
  • the electronic apparatus may improve speed of the convolution operation by omitting the operation of part of the target data and part of the kernel data according to zero included in the target data.
  • FIGs. 1A and 1B illustrate a convolution operation between three-dimensional input data according to an embodiment
  • FIG. 2 illustrates an electronic apparatus according to an embodiment
  • FIG. 3 illustrates a plurality of processing elements according to an embodiment
  • FIGs. 4A to 4D illustrate a method for inputting a non-zero element from among target data and kernel data according to an embodiment
  • FIGs. 5A to 5M illustrate operation cycles of a processing element according to an embodiment
  • FIGs. 6A and 6B illustrate a method for processing data sparsity of kernel data according to an embodiment
  • FIGs. 7A and 7B illustrate a method for processing data sparsity of target data according to an embodiment
  • FIG. 8 illustrates a processing element according to an embodiment
  • FIG. 9 is a flowchart illustrating a method of controlling an electronic apparatus according to an embodiment.
  • FIGs. 1A and 1B illustrate a convolution operation between three-dimensional input data according to an embodiment.
  • a convolution operation is an operation performed with a very high weight in deep learning, which highlights characteristics corresponding to kernel data from target data through operation of the target data and the kernel data.
  • the left side of FIG. 1A illustrates an example of three-dimensional target data (Feature Map Data), and the right side of FIG. 1A illustrates an example of three-dimensional kernel data.
  • the target data is three-dimensional data including four rows, four columns, and a depth of five
  • the kernel data is three-dimensional data including two rows, two columns, and a depth of five.
  • the output data is two-dimensional data including three rows and three columns.
  • Out11 can be calculated using Equation (1).
  • Out11 F11,1 ⁇ A,1 + F11,2 ⁇ A,2 + F11,3 ⁇ A,3 + F11,4 ⁇ A,4 + F11,5 ⁇ A,5 + F12,1 ⁇ B,1 + F12,2 ⁇ B,2 + F12,3 ⁇ B,3 + F12,4 ⁇ B,4 + F12,5 ⁇ B,5 + F21,1 ⁇ D,1 + F21,2 ⁇ D,2 + F21,3 ⁇ D,3 + F21,4 ⁇ D,4 + F21,5 ⁇ D,5 + F22,1 ⁇ C,1 + F22,2 ⁇ C,2 + F22,3 ⁇ C,3 + F22,4 ⁇ C,4 + F22,5 ⁇ C,5 ...
  • the left side of the comma of F11,1 indicates the row and column of the target data
  • the right side of F11,1 indicates the depth of the target data.
  • F21,3 indicates the second row, the first column and the third depth of the target data, and the remaining target data are also displayed in the same manner.
  • the left comma of A,1 indicates the row and column of the kernel data
  • the right side of the comma indicates the depth of the kernel data.
  • D,4 represents the second row, the first column and the fourth depth of the kernel data, and the remaining kernel data are displayed in the same manner.
  • the above-described notation is used for easier description.
  • the remainder of the output data can be calculated by operating the same kernel data and other rows and columns of the target data.
  • Out23 out of the output data can be calculated by operating the data included in all of the depths of F23, F24, F33, and F34 and the kernel data from the target data.
  • the depth of the three-dimensional input data needs to be the same. Further, even if the input data is three-dimensional data, the output data can be changed into two-dimensional data.
  • FIG. 1B illustrates a result of omitting an operation with respect to the outline pixels of the target data, and another type of output data may be generated as the operation on the outline pixel is added.
  • individual data which constitutes target data, such as F11,1, F11,2, F11,3, F11,4, F11,5, F21,1 ..., F44,4, and F44,5, is described as a first element; individual data, which constitutes kernel data, such as A,1, A,2, A,3, A,4, B,1, ..., C,4, D,1, D,2, D,3, and D,4, is described as a second element.
  • target data such as F11,1, F11,2, F11,3, F11,4, F11,5, F21,1 ..., F44,4, and F44,5
  • kernel data such as A,1, A,2, A,3, A,4, B,1, ..., C,4, D,1, D,2, D,3, and D,4 is described as a second element.
  • the reference directions of the rows, columns, and depths illustrated in FIGs. 1A and 1B are the same in the following drawings.
  • FIG. 2 illustrates an electronic apparatus according to an embodiment.
  • the electronic apparatus 100 includes a storage 110 and a processor 120.
  • the electronic apparatus 100 may perform deep learning, i.e., a convolution operation.
  • the electronic apparatus 100 may be a desktop personal computer (PC), a notebook, a smart phone, a tablet PC, a server, etc.
  • the electronic apparatus 100 may be a system itself, in which a cloud computing environment is built.
  • the present disclosure is not limited thereto, and the electronic apparatus 100 may be any device capable of performing a convolution operation.
  • the storage 110 may store target data, kernel data, etc.
  • the target data and the kernel data may be stored so as to correspond to a type of the storage 110.
  • the storage 110 may include a plurality of two-dimensional cells, and three-dimensional target data and kernel data may be stored in a plurality of two-dimensional cells.
  • the processor 120 may identify data stored in a plurality of two-dimensional cells as three-dimensional target data and kernel data. For example, the processor 120 may identify the data stored in cells 1 to 25, among the plurality of cells, as data of a first depth of the target data, and the data stored in cells 26 to 50, among the plurality of cells, as data of a second depth of the target data.
  • the kernel data may be generated by the electronic apparatus 100, or may generated and received by an external electronic apparatus, i.e., not the electronic apparatus 100.
  • the target data may be information received from an external electronic apparatus.
  • the storage 110 may be implemented as a hard disk, a non-volatile memory, a volatile memory, etc.
  • the processor 120 generally controls the operation of electronic apparatus 100.
  • the processor 120 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON), but is not limited thereto, and may include at least one of a central processing unit (CPU), a microcontroller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), and an ARM processor.
  • the processor 120 may be implemented as a system on chip (SoC), a large scale integration (LSI) with a processing algorithm embedded therein, or in a format of a field programmable gate array (FPGA).
  • SoC system on chip
  • LSI large scale integration
  • FPGA field programmable gate array
  • the processor 120 may include a plurality of processing elements arranged in a matrix form, and may control the operation of a plurality of processing elements.
  • FIG. 3 illustrates a plurality of processing elements according to an embodiment.
  • FIG. 3 a plurality of processing elements (PEs) are arranged in a matrix form, and data can be shared between adjacent processing elements.
  • FIG. 3 illustrates the data being transmitted from an upper side to a lower side, the present disclosure is not limited thereto, and data may be transmitted from the lower side to the upper side.
  • Each of the plurality of processing elements includes a multiplier and an arithmetic logic unit (ALU).
  • the ALU may include at least one adder.
  • Each of the plurality of processing elements can perform arithmetic operations using a multiplier and an ALU. Further, each of the plurality of processing elements may include a plurality of register files.
  • the processor 120 may input a first non-zero element among the plurality of first elements included in the target data to each of the plurality of processing elements. For example, the processor 120 may identify a first non-zero element, i.e., an element that is not zero, from the target data stored in the storage 100, and input the identified first non-zero element into the plurality of processing elements. That is, the processor 120 may extract only the first non-zero element from the target data stored in the storage 110 in real time.
  • a first non-zero element i.e., an element that is not zero
  • the processor 120 may extract only the first non-zero element from the target data, prior to inputting the first non-zero element to the plurality of processing elements, and store the first non-zero element in the storage 110.
  • the storage 110 may store the target data and the extracted first non-zero element.
  • the processor 120 may directly input the extracted first non-zero element into the plurality of processing elements.
  • the processor 120 may identify the corresponding processing element among the plurality of processing elements based on the row information and the column information of the first non-zero element, and input the first non-zero element to the identified processing element.
  • the processor 120 may be configured to input the first non-zero element to a first processing element from among a plurality of processing elements, if the first non-zero element is a first row and a first column, and if the first non-zero element is the second row and the second column, the first non-zero element may be input to the second processing element from among a plurality of processing elements.
  • the first non-zero element which belongs to the first row and the first column, may include a plurality of elements with different depths, and the processor 120 may input a plurality of first non-zero elements belonging to the first row and the first column to each of a plurality of register files of the first processing element.
  • the processor 120 may input the first non-zero element into the corresponding register file from among the plurality of register files included in the processing element identified based on the depth information of the first non-zero element.
  • the processing element may include a plurality of register files corresponding to each of the depths of the target data.
  • the processing element may include a first register file corresponding to the first depth of the target data, a second register file corresponding to the second depth, ..., and an n-th register file corresponding to the n-th depth
  • the processor 120 may input an element of the first depth from among the first non-zero elements belonging to the first row and the first column to the first register file included in the first processing element, and input the element of the second depth to the second register file included in the first processing element. If there is no element of the second depth from among the first non-zero elements belonging to the first row and the first column, the second register file included in the first processing element may not store the element.
  • the processor 120 may sequentially input the first non-zero element into a plurality of register files included in the identified processing element, without considering the depth information of the first non-zero element.
  • the processor 120 may store the depth information of the first non-zero element stored in each register file along with the first non-zero element.
  • the processor 120 may input the first non-zero element to the first register, file, the second register file, and the third register file sequentially.
  • the processor 120 may store that the first non-zero element stored in the first register file as an element of the first depth, the first non-zero element stored in the second register file as an element of the third depth, and the first non-zero element stored in the third register file as an element of the fourth depth.
  • the processor 120 may sequentially input the second non-zero element from among a plurality of second elements included in the kernel data to each of the plurality of first processing elements included in the first row among the plurality of processing elements.
  • the processor 120 may identify the second non-zero element from the kernel data stored in the storage 110 and sequentially input the identified second non-zero element to each of the plurality of first processing elements. That is, the processor 120 may extract only the second non-zero element in real time from the kernel data stored in the storage 110.
  • an operation to sequentially input refers to the input order of the elements in the plurality of second non-zero elements.
  • the processor 120 may input the second non-zero element of the first depth to each of the plurality of first processing elements in the first cycle, input the second non-zero element of the second depth to each of the plurality of first processing elements in the second cycle, and input the second non-zero element of the third depth to each of the plurality of first processing elements in the third cycle.
  • the processor 120 may extract only the second non-zero element from the kernel data, before inputting the second non-zero element to each of the plurality of first processing elements, and store the extracted second non-zero element in the storage 110.
  • the storage 110 may store the kernel data and the extracted second non-zero element.
  • the processor 120 may sequentially input the extracted second non-zero element into each of the plurality of first processing elements.
  • the plurality of first processing elements included in the first row among the plurality of processing elements may be a plurality of processing elements arranged at one corner of the plurality of processing element matrices.
  • the plurality of first processing elements may be four processing elements arranged at the top portion of FIG. 3.
  • the processor 120 may sequentially input the second non-zero element to each of the plurality of first processing elements based on the row information, the column information, and the depth information of the second non-zero element.
  • the processor 120 may sequentially input the second non-zero element along with the depth information of the second non-zero element to the plurality of first processing elements.
  • the processor 120 sequentially inputs the second non-zero element included in one row and one column of the second non-zero elements to each of the plurality of first processing elements based on the depth. When all of the second non-zero elements included in one row and one column are input to each of the plurality of first processing elements, the second non-zero element included in a row and a column, which are different from one row and column, to each of the plurality of first processing elements.
  • the processor 120 may sequentially input the second non-zero element included in a first row and a second column to each of the plurality of first processing elements, and when input of the second non-zero element included in the first row and the first column is completed, the processor may sequentially input the second non-zero element included in the first row and the second column to each of the plurality of first processing elements in an order of depth.
  • the processor 120 may input one second non-zero element into each of the plurality of first processing elements, and when the cycle is changed, may input the second non-zero element in a next order to each of the plurality of first processing elements.
  • the processor 120 inputs a zero into each of the plurality of first processing elements when there is no second non-zero element in one row and one column, and when zero is input to each of the plurality of first processing elements, may input the second non-zero element or zero included in a different row or column to each of a plurality of the first processing elements based on the number of second non-zero elements included in a different row and column.
  • the processor 120 when a depth which has no first non-zero element in all of the rows and columns is identified from among first non-zero elements stored in each of the plurality of processing elements, may omit input of a second non-zero element that corresponds to the depth and sequentially input the second non-zero element that does not correspond to the depth to each of the plurality of first processing elements.
  • the processor 120 may omit input of the second non-zero element corresponding to the third depth from among the second elements.
  • the processor 120 may input the element of the first depth from among the second non-zero elements belonging to the first row and the first column to each of the plurality of processing elements, and if a cycle is changed, the processor 120 may input the element of the fourth depth to each of the plurality of first processing elements from among the second non-zero element belonging to the first row and the first column.
  • the processor 120 may shorten the cycle by not inputting the element of the third depth from among the second non-zero elements belonging to the first row and the first column.
  • the processor 120 may further include a plurality of preliminary processing elements.
  • the processor 120 may omit input of the second non-zero element corresponding to the depth and sequentially input the second non-zero elements not corresponding to the depth to each of the plurality of first processing elements, and input the first non-zero element corresponding to the depth and the second non-zero element corresponding to the depth to a plurality of preliminary processing elements to perform the operation.
  • the processor 120 may omit input of the second non-zero element corresponding to the third depth and sequentially input the second non-zero elements not corresponding to the third depth to each of the plurality of first processing elements, and input the first non-zero element corresponding to the third depth and the second non-zero element corresponding to the third depth to a plurality of preliminary processing elements to perform the operation.
  • Each of the plurality of first processing elements may perform an operation on the input first non-zero element and the input second subject, based on the depth information of the first non-zero element and the depth information of the second non-zero element.
  • the remaining processing elements from among the plurality of processing elements may receive the second non-zero elements from the adjacent processing elements.
  • Each of the remaining processing elements may perform an operation between the input first non-zero element and the input second non-zero element based on the depth information of the first non-zero element and the depth information of the second non-zero element.
  • the first non-zero element and the second non-zero element can be input to each of the plurality of processing elements on a cycle-by-cycle basis.
  • each of the plurality of processing elements can perform operation between the first non-zero elements and the second non-zero elements that are input by cycles based on the respective depth information.
  • the first non-zero element may be preliminarily input to the plurality of processing elements at a time, and the second non-zero element may be input to each of the plurality of processing elements for each cycle.
  • each of the plurality of processing elements may perform an operation between a prestored first non-zero element and a second non-zero element, which is input by cycles, based on the respective depth information.
  • the processor 120 may control the plurality of processing elements to shift the second non-zero elements that are input to the plurality of first processing elements to each of the plurality of second processing elements included in the second row.
  • the processor 120 may control the plurality of processing elements to shift the second non-zero elements which are shift to the plurality of second processing elements to each of the plurality of third processing elements included in the third row from among the plurality of processing elements.
  • the processor 120 may accumulate the operation result by the input second non-zero element to the previous operation result and store the accumulated operation result in one of the plurality of register files.
  • the plurality of register files may include a register file for accumulating and storing a plurality of register files in which the first non-zero element is stored and the operation result.
  • the processor 120 may shift the operation result that is stored in one of the plurality of register files of the plurality of processing elements to an adjacent processing element, and store the operation result by the input second non-zero element to one of the plurality of register files by accumulating the operation result to the shifted operation result.
  • the processor 120 may shorten unnecessary operations between the target data and the kernel data.
  • FIGs. 4A to 4D illustrate a method for inputting a non-zero element from among target data and kernel data according to an embodiment.
  • FIG. 4A the left side of FIG.4A illustrates three-dimensional target data, and the right side of FIG. 4A illustrates three-dimensional first kernel data and the three-dimensional second kernel data.
  • the kernel data is sequentially input to the plurality of first processing elements, it is possible to easily operate a plurality of kernel data.
  • the first arrow direction toward the right upper end indicates the depth direction
  • the second arrow direction rotating in a clockwise direction indicates the order of operation of the kernel data.
  • FIG. 4B the left upper end of FIG. 4B illustrates the first row in the target data, and the lower left end of FIG. 4B illustrates the second row in the target data.
  • the arrow direction represents the depth direction as shown by the first arrow direction in FIG. 4A.
  • the number shown on the left side of FIG. 4B represents the index of the depth, and the element is not zero, and the depth without the number represents when the element is zero.
  • the elements of the first depth, the fourth depth, and the fifth depth are not zero, and the elements of the second depth and the third depth are zero.
  • the right side of FIG. 4B illustrates only the first non-zero element from the left side of FIG. 4B.
  • the processor 120 may identify the first non-zero element from the target data as shown in the left side of FIG. 4B, and input the identified first non-zero element into the plurality of processing elements.
  • the processor 120 may extract only the first non-zero element as shown in the right side of FIG. 4B, separately store the extracted first non-zero elements in the storage 110, and extract the stored first non-zero elements to input the elements to the plurality of processing elements.
  • the processor 120 may first extract the first non-zero element in a depth direction of F11 of the first row, and then move to the side to extract the first non-zero element in the depth direction of F12 as illustrated in FIG. 4B.
  • the processor 120 may extract the first non-zero element in the depth direction of each of F13 and F14 in the same manner.
  • the processor 120 may extract the first non-zero element for the second row in the same manner.
  • FIG. 4B only the first row and the second row are shown in the target data for convenience of description, and only the first row and the second row of the target data will be described below for convenience of description. However, the operation for the remaining rows is the same as for the first row and the second row.
  • FIG. 4C the left side of FIG. 4C illustrates first kernel data and second kernel data in accordance with a row and a column.
  • the direction of the arrow in FIG. 4C indicates the depth direction as shown by the first arrow direction in FIG. 4A.
  • the numbers illustrated on the left side of FIG. 4C represent the index of the depth and that the element is not zero, and a depth without a number represents that the element is a zero.
  • the elements of the first and third depths are not zero, and the elements of the second depth, fourth depth, and fifth depth are zero.
  • the right side of FIG. 4C illustrates only the second non-zero element from the left side of FIG. 4C.
  • the processor 120 may identify the second non-zero element from the kernel data as illustrated in the left side of FIG. 4C, and sequentially input the identified second non-zero element into the plurality of first processing elements.
  • the processor 120 may extract only the second non-zero element as shown in the right side of FIG. 4C, separately store the identified second non-zero element in the storage 110, extract the stored second non-zero element to sequentially input to the plurality of the first processing element.
  • the processor 120 may first extract the first non-zero element in a depth direction as illustrated in FIG. 4C, and then move to the side to extract the second non-zero element in the depth direction of B as shown in FIG. 4C.
  • the processor 120 may extract the second non-zero element in the depth direction of each of C and D in the same manner.
  • the processor 120 may include a plurality of processing elements in the form of 4 ⁇ 4 matrix, e.g., as illustrated in FIG. 4D.
  • the four processing elements included in the first row 410 at the upper end of the plurality of processing elements are referred to as a plurality of the first processing elements.
  • the processor 120 may input the first non-zero element included in the first row of the target data to the plurality of the first processing elements.
  • the processor 120 may input the elements of the first depth, the fourth depth, and the fifth depth included in the first row and the first column of the target data to a processing element located in the first from the left side from among the plurality of first processing elements, input the elements of the first depth, the third depth, and the fourth depth included in the first row and the second column of the target data to a processing element located in the second from the left side from among the plurality of first processing elements, input the elements of the first depth, the third depth, and the fifth depth included in the first row and the third column of the target data to a processing element located in the third from the left side from among the plurality of first processing elements, and input the elements of the first depth, the second depth, and the fifth depth included in the first row and the fourth column of the target data to a processing element located in the fourth from the left side from among the plurality of first processing elements.
  • the processor 120 may input the first non-zero element included in the second row of the target data to four processing elements (hereinafter, referred to as the plurality of second processing elements) included in a row that is positioned below the first row 410.
  • the processor 120 may input the elements of the first depth, the second depth, the third depth, and the fourth depth included in the second row and the first column of the target data to a processing element located in the first from the left side from among the plurality of second processing elements, input the elements of the fourth depth and the fifth depth included in the second row and the second column of the target data to a processing element located in the second from the left side from among the plurality of the second processing elements, input the elements of the third depth included in the second row and the third column of the target data to a processing element located in the third from the left side from among the plurality of the second processing elements, and input the elements of the second depth, the third depth, the forth depth, and the fifth depth included in the second row and the fourth column of the target data to a processing element located in the
  • the processor 120 may sequentially input the second non-zero element included in the first row and the first column of the first kernel data to a plurality of the first processing elements in an order of depth.
  • the processor 120 may sequentially input the second non-zero element included in the first row and the first column of the first kernel data to the plurality of first processing elements, sequentially input the second non-zero element included in the first row and the second column of the first kernel data to the plurality of first processing elements, sequentially input the second non-zero elements included in the second row and the second column of the first kernel data to the plurality of the first processing elements, and sequentially input the second non-zero elements included in the second row and the first column of the first kernel data to a plurality of the first processing elements.
  • the processor 120 may sequentially input the second non-zero element included in the first kernel data to the plurality of first processing elements, and sequentially input the second non-zero elements included in the second kernel data to the plurality of the first processing elements.
  • the processor 120 may sequentially input the elements of the first depth and the third depth included in the first row and the first column of the first kernel data to the plurality of first processing elements, sequentially input the elements of the first depth, second depth, third depth, fourth depth, and fifth depth included in the first row and the second column of the first kernel data to a plurality of the first processing elements, and sequentially input the elements of the first depth, second depth, third depth, and fifth depth included in the second row and the second column to the plurality of first processing elements.
  • the processor 120 may input zero to the plurality of the first processing elements if the second non-zero element is not included in the second row and the first column of the first kernel data.
  • the processor 120 may sequentially input the second non-zero element of the second kernel data to the plurality of the first processing elements, and the input order may be the same as the first kernel data.
  • the processor 120 may input one second non-zero element into the plurality of first processing elements, and sequentially input another second non-zero element to the plurality of first processing elements when the cycle is changed.
  • Each of the plurality of first processing elements can shift the input second non-zero element to an adjacent second processing element from among a plurality of the second processing elements when the cycle is changed.
  • Each of the plurality of the second processing elements can shift the input second non-zero element to an adjacent processing element in a lower direction.
  • the processor 120 may input all of the first non-zero elements into the plurality of processing elements in the first cycle, and input the second non-zero element, which is the first, to the plurality of first processing elements. Thereafter, the processor 120 may input the second non-zero element, which is the second, to the plurality of first processing elements in the second cycle which follows the first cycle. That is, the processor 120 may only input the second non-zero element to the plurality of first processing elements in a following cycle.
  • the processor 120 may input all of the first non-zero elements in the first cycle and the first non-zero elements corresponding to the plurality of first processing elements into a plurality of first processing elements, and input the second non-zero element, which is the first, to the plurality of first processing elements. Thereafter, the processor 120 may input the first non-zero element, which corresponds to the plurality of second processing elements, to the plurality of second processing elements in the second cycle, and input the second non-zero element, which is the second, to the plurality of second processing elements. That is, the processor 120 may input a part of the first non-zero element to a plurality of the first processing element by cycles.
  • FIGs. 5A to 5M illustrate an operation of a processing an element by cycles according to an embodiment.
  • FIGs. 5A to 5M will be described with reference to the plurality of first processing elements and the plurality of second processing elements in FIGs. 4A to 4D.
  • FIGS. 5A to 5M illustrate a plurality of first processing elements on an upper side and a plurality of second processing elements on a lower side. Further, in each processing element, the left side represents the first non-zero element, the middle side indicates the second non-zero element, and the right side indicates the processing result.
  • the left upper side of FIG. 5A illustrates one of the plurality of first processing elements
  • the left side 510 represents the first non-zero elements of the first depth, the fourth depth, and the fifth depth included in the first row and the first column in the target data
  • the middle element 520 indicates the second non-zero element of the first depth included in the first row and the first column in the first kernel data
  • the right side 530 indicates the operation result.
  • the description of the concrete operation result value is omitted in the right side 530.
  • the processor 120 may input the first non-zero element into a first plurality of processing elements and a plurality of second processing elements in a first cycle.
  • the present disclosure is not limited thereto, and the processor 120 may input the first non-zero element to the plurality of first processing elements in the first cycle and input the first non-zero element to the plurality of second processing elements in the second cycle.
  • the processor 120 may input the first non-zero element corresponding to each processing element and further description will be omitted.
  • the processor 120 may input the second non-zero element to the first processing element in the first cycle.
  • the input second non-zero element is the second non-zero element of the first depth included in the first row and the first column of the first kernel data.
  • Each of the plurality of the first processing elements may perform an operation between the input first non-zero element and the input second non-zero element and store the operation result.
  • the input second non-zero element is the element of the first depth
  • the first, third, and fourth processing elements from the left side where the first non-zero element of the first depth is stored can perform an operation between the first non-zero element and the second non-zero element.
  • the second processing element from the left side in which the first non-zero element of the first depth is not stored does not perform operation between the first non-zero element and the second non-zero element.
  • the operation result is stored in each processing element and is not shifted to an adjacent processing element.
  • the plurality of second processing elements do not perform the operation because the second non-zero element is not input.
  • the processor 120 can input the second non-zero element to the plurality of first processing elements.
  • the input second non-zero element is the second non-zero element of the third depth included in the first row and the first column of the first kernel data.
  • Each of the plurality of first processing elements can shift the second non-zero element to the adjacent second processing element in the first cycle.
  • Each of the plurality of first processing elements may perform inter-element operation between the input first non-zero element and the input second non-zero element.
  • Each of the plurality of the first processing elements can shift the operation result to an adjacent processing element by adding the operation result of the second cycle with the operation result of the first cycle.
  • the reason for shifting is that all of the second non-zero elements included in the first row and the first column are input in the first kernel data. That is, the second non-zero element input in the second cycle is the last second non-zero element included in the first row and the first column of the first kernel data.
  • the shift direction is determined according to the row and column where the element is located in the first kernel data in the next cycle.
  • the second non-zero element of the first depth included in the first row and the second column of the first kernel data will be input, and it is to the right side from the first row and the first column of the first kernel data. That is, the shift direction may be to the right side. If, in the third cycle, the second non-zero elements of the first depth included in the second row and the first column are to be input, this is a lower side from the first row and the first column of the first kernel data, and the shift direction may be to a lower side.
  • Each of the plurality of second processing elements can perform an inter-element operation between the first non-zero element and the second non-zero element inputted by the same operation method as the operation of the plurality of first processing elements in the previous cycle.
  • the processor 120 may input the second non-zero element to the plurality of first processing elements in the third cycle.
  • the input second non-zero element is the second non-zero element of the first depth included in the first row and the second column of the first kernel data.
  • Each of the plurality of first processing elements can shift the second non-zero element that is input in the second cycle into the adjacent second processing element.
  • each of the plurality of second processing elements can shift the second non-zero element that is input in the second cycle to a processing element (not shown) adjacent to the lower side which is input in the second cycle.
  • the plurality of first processing elements and the plurality of second processing elements can be shifted in the previous cycle when the cycle is changed, and the element can be shifted to the lower processing element with the inputted second non-zero element. Because the same operation is repeated, description of the shift of the second non-zero element will be omitted.
  • Each of the plurality of first processing elements may perform an inter-element operation on the input first non-zero element and the input second non-zero element.
  • Each of the plurality of first processing elements can add the operation result shifted from the second cycle to the operation result of the third cycle and store the summed operation result.
  • Each of the plurality of second processing elements may perform an inter-element operation between the input first non-zero element and the input second non-zero element that is input in the same operation method as the operation of the plurality of first processing element in the previous cycle and shift the operation result to a right side.
  • each of the plurality of second processing elements can be operating in the same manner as the operation of the plurality of first processing elements in the previous cycle.
  • the operations of the plurality of second processing elements are the same as those of the plurality of first processing elements in the previous cycle.
  • FIGs. 5D, 5E, and 5F illustrates an operation according to the input of the second non-zero element of the second depth, third depth, and fourth depth included in the first row and the second column of the first kernel data.
  • the operation is the same as the above and thus, detailed description is omitted.
  • the processor 120 may input the second non-zero element to a plurality of first processing elements in the seventh cycle.
  • the input second non-zero element is the second non-zero element of the fifth depth included in the first row and the second column of the first kernel data.
  • Each of the plurality of first processing elements may perform an inter-element operation on the input first non-zero element and the input second non-zero element.
  • Each of the plurality of first processing elements can add the operation result of the seventh cycle to the operation result of the sixth cycle and shift it to the adjacent second processing element.
  • the second non-zero element of the first depth included in the second row and the second column of the first kernel data will be input, which corresponds to a lower side of the first row and the second column of the first kernel data, and a shift direction may be downward.
  • Each of the plurality of second processing elements may perform an inter-element operation between the input first non-zero element and the input second non-zero element.
  • Each of the plurality of second processing elements may store the operation result shifted from the adjacent first processing element separately from the operation result in the seventh cycle. That is, the operation result shifted from the processing element adjacent to the upper side in the downward direction is not added to the operation result of the current cycle.
  • the processor 120 may input the second non-zero element to the plurality of first processing elements in the eighth cycle.
  • the input second non-zero element is the second non-zero element of the first depth included in the second row and the second column of the first kernel data.
  • Each of the plurality of first processing elements may perform an inter-element operation between the input first non-zero element and the input second non-zero element.
  • Each of the plurality of second processing elements may perform an inter-element operation on the input first non-zero element and the input second non-zero element.
  • Each of the plurality of second processing elements may add the operation result in the seventh cycle and the operation result in the eighth cycle, and shift the summed operation result to the processing element adjacent to the lower side. However, the operation result shifted from the processing element adjacent to the upper side in the seventh cycle may be stored as it is in each of the plurality of second processing elements.
  • the processor 120 can input the second non-zero element to the plurality of first processing elements in the ninth cycle.
  • the input second non-zero element is a second non-zero element of the second depth included in the second row and the second column of the first kernel data.
  • Each of the plurality of first processing elements performs an inter-element operation between the input first non-zero element and the input second non-zero element, and by adding the operation result of the previous cycle and the operation result of the present cycle, stores the added operation result.
  • Each of the plurality of second processing elements performs an inter-element operation between the input first non-zero element and the input second non-zero element, adds the operation result shifted from the processing element adjacent to the upper side in the seventh cycle and the operation result of the present cycle, and stores the added operation result.
  • FIGs. 5J and 5K illustrate operations according to the input of the second non-zero element of the third depth and the fifth depth included in the second row and the second column of the first kernel data.
  • the operation method, the adding method, and the shifting method are the same, and as such, a detailed description is omitted.
  • the result of the added operation can be shifted to the left side. That is, the shift direction of the added result of FIG. 5K may be opposite to the shift direction of the added result of FIG. 5B.
  • the processor 120 may input zero to the plurality of first processing elements in the 12th cycle. Because there is no second non-zero element in the second row and the first column of the first kernel data, the processor 120 can input zero to the plurality of first processing elements.
  • the shift is unnecessary.
  • the second non-zero element to be input in the next cycle is a second non-zero element of the same first kernel data, the shift is performed.
  • the processor 120 inputs zero to the plurality of first processing elements, and the operation result stored in each of the plurality of first processing elements can be shifted to the adjacent processing elements.
  • the processor 120 may input the second non-zero element to the plurality of first processing elements in the 13th cycle.
  • the input second non-zero element is the second non-zero element of the second depth included in the first row and the first column of the second kernel data.
  • the operations of the plurality of first processing elements and the plurality of second processing elements are the same as those described above.
  • continued convolution operation can be performed on a plurality of kernel data.
  • the processor 120 may output the operation result for the first kernel data.
  • FIGs. 5A to 5M illustrate a plurality of processing elements in the form of a 4 ⁇ 4 matrix, the present disclosure is not limited thereto, and the number of processing elements may vary.
  • the target data has been described in the form of 4x4x5, it is not limited thereto, and it may be any other form.
  • the processor 120 may divide the target data into four, based on the row and column of the target data, and the convolution operation may be performed.
  • FIGs. 6A and 6B illustrate a method of processing data sparsity of kernel data according to an embodiment.
  • the processor 120 may omit input of the second non-zero element corresponding to the depth from among the second element and sequentially input the second non-zero element not corresponding to the depth to each of the plurality of first processing elements.
  • the processor 120 may identify that there is no first non-zero element corresponding to the second depth from among the first non-zero elements stored in each of the plurality of processing elements. In this case, the processor 120 may remove the second non-zero element of the second depth included in the first kernel data and the second kernel data, and sequentially input the remaining second non-zero element to a plurality of the first processing elements.
  • the processor 120 may remove the second non-zero element of the second depth included in the first kernel data and the second kernel data, separately store the remaining second non-zero element in the storage 110, and sequentially extract the remaining second non-zero element to input to the plurality of first processing elements. Alternatively, the processor 120 may sequentially extract the second non-zero element from the first kernel data and the second kernel data, and when the second non-zero element of the second depth is identified, this will be skipped, and the second non-zero element, which is not the second depth, may be extracted and input to the plurality of first processing elements.
  • the processor 120 may identify a depth with no first non-zero element in all rows and all columns before the first non-zero element is input into each of the plurality of processing elements.
  • FIGs. 7A and 7B illustrate a method for processing data sparsity of target data according to an embodiment.
  • the processor 120 may omit input of the second non-zero element corresponding to the identified depth from among the second element and sequentially input the second non-zero element not corresponding to the depth to each of the plurality of the first processing elements.
  • the processor 120 may, when the second depth which has the first non-zero element which is less than three in all of the rows and columns, from among the first non-zero elements stored in each of the plurality of processing elements, is identified, input of the second non-zero element 720 corresponding to the second depth from among the second elements is omitted, and the second non-zero element that does not correspond to the second depth may be sequentially input to each of the plurality of the first processing elements.
  • the first non-zero element of the identified depth may be stored in a part of the plurality of processing elements, but unless the second non-zero element 720 of the identified depth is input, an operation is not performed, and thus, cycle can be shortened.
  • the shortened cycle is the same as illustrated in FIGs. 6A and 6B.
  • the processor 120 may further include a plurality of preliminary processing elements, and the first non-zero element that corresponds to the identified depth and the second non-zero element that corresponds to the identified depth may be input to a plurality of preliminary processing elements to perform a separate operation.
  • the processor 120 may further include a plurality of pre-processing elements 730, and may input the first non-zero element 710 corresponding to the identified depth and the second non-zero element 720 corresponding to the identified depth to the plurality of the pre-processing elements 730 to perform a separate operation.
  • the processor 120 may perform operations illustrated in FIGs. 5A to 5M using a plurality of processing elements, and operate the first non-zero element 710 corresponding to the identified depth and the second non-zero element 720 corresponding to the identified depth using a plurality of the pre-processing elements 730 in parallel.
  • the processor 120 may add the operation results output from the plurality of pre-processing elements 730 to the corresponding operation results from among the operation results output from the plurality of processing elements.
  • FIG. 8 illustrates a processing element according to an embodiment.
  • a processing element includes a Kernel terminal 811, an FMap terminal 812, a PSum terminal 813, a BottomAcc terminal 814, a LeftAcc terminal 821, a RightAcc terminal 822, a Ctrl_Inst terminal 823, a LeftAcc terminal 831, a RightAcc terminal 832, a Kernel terminal 841, a PSum terminal 842, a BottomAcc terminal 843, a register file 850, a multiplier 860, a multiplexer 870, and an adder 880.
  • the processing element may receive the second non-zero element, the first non-zero element, and data and an instruction stored in the storage 110 through each of the kernel terminal 811, the Fmap terminal 812, the Psum terminal 813, and the Ctrl_Inst terminal 823.
  • the processing element can shift the second non-zero element to the processing element adjacent to the lower part via the Kernel terminal 841.
  • the processing element can receive or output data directly to the storage 110 using the PSum terminal 813 and the PSum terminal 842.
  • the processing element can receive the operation result from the adjacent processing element through the BottomAcc terminal 814, the RightAcc terminal 822, and the LeftAcc terminal 831. Further, the processing element can shift the operation result directly processed to the adjacent processing element through the LeftAcc terminal 821, the RightAcc terminal 832, and the BottomAcc terminal 843.
  • the register file 850 may store the first non-zero element and the operation result input through the FMap terminal 812.
  • the multiplier 860 may perform a multiplication operation of the second non-zero element input through the Kernel terminal 811 and the first non-zero element input from the Register File 850.
  • the multiplexer 870 may provide one of the operation result that is input from an adjacent processing element, the operation result processed in a processing element, data input from the PSum terminal 813, and data input from the register file 850 to the adder 8810.
  • the Adder 880 can perform addition operations of the multiplication result input from the multiplier 860 and the data input from the multiplexer 870.
  • a processing element may further include a multiplexer.
  • FIG. 9 is a flowchart illustrating a method of controlling an electronic apparatus according to an embodiment.
  • the electronic apparatus may include a processor that performs deep learning, a storage that stores target data and kernel data, and a plurality of processing elements arranged in a matrix form.
  • the first non-zero element among a plurality of the first elements included in the target data is input to each of the plurality of processing elements in step S910.
  • step S920 the second non-zero element from among the plurality of second elements included in the kernel data is sequentially input to each of the plurality of first processing elements included in the first row of the plurality of processing elements.
  • step S930 Based on the input depth information of the first non-zero element and the input depth information of the second non-zero element input from each of the plurality of first processing elements, the operation between the input first non-zero element and the input second non-zero element is performed in step S930.
  • Each of the plurality of processing elements includes a plurality of register files
  • inputting the first non-zero element in step S910 may include identifying a corresponding processing element from among a plurality of processing elements based on the row information and the column information of the first non-zero element and inputting the first non-zero element to a corresponding register file from among a plurality of register files included in the identified processing element.
  • the step S920 of sequentially inputting the second non-zero element may include sequentially inputting the second non-zero elements to the plurality of first processing elements, based on the row information, the column information, and the depth information of the second non-zero element.
  • the step S920 of sequentially inputting the second non-zero element may include sequentially inputting the second non-zero element included in one row and one column from among the second non-zero elements to each of the plurality of the first processing elements based on the depth and, if all the second non-zero element included in one row and one column is input to each of the plurality of processing elements, inputting the second non-zero element included in a row and a column different from the one row and the one column to each of the plurality of the first processing elements.
  • the step S920 of sequentially inputting the second non-zero element includes, when there is no second non-zero element in one row and one column, inputting zero to each of the plurality of the first processing elements, and if zero is input to each of the plurality of processing elements, inputting the second non-zero element included in another row and column or zero to each of the plurality of the first processing elements based on the number of the second non-zero element included in another row and column.
  • the step S920 of sequentially inputting the second non-zero element may include, when a depth which has no first non-zero element in all the rows and columns is identified from among the first non-zero elements stored in each of the plurality of processing elements, omitting input of the second non-zero element corresponding to the depth from among the second elements and sequentially inputting the second non-zero element not corresponding to the depth to each of the first plurality of first processing elements.
  • the step S920 of sequentially inputting the second non-zero element includes, when the depth in which the first non-zero element is within a predetermined number in all the rows and columns is identified from among the first non-zero element stored in each of the plurality of processing elements, omitting input of the second non-zero element corresponding to the depth from among the second elements, sequentially inputting the second non-zero element not corresponding to the depth to each of the plurality of the first processing elements, and inputting the first non-zero element corresponding to the depth and the second non-zero element corresponding to the depth to a plurality of preliminary processing elements included in the process.
  • the input second non-zero element may be shifted to each of the plurality of second processing elements included in the second row. If an operation between the non-zero elements is completed in the plurality of the second processing elements, the shifted second non-zero element may be shifted from the plurality of second processing elements to each of the plurality of third processing elements included in the third row.
  • the input second non-zero element may be accumulated with the previous operation result, and the result thereof may be stored to one of the plurality of register files.
  • the operation result stored in one of the plurality of register files of each of the plurality of processing elements may be shifted to an adjacent processing element, and the input second non-zero element may be accumulated with the shifted operation result and then stored in one of the plurality of register files.
  • an electronic apparatus can improve the speed of a convolution operation by omitting calculations of a part of target data and a part of kernel data according to a zero included in the target data.
  • the target data and the kernel data described above may be in any form of three-dimensional data. Also, the number of the plurality of processing elements included in the processor may be different as well.
  • the various embodiments described above may be implemented with software that includes instructions stored on a machine-readable storage medium which can be read by a machine (e.g., a computer).
  • the device calls an instruction stored from a storage medium and is operable according to a called instruction, and may include an electronic apparatus (e.g.: an electronic apparatus).
  • an instruction is executed by a processor, the processor may perform functions corresponding to the instruction, either directly or under the control of the processor, using other components.
  • the instruction may include code generated or executed by a compiler or an interpreter.
  • a machine-readable storage medium may be provided in the form of a non-transitory storage medium.
  • a method according to various embodiments described above may be provided in a computer program product.
  • a computer program product may be traded between a seller and a purchaser as a commodity.
  • the computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)) or distributed online through an application store (e.g., PlayStoreTM).
  • an application store e.g., PlayStoreTM
  • at least a portion of the computer program product may be stored temporarily or at least provisionally in a storage medium, such as a manufacturer’s server, a server of an application store, or a memory of a relay server.
  • the various embodiments described above may be implemented within a computer readable medium, such as a computer or a similar device, using software, hardware, or combination thereof.
  • the embodiments described herein may be implemented by the processor itself.
  • embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more of the functions and operations described herein.
  • Non-transitory computer-readable media is a medium that stores data for a short period of time, such as a register, cache, memory, etc., but semi-permanently stores data and is readable by the device.
  • Specific examples of non-transitory computer readable media include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.
  • each of the components may include one or a plurality of entities, and some subcomponents of the subcomponents described above may be omitted.
  • the components may be further included in various embodiments.
  • some components e.g., modules or programs

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Neurology (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un appareil électronique et un procédé associé pour réaliser un apprentissage en profondeur. L'appareil électronique comprend une mémoire configurée pour mémoriser des données cibles et des données de noyau; et un processeur comprenant une pluralité d'éléments de traitement qui sont agencés selon une forme de matrice. Le processeur est configuré pour entrer, pour chaque élément parmi la pluralité d'éléments de traitement, un premier élément non nul parmi une pluralité de premiers éléments inclus dans les données cibles et entrer séquentiellement, pour chaque élément parmi une pluralité de premiers éléments de traitement inclus dans une première rangée parmi la pluralité d'éléments de traitement, un second élément non nul parmi la pluralité d'éléments inclus dans les données de noyau. Chaque élément parmi la pluralité de premiers éléments de traitement est configuré pour effectuer une opération entre le premier élément non nul d'entrée et le second élément non nul d'entrée, sur la base d'informations de profondeur du premier élément non nul et d'informations de profondeur du second élément non nul.
EP18866233.2A 2017-10-12 2018-06-08 Appareil électronique et procédé de commande associé Pending EP3659073A4 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201762571599P 2017-10-12 2017-10-12
KR1020180022960A KR20190041388A (ko) 2017-10-12 2018-02-26 전자 장치 및 그 제어 방법
PCT/KR2018/006509 WO2019074185A1 (fr) 2017-10-12 2018-06-08 Appareil électronique et procédé de commande associé

Publications (2)

Publication Number Publication Date
EP3659073A1 true EP3659073A1 (fr) 2020-06-03
EP3659073A4 EP3659073A4 (fr) 2020-09-30

Family

ID=66282988

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18866233.2A Pending EP3659073A4 (fr) 2017-10-12 2018-06-08 Appareil électronique et procédé de commande associé

Country Status (3)

Country Link
EP (1) EP3659073A4 (fr)
KR (1) KR20190041388A (fr)
CN (1) CN111095304A (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3714406A4 (fr) * 2018-03-07 2021-02-17 Samsung Electronics Co., Ltd. Appareil électronique et son procédé de commande

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20210111014A (ko) * 2020-03-02 2021-09-10 삼성전자주식회사 전자 장치 및 그 제어 방법
KR102565826B1 (ko) * 2020-12-29 2023-08-16 한양대학교 산학협력단 데이터 재사용을 통해 컨볼루션 연산 속도를 향상시킨 3차원 객체 인식 방법 및 장치

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1923793A2 (fr) * 2006-10-03 2008-05-21 Sparsix Corporation Contrôleur de mémoire pour système de calcul de données rares et procédés correspondant
EP2657842B1 (fr) * 2012-04-23 2017-11-08 Fujitsu Limited Optimisation de la charge de travail dans un système multiprocesseur exécutant une multiplication de vecteur de matrice rare
JP6083300B2 (ja) * 2013-03-29 2017-02-22 富士通株式会社 プログラム、並列演算方法および情報処理装置
CN104915322B (zh) * 2015-06-09 2018-05-01 中国人民解放军国防科学技术大学 一种卷积神经网络硬件加速方法
KR101843243B1 (ko) * 2015-10-30 2018-03-29 세종대학교산학협력단 제로값을 피연산자로 갖는 연산자에 대한 연산을 스킵하는 연산 방법 및 연산 장치
CN107239824A (zh) * 2016-12-05 2017-10-10 北京深鉴智能科技有限公司 用于实现稀疏卷积神经网络加速器的装置和方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3714406A4 (fr) * 2018-03-07 2021-02-17 Samsung Electronics Co., Ltd. Appareil électronique et son procédé de commande

Also Published As

Publication number Publication date
KR20190041388A (ko) 2019-04-22
CN111095304A (zh) 2020-05-01
EP3659073A4 (fr) 2020-09-30

Similar Documents

Publication Publication Date Title
WO2020159232A1 (fr) Procédé, appareil, dispositif électronique et support d'informations lisible par ordinateur permettant de rechercher une image
WO2020138745A1 (fr) Procédé de traitement d'image, appareil, dispositif électronique et support d'informations lisible par ordinateur
AU2018319215B2 (en) Electronic apparatus and control method thereof
WO2018164378A1 (fr) Appareil électronique permettant de compresser un modèle linguistique, appareil électronique permettant de fournir un mot de recommandation et procédés de fonctionnement associés
WO2019164251A1 (fr) Procédé de réalisation d'apprentissage d'un réseau neuronal profond et appareil associé
WO2019074185A1 (fr) Appareil électronique et procédé de commande associé
EP3659073A1 (fr) Appareil électronique et procédé de commande associé
WO2020180084A1 (fr) Procédé permettant d'achever la coloration d'une image cible, et dispositif et programme informatique associés
WO2019135631A1 (fr) Dispositif électronique permettant d'obscurcir et de décoder des données et procédé permettant de commander ce dernier
EP3577571A1 (fr) Appareil électronique permettant de compresser un modèle linguistique, appareil électronique permettant de fournir un mot de recommandation et procédés de fonctionnement associés
WO2019231095A1 (fr) Appareil électronique et procédé de commande associé
WO2020045794A1 (fr) Dispositif électronique et procédé de commande associé
EP4367628A1 (fr) Procédé de traitement d'image et dispositif associé
WO2020141720A1 (fr) Appareil et procédé de gestion de programme d'application
WO2021194089A1 (fr) Procédé pour changer l'interface utilisateur graphique d'un bloc de circuit, et support de stockage lisible par ordinateur doté d'un programme enregistré comprenant des instructions pour effectuer chaque étape d'un procédé destiné à changer l'interface utilisateur graphique d'un bloc de circuit
WO2021125496A1 (fr) Dispositif électronique et son procédé de commande
EP3746952A1 (fr) Appareil électronique et procédé de commande associé
WO2018191889A1 (fr) Procédé et appareil de traitement de photo, et dispositif informatique
WO2019198900A1 (fr) Appareil électronique et procédé de commande associé
WO2022244997A1 (fr) Procédé et appareil pour le traitement de données
WO2022139479A1 (fr) Procédé et dispositif de prédiction d'événement ultérieur devant se produire
WO2023286914A1 (fr) Procédé de construction d'un modèle de transformateur pour répondre à une question d'histoire vidéo, et dispositif informatique pour mettre en œuvre ledit procédé
WO2021158040A1 (fr) Dispositif électronique fournissant un énoncé correspondant au contexte d'une conversation, et procédé d'utilisation associé
WO2020213885A1 (fr) Serveur et son procédé de commande
WO2024128579A1 (fr) Dispositif de virtualisation de clavier et son procédé de fonctionnement

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200224

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

A4 Supplementary search report drawn up and despatched

Effective date: 20200827

RIC1 Information provided on ipc code assigned before grant

Ipc: G06N 3/08 20060101ALI20200821BHEP

Ipc: G06N 3/04 20060101ALI20200821BHEP

Ipc: G06N 3/063 20060101AFI20200821BHEP

Ipc: G06F 17/15 20060101ALI20200821BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220704