CN113269211A - Method and system for integrating in-sensor processing unit and in-memory arithmetic unit - Google Patents

Method and system for integrating in-sensor processing unit and in-memory arithmetic unit Download PDF

Info

Publication number
CN113269211A
CN113269211A CN202010093130.XA CN202010093130A CN113269211A CN 113269211 A CN113269211 A CN 113269211A CN 202010093130 A CN202010093130 A CN 202010093130A CN 113269211 A CN113269211 A CN 113269211A
Authority
CN
China
Prior art keywords
data
memory
module
difference
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010093130.XA
Other languages
Chinese (zh)
Inventor
郑桂忠
张孟凡
谢志成
谢轩颢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Egis Technology Inc
Original Assignee
郑桂忠
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 郑桂忠 filed Critical 郑桂忠
Priority to CN202010093130.XA priority Critical patent/CN113269211A/en
Publication of CN113269211A publication Critical patent/CN113269211A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/60Memory management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/28Indexing scheme for image data processing or generation, in general involving image processing hardware

Abstract

The invention relates to a method and a system for integrating an in-sensor processing unit and an in-memory operation unit. The method for integrating the in-sensor processing unit and the in-memory operation unit comprises the following steps of driving the in-sensor processing unit to provide a first instruction signal and a plurality of initial data and transmitting the first instruction signal and the plurality of initial data to the bus unit. The conversion step drives the first command signal and the initial data to be converted into a second command signal and a plurality of input data through the synchronization module. The obtaining step drives the picture difference module to receive input data to obtain a plurality of difference data. The dividing step drives the bit dividing module to receive the difference data and divide each difference data into a plurality of bit slices. The control step drives the coding module to receive the differential address and codes the differential address into a control signal, and the operation unit in the memory accesses each bit slice according to the control signal. Therefore, the energy consumption and the time of operation are reduced.

Description

Method and system for integrating in-sensor processing unit and in-memory arithmetic unit
Technical Field
The present invention relates to a method and system for integrating an in-sensor processing unit and an in-memory computing unit, and more particularly, to a method and system for integrating an in-sensor processing unit and an in-memory computing unit for image recognition.
Background
The image recognition technology applied by the general deep neural network has two keys: In-Sensor (PIS) and In-Memory Computing (CIM). The PIS is an image acquisition device (e.g., a digital camera) that performs preliminary preprocessing on recorded raw image data to achieve edge acquisition, quantization, or calculation of a low-level neural network. CIM is the operation of convolutional layer or even fully-connected layer in deep neural network architecture, and it has the functions of memory and accelerator. An image recognition system needs to have two characteristics to realize complete and efficient image recognition application.
Conventionally, based on the Von Neumann (Von Neumann) model, if the Pixel Array (Pixel Array) or the in-memory data needs to be processed, the data needs to be transferred to an arithmetic unit (e.g. a processor) for calculation. However, the calculation and the data transfer between the units are performed independently and separately through various carriers, wherein the large amount of data transfer process easily causes system inefficiency and energy consumption, and at present, there is no system for integrating the two.
Disclosure of Invention
Therefore, the present invention is directed to a method and system for integrating an in-sensor processing unit and an in-memory operation unit, which integrates and transmits a large amount of data between the in-sensor processing unit and the in-memory operation unit through a bus, thereby reducing the power consumption and time of the convolutional neural network operation.
According to an embodiment of the present invention, an integration method of an in-sensor processing unit and an in-memory computing unit is provided, the in-sensor processing unit operates at a first clock, the in-memory computing unit operates at a second clock, and the integration method of the in-sensor processing unit and the in-memory computing unit includes a providing step, a converting step, an acquiring step, a dividing step, and a controlling step. The providing step drives the sensor internal processing unit to provide a first instruction signal and a plurality of initial data and transmit the first instruction signal and the plurality of initial data to a bus unit, wherein the first instruction signal and each initial data are operated in a first clock, and the bus unit comprises a synchronization module and a picture difference module. The conversion step drives the synchronization module to receive the first instruction signal and the initial data, the first instruction signal and the initial data are converted into a second instruction signal and a plurality of input data through the synchronization module, the second instruction signal and each input data are operated in a second clock, the first clock is different from the second clock, and therefore the second instruction signal is transmitted to the operation unit in the memory. The obtaining step drives the picture difference module to receive the input data and obtain a plurality of difference data according to the input data, wherein each difference data has a difference address. The dividing step drives a bit dividing module to receive the difference data and divide each difference data into a plurality of bit slices. The control step drives an encoding module to receive the differential address and encode the differential address into a control signal, and the operation unit in the memory accesses each bit slice according to the control signal.
Therefore, the method for integrating the in-sensor processing unit and the in-memory operation unit according to the present invention can transmit the data output from the in-sensor processing unit to the in-memory operation unit for operation through the providing step, the converting step, the obtaining step, the dividing step and the controlling step.
The method for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the previous paragraphs, wherein in the providing step, when the first command signal is 1, the input data are sequentially transmitted to the bus unit. When the first command signal is 0, the input data is not transmitted to the bus unit.
The method for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the preceding paragraphs, wherein the bus unit further comprises a memory, and the acquiring step comprises a data storing sub-step and a data acquiring sub-step. The data storage substep drives the memory to store an input data. A data acquisition sub-step drives the picture difference module to read an input data and compare with another input data to acquire each difference data.
The method for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the previous paragraphs further comprises an output step. The output step drives a transmission module to receive the bit slice and converts the bit slice into output data, so that the output data is transmitted to the operation unit in the memory. Each bit slice has a first bandwidth, the output data has a second bandwidth, and the first bandwidth is different from the second bandwidth.
The method for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the previous paragraphs, wherein the in-memory computing unit includes an sram, and in the step of controlling, when the control signal is 1, each bit slice is sequentially written into the sram. When the control signal is 0, each bit slice is not written into the SRAM.
According to an embodiment of the present invention, an integrated system of an in-sensor processing unit and an in-memory operation unit includes an in-sensor processing unit, a bus unit and an in-memory operation unit. The sensor processing unit comprises a microprocessor and a sensor, wherein the microprocessor is electrically connected with the sensor and is used for generating a first instruction signal and a plurality of initial data. The bus unit is electrically connected with the sensor internal processing unit and comprises a synchronization module, a picture difference module, a bit segmentation module and an encoding module. The synchronization module is electrically connected with the microprocessor and receives the first instruction signal and the initial data to generate a second instruction signal and a plurality of input data. The picture difference module is electrically connected to the synchronization module, receives the input data and generates a plurality of difference data, and each difference data has a difference address. The bit dividing module is electrically connected to the picture difference module and receives the difference data to generate a plurality of bit pieces. The coding module is electrically connected to the picture difference module and receives the difference address to generate a control signal. The memory internal operation unit is electrically connected with the bus unit and accesses each bit slice according to the control signal.
Therefore, the integrated system of the in-sensor processing unit and the in-memory operation unit can transmit the data output by the in-sensor processing unit to the in-memory operation unit for operation through the bus unit.
The system of integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the previous paragraphs, wherein the in-sensor processing unit operates on a first clock. The in-memory operation unit operates at a second clock. The first clock is converted into a second clock through the synchronization module. Wherein the first clock is different from the second clock.
The system of integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the preceding paragraphs, wherein the bus unit further includes a memory. The memory is electrically connected to the picture difference module. The memory stores input data, and the picture difference module reads the input data of the memory and compares the input data with another input data to obtain different data.
The system for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments described in the previous paragraphs, wherein the in-memory computing unit includes a processing unit and a static random access memory. The processing unit is electrically connected to the synchronization module and the encoding module and receives the second instruction signal and the control signal. The static random access memory is electrically connected with the processing unit and accesses each bit sheet according to the control signal.
The system for integrating an in-sensor processing unit and an in-memory computing unit according to the embodiments of the present invention as described in the preceding paragraphs, wherein the bus unit further includes a transmission module. The transmission module is electrically connected with the bit dividing module and the static random access memory, receives the bit slice, converts the bit slice into output data, and transmits the output data to the static random access memory. Each bit slice has a first bandwidth, the output data has a second bandwidth, and the first bandwidth is different from the second bandwidth.
Drawings
FIG. 1 is a block diagram illustrating an integrated system of an in-sensor processing unit and an in-memory arithmetic unit in accordance with one embodiment of an aspect of the present invention;
FIG. 2 is a block diagram illustrating an integrated system of an in-sensor processing unit and an in-memory computing unit in accordance with another aspect of the present invention;
FIG. 3 is a diagram illustrating the conversion of a first command signal and initial data into a second command signal and input data for the integrated system of the in-sensor processing unit and the in-memory computing unit according to the embodiment of the aspect of FIG. 2;
FIG. 4 is a schematic diagram illustrating a differential data partitioning into bit-slices for an integrated system of an in-sensor processing unit and an in-memory computing unit in accordance with the embodiment of the aspect of FIG. 2;
FIG. 5 is a block diagram illustrating steps in a method for integrating an in-sensor processing unit and an in-memory arithmetic unit in accordance with one embodiment of a method aspect of the present invention; and
FIG. 6 is a block diagram illustrating the steps of an acquisition step of an integrated method of an in-sensor processing unit and an in-memory arithmetic unit in accordance with an embodiment of the method aspect of FIG. 5.
Wherein the reference numerals are as follows:
s100: method for integrating in-sensor processing unit and in-memory operation unit
S110: providing step
S120: step of conversion
S130: obtaining step
S140: step of dividing
S150: control step
S160: output step
S131: data storage substep
S132: data acquisition substep
100: integrated system of in-sensor processing unit and in-memory operation unit
110: in-sensor processing unit
111: microprocessor
1111: a first command signal
112: sensor device
1121: initial data
120: in-memory arithmetic unit
121: processing unit
122: static random access memory
1221: static random access memory address
130: bus unit
131: synchronization module
1311: second command signal
1312: inputting data
132: picture difference module
1321: difference data
1322: differential address
133: bit division module
1331: bit sheet
134: coding module
1341: control signal
135: memory device
136: transmission module
1361: outputting the data
clk 1: first clock
clk 2: second clock
Detailed Description
Various embodiments of the present invention will be described below with reference to the accompanying drawings. For the purpose of clarity, numerous implementation details are set forth in the following description. It should be understood, however, that these implementation details are not to be interpreted as limiting the invention. That is, in some embodiments of the invention, these implementation details are not necessary. In addition, some conventional structures and elements are shown in simplified schematic form in the drawings for the sake of simplifying the drawings; and repeated elements will likely be referred to using the same reference numerals.
In addition, when an element (or a mechanism or a module, etc.) is "connected," "disposed" or "coupled" to another element, it can be directly connected, disposed or coupled to the other element, or it can be indirectly connected, disposed or coupled to the other element, that is, there are other elements between the element and the other element. When an element is explicitly connected, directly disposed, or directly coupled to another element, it is intended that no other element is interposed between the element and the other element. The terms first, second, third, etc. are used merely to describe various elements or components, but the elements/components themselves are not limited, so that the first element/component can be also referred to as the second element/component. And the combination of elements/components/mechanisms/modules herein is not a commonly known, conventional or existing combination in the art, and cannot be readily determined by one of ordinary skill in the art based on whether the elements/components/mechanisms/modules themselves are present.
Referring to fig. 1 and 2 together, fig. 1 is a block diagram illustrating an integrated system 100 of an in-sensor processing unit and an in-memory computing unit according to an embodiment of an aspect of the present invention. FIG. 2 is a block diagram of an integrated system 100 of in-sensor processing units and in-memory arithmetic units according to another embodiment of the present invention. As shown in fig. 1 and 2, the system 100 integrating the in-sensor processing unit and the in-memory computing unit includes an in-sensor processing unit 110, an in-memory computing unit 120, and a bus unit 130. The in-sensor processing unit 110 includes a microprocessor 111 and a sensor 112, wherein the microprocessor 111 is electrically connected to the sensor 112 for generating a first instruction signal 1111 and a plurality of initial data 1121.
The bus unit 130 is electrically connected to the in-sensor processing unit 110, and includes a synchronization module 131, a frame difference module 132, a bit segmentation module 133, and an encoding module 134. The synchronization module 131 is electrically connected to the microprocessor 111 and receives the first command signal 1111 and the initial data 1121 to generate a second command signal 1311 and a plurality of input data 1312. The frame difference module 132 is electrically connected to the synchronization module 131, and the frame difference module 132 receives the input data 1312 to generate a plurality of difference data 1321, and each difference data 1321 has a difference address 1322. The bit dividing module 133 is electrically connected to the frame difference module 132, and the bit dividing module 133 receives the difference data 1321 to generate a plurality of bit slices 1331. The encoding module 134 is electrically connected to the frame difference module 132, and the encoding module 134 receives the difference address 1322 to generate a control signal 1341. The in-memory unit 120 is electrically connected to the bus unit 130, and the in-memory unit 120 accesses each bit slice 1331 according to the control signal 1341.
Therefore, the system 100 for integrating the in-sensor processing unit and the in-memory computing unit of the present invention can transmit the data output by the in-sensor processing unit 110 to the in-memory computing unit 120 through the bus unit 130 for computing, which is beneficial to reducing the energy consumption and time of the data during the transmission process.
Referring to fig. 2 and 3 together, fig. 3 is a schematic diagram illustrating a conversion of a first instruction signal 1111 and initial data 1121 of the integrated system 100 of the in-sensor processing unit and the in-memory operation unit according to the embodiment of the aspect of fig. 2 into a second instruction signal 1311 and input data 1312. As can be seen from FIG. 3, when the first command signal 1111 rises to 1, it indicates that the in-sensor processing unit 110 is ready and can transmit the pre-processed data to the bus unit 130. The initial data 1121 will pass through a data port (not shown) of the in-sensor processing unit 110 and be output sequentially. The in-sensor processing unit 110 may operate at a first clock clk1 (e.g., 10KHz), and the in-memory operation unit 120 may operate at a second clock clk2 (e.g., 100 KHz). The first clock clk1 is different from the second clock clk 2. Accordingly, the first instruction signal 1111 and the first clock clk1 of the initial data 1121 are converted into the second instruction signal 1311 and the second clock clk2 of the input data 1312 through the synchronization module 131. Thereby, the data is synchronized and transmitted from the in-sensor processing unit 110 to the in-memory operation unit 120.
It is noted that the bus unit 130 may further include a memory 135. The memory 135 is electrically connected to the frame difference module 132. The memory 135 stores an input data 1312, and the frame difference module 132 reads the input data 1312 of the memory 135 and compares the read input data 1312 with another input data 1312 to obtain difference data 1321. In detail, the frame difference module 132 receives each input data 1312 sequentially, wherein one input data 1312 is stored in the memory 135, and compares the next input data 1312 (i.e. the other input data 1312) with the difference of the input data 1312 of the memory 135 to obtain each difference data 1321 (e.g. "012345678", "12" in fig. 3). Therefore, the transmission of redundant data volume is reduced, and the transmission efficiency is improved.
Referring to fig. 2 and 4 together, fig. 4 is a schematic diagram illustrating a division of difference data 1321 of the integrated system 100 of in-sensor processing unit and in-memory operation unit into bit slices 1331 according to the embodiment of the aspect of fig. 2. The bit dividing module 133 receives the plurality of difference data 1321 and divides each difference data 1321 into a plurality of bit slices 1331. The encoding module 134 receives the plurality of differential addresses 1322 and encodes them into control signals 1341, and the in-memory unit 120 accesses each bit slice 1331 according to the control signals 1341. In addition, the in-memory operation unit 120 includes a processing unit 121 and a static random access memory 122. The processing unit 121 is electrically connected to the synchronization module 131 and the encoding module 134 and receives the second command signal 1311 and the control signal 1341. The sram 122 is electrically connected to the processing unit 121, and the sram 122 accesses each bit slice 1331 according to the control signal 1341. Specifically, each bit of each data in the in-memory unit 120 is separately stored in a different address, so that the data needs to be cut into single-bit serial (bitserial) inputs and written into a corresponding sram address 1221. When control signal 1341 rises to 1, it represents that bit slice 1331 begins to be written into SRAM 122 according to SRAM address 1221.
Furthermore, the bus unit 130 may further include a transmission module 136. The transmission module 136 is electrically connected to the bit dividing module 133 and the sram 122, and the transmission module 136 receives the bit slices 1331 and converts the bit slices 1331 into output data 1361, so that the output data 1361 is transmitted to the sram 122. In detail, each bit 1331 has a first bandwidth (e.g., 1' b1), the output data 1361 has a second bandwidth (e.g., 16bits/cycle), and the first bandwidth is different from the second bandwidth. Thus, the output data 1361 can satisfy the input bandwidth of the in-memory arithmetic unit 120.
FIG. 5 is a block diagram illustrating a method S100 for integrating an in-sensor processing unit and an in-memory computing unit according to one embodiment of a method aspect of the invention. In fig. 5, the method S100 for integrating the in-sensor processing unit and the in-memory operation unit includes a providing step S110, a converting step S120, an acquiring step S130, a dividing step S140, and a controlling step S150.
Referring to FIG. 2, in detail, the in-sensor processing unit 110 operates at the first clock clk1, and the in-memory operation unit 120 operates at the second clock clk 2. The providing step S110 drives the in-sensor processing unit 110 to provide the first command signal 1111 and the plurality of initial data 1121 and transmit the first command signal 1111 and the plurality of initial data 1121 to the bus unit 130, wherein the first command signal 1111 and each of the plurality of initial data 1121 operate at the first clock clk1, and the bus unit 130 includes the synchronization module 131 and the frame difference module 132. The conversion step S120 drives the synchronization module 131 to receive the first command signal 1111 and the initial data 1121, the first command signal 1111 and the initial data 1121 are converted into a second command signal 1311 and a plurality of input data 1312 through the synchronization module 131, so that the second command signal 1311 and each input data 1312 are operated by the second clock clk2, the first clock clk1 is different from the second clock clk2, thereby transmitting the second command signal 1311 to the in-memory operation unit 120. The acquiring step S130 drives the frame difference module 132 to receive the input data 1312 and acquire a plurality of difference data 1321 according to the input data 1312, wherein each difference data 1321 has a difference address 1322. The dividing step S140 drives the bit dividing module 133 to receive the difference data 1321 and divide each difference data 1321 into a plurality of bit slices 1331. In the control step S150, the encoding module 134 receives the differential address 1322 and encodes the differential address 1322 into the control signal 1341, and the in-memory operation unit 120 accesses each bit slice 1331 according to the control signal 1341.
Thus, the method S100 for integrating the in-sensor processing unit and the in-memory operation unit according to the present invention can transmit the data output by the in-sensor processing unit 110 to the in-memory operation unit 120 for operation through the providing step S110, the converting step S120, the obtaining step S130, the dividing step S140 and the controlling step S150.
For example, in FIG. 5, in the providing step S110 of the method S100 for integrating the in-sensor processing unit and the in-memory operation unit, when the first instruction signal 1111 is 1, each input data 1312 is sequentially transmitted to the bus unit 130. When the first command signal 1111 is 0, each input data 1312 is not transmitted to the bus unit 130. Further, the first clock clk1 may be 10KHz, and the second clock clk2 may be 100 KHz. Continuously, in the converting step S120, the first command signal 1111 and the first clock clk1 of the initial data 1121 are converted into the second command signal 1311 and the second clock clk2 of the input data 1312 through the synchronization module 131. Thereby, the data is synchronized and transmitted from the in-sensor processing unit 110 to the in-memory operation unit 120.
Referring to fig. 5 and 6 together, fig. 6 is a block diagram illustrating the obtaining step S130 of the method S100 for integrating the in-sensor processing unit and the in-memory computing unit according to the embodiment of the method aspect of fig. 5. The bus unit 130 may further comprise a memory 135, wherein the acquiring step S130 comprises a data storing sub-step S131 and a data acquiring sub-step S132. The data storage substep S131 drives the memory 135 to store one of the input data 1312. The data obtaining sub-step S132 drives the frame difference module 132 to read an input data 1312 and compare the input data 1312 with another input data 1312 to obtain difference data 1321. Therefore, the transmission of redundant data volume is reduced, and the transmission efficiency is improved.
Specifically, the difference data 1321 may be 4bits or 8bits between one cycle, but the in-memory unit 120 may only store 1bit when writing or storing data. Therefore, in the dividing step S140, the bit dividing module 133 receives the plurality of difference data 1321 and divides each difference data 1321 into a plurality of bit slices 1331.
Referring to fig. 2 and 4 together, the in-memory unit 120 may include an sram 122. In the control step S150, the encoding module 134 receives the plurality of differential addresses 1322 and encodes them into the control signal 1341, and the in-memory operation unit 120 accesses each bit slice 1331 according to the control signal 1341. Specifically, each bit of each data in the in-memory unit 120 is separately stored in a different address, so that the data needs to be cut into single-bit serial (bitserial) inputs and written into the corresponding sram address 1221. When the control signal 1341 is 1, each bit slice 1331 is sequentially written into the sram 122. When the control signal 1341 is 0, each bit slice 1331 is not written to the sram 122.
Furthermore, the method S100 for integrating the in-sensor processing unit and the in-memory operation unit may further include an output step S160. The output step S160 drives the transmission module 136 to receive the bit slices 1331 and convert the bit slices 1331 into the output data 1361, so that the output data 1361 is transmitted to the in-memory operation unit 120. Each bit 1331 has a first bandwidth (e.g., 1'b1), and the output data 1361 has a second bandwidth (e.g., 1' b 1). Thus, the output data 1361 can satisfy the input bandwidth of the in-memory arithmetic unit 120.
In summary, the present invention has the following advantages: first, the data outputted from the processing unit in the sensor can be transmitted to the operation unit in the memory for operation. And secondly, the energy consumption and the time of operation are reduced. Thirdly, the transmission of redundant data volume is reduced, and the transmission efficiency is improved.
Although the present invention has been described with reference to the above embodiments, it should be understood that various changes and modifications can be made therein by those skilled in the art without departing from the spirit and scope of the invention.

Claims (10)

1. A method for integrating an in-sensor processing unit operating at a first clock and an in-memory computing unit operating at a second clock, the method comprising:
a providing step, driving the sensor internal processing unit to provide a first instruction signal and a plurality of initial data and transmitting the first instruction signal and the plurality of initial data to a bus unit, wherein the first instruction signal and each initial data operate on the first clock, and the bus unit comprises a synchronization module and a picture difference module;
a conversion step, driving the synchronization module to receive the first instruction signal and the initial data, wherein the first instruction signal and the initial data are converted into a second instruction signal and a plurality of input data through the synchronization module, so that the second instruction signal and each input data are operated in the second clock, and the first clock is different from the second clock, thereby transmitting the second instruction signal to the operation unit in the memory;
an obtaining step, driving the picture difference module to receive the input data, and obtaining a plurality of difference data according to the input data, wherein each difference data has a difference address;
a dividing step, driving a bit dividing module to receive the difference data and dividing each difference data into a plurality of bit pieces; and
a control step, driving a coding module to receive the differential address and coding the differential address into a control signal, and the operation unit in the memory accesses each bit slice according to the control signal.
2. The method of claim 1, wherein in said providing step,
when the first command signal is 1, the input data are transmitted to the bus unit in sequence; and
when the first command signal is 0, the input data is not transmitted to the bus unit.
3. The method of claim 1, wherein said bus unit further comprises a memory, and said obtaining step comprises:
a data storage sub-step, driving the memory to store the input data; and
and a data acquisition sub-step, driving the picture difference module to read one input data, and comparing the input data with another input data to acquire each difference data.
4. The method of claim 1, further comprising:
an output step, driving a transmission module to receive the bit slice and convert the bit slice into output data, thereby transmitting the output data to the operation unit in the memory;
wherein each bit slice has a first bandwidth, the output data has a second bandwidth, and the first bandwidth is different from the second bandwidth.
5. The method of claim 4, wherein said in-memory computing unit comprises a static random access memory, and wherein in said controlling step,
when the control signal is 1, each bit slice is written into the static random access memory in sequence; and
when the control signal is 0, each bit slice is not written into the SRAM.
6. An integrated system of an in-sensor processing unit and an in-memory arithmetic unit, comprising:
the sensor internal processing unit comprises a microprocessor and a sensor, wherein the microprocessor is electrically connected with the sensor and is used for generating a first instruction signal and a plurality of initial data;
a bus unit electrically connected to the in-sensor processing unit, and comprising:
the synchronization module is electrically connected with the microprocessor and receives the first instruction signal and the initial data to generate a second instruction signal and a plurality of input data;
the picture difference module is electrically connected with the synchronization module and receives the input data to generate a plurality of difference data, and each difference data has a difference address;
a bit dividing module electrically connected to the picture difference module and receiving the difference data to generate a plurality of bit slices; and
the coding module is electrically connected with the picture difference module and receives the difference address to generate a control signal; and
and the in-memory operation unit is electrically connected with the bus unit and accesses each bit slice according to the control signal.
7. The system of claim 6, wherein the processor unit and the computing unit are integrated into a single chip,
the processing unit in the sensor is operated at a first clock; and
the operation unit in the memory is operated at a second clock;
wherein the first clock is converted into the second clock by the synchronization module;
wherein the first clock is different from the second clock.
8. The system of claim 6, wherein the bus unit further comprises:
a memory electrically connected to the picture difference module;
the memory stores the input data, and the picture difference module reads the input data of the memory and compares the input data with the other input data to obtain the difference data.
9. The system of claim 6, wherein the in-memory computing unit comprises:
a processing unit electrically connected to the synchronization module and the encoding module and receiving the second command signal and the control signal; and
and the static random access memory is electrically connected with the processing unit and accesses each bit slice according to the control signal.
10. The system of claim 9, wherein the bus unit further comprises:
a transmission module electrically connected to the bit dividing module and the SRAM, and receiving the bit slice and converting the bit slice into an output data, thereby transmitting the output data to the SRAM;
wherein each bit slice has a first bandwidth, the output data has a second bandwidth, and the first bandwidth is different from the second bandwidth.
CN202010093130.XA 2020-02-14 2020-02-14 Method and system for integrating in-sensor processing unit and in-memory arithmetic unit Pending CN113269211A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010093130.XA CN113269211A (en) 2020-02-14 2020-02-14 Method and system for integrating in-sensor processing unit and in-memory arithmetic unit

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010093130.XA CN113269211A (en) 2020-02-14 2020-02-14 Method and system for integrating in-sensor processing unit and in-memory arithmetic unit

Publications (1)

Publication Number Publication Date
CN113269211A true CN113269211A (en) 2021-08-17

Family

ID=77227279

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010093130.XA Pending CN113269211A (en) 2020-02-14 2020-02-14 Method and system for integrating in-sensor processing unit and in-memory arithmetic unit

Country Status (1)

Country Link
CN (1) CN113269211A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5305446A (en) * 1990-09-28 1994-04-19 Texas Instruments Incorporated Processing devices with improved addressing capabilities, systems and methods
WO1997040342A2 (en) * 1996-04-24 1997-10-30 Cyra Technologies, Inc. Integrated system for imaging and modeling three-dimensional objects
US20030105906A1 (en) * 2001-11-15 2003-06-05 Nokia Corporation Data processor architecture employing segregated data, program and control buses
KR20050049914A (en) * 2003-11-24 2005-05-27 주식회사 대한전광 Display system using screen split serial transmission form and displaying method thereof
TW200620106A (en) * 2004-12-01 2006-06-16 Tatung Co Ltd Multi-operator address generation circuit
US20130101275A1 (en) * 2011-10-20 2013-04-25 Broadcom Corporation Video Memory Having Internal Programmable Scanning Element
CN108733594A (en) * 2017-04-13 2018-11-02 慧荣科技股份有限公司 Memory controller and data storage device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5305446A (en) * 1990-09-28 1994-04-19 Texas Instruments Incorporated Processing devices with improved addressing capabilities, systems and methods
WO1997040342A2 (en) * 1996-04-24 1997-10-30 Cyra Technologies, Inc. Integrated system for imaging and modeling three-dimensional objects
US20030105906A1 (en) * 2001-11-15 2003-06-05 Nokia Corporation Data processor architecture employing segregated data, program and control buses
KR20050049914A (en) * 2003-11-24 2005-05-27 주식회사 대한전광 Display system using screen split serial transmission form and displaying method thereof
TW200620106A (en) * 2004-12-01 2006-06-16 Tatung Co Ltd Multi-operator address generation circuit
US20130101275A1 (en) * 2011-10-20 2013-04-25 Broadcom Corporation Video Memory Having Internal Programmable Scanning Element
CN103226279A (en) * 2011-10-20 2013-07-31 美国博通公司 Method and system for an adaptive auto-focus algorithm
CN108733594A (en) * 2017-04-13 2018-11-02 慧荣科技股份有限公司 Memory controller and data storage device

Similar Documents

Publication Publication Date Title
CN102369552B (en) Memory subsystem
US20210224125A1 (en) Operation Accelerator, Processing Method, and Related Device
CN102369522B (en) The parallel pipeline formula integrated circuit of computing engines realizes
EP3564864A1 (en) Devices for compression/decompression, system, chip, and electronic device
CN104954795A (en) Image acquisition and transmission system based on JPEG2000
CN110909870B (en) Training device and method
CN109525844B (en) Acceleration system and method for multi-channel video coding and decoding
CN111930681B (en) Computing device and related product
US11048650B1 (en) Method and system for integrating processing-in-sensor unit and in-memory computing unit
CN116114263A (en) Method and apparatus for integrated image signal processing
CN113269211A (en) Method and system for integrating in-sensor processing unit and in-memory arithmetic unit
CN114490023A (en) High-energy physical calculable storage device based on ARM and FPGA
US9330112B2 (en) Grouping and compressing similar photos
CN116185499B (en) Register data transmission method, register cache module, intelligent device and medium
CN114363541B (en) Miniaturized CMOS imaging system and method for cube satellite
US6473527B1 (en) Module and method for interfacing analog/digital converting means and JPEG compression means
CN111382853B (en) Data processing device, method, chip and electronic equipment
CN111382856B (en) Data processing device, method, chip and electronic equipment
CN113627620A (en) Processor module for deep learning
KR102642744B1 (en) Cooperative access method and system for external memory, cooperative access architecture
US11704271B2 (en) Scalable system-in-package architectures
CN113495669A (en) Decompression device, accelerator and method for decompression device
CN111382852A (en) Data processing device, method, chip and electronic equipment
Joshi et al. Implementation of CCSDS Hyperspectral Image Compression Algorith-mon on FPGA on board a nanosatellite
CN212873459U (en) System for data compression storage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20220217

Address after: 1 / F, 30 / F, 118 Ciyun Road, East District, Hsinchu, Taiwan, China

Applicant after: Egis Technology Inc.

Address before: 2 / F, No. 7, Lane 5, Sanfu street, Wenshan District, Taipei, Taiwan, China

Applicant before: Zheng Guizhong