KR102408936B1 - Image recognition processor including functional safety processor core and operation method thereof - Google Patents

Abstract

The image recognition processor according to the present invention includes a plurality of nanocores arranged in rows and columns that perform a pattern recognition operation using kernel coefficients on input features in response to each instruction, and the instruction to each of the plurality of nanocores. A command memory providing the input feature, a feature memory providing the input feature to each of the plurality of nanocores, a kernel memory providing the kernel coefficients to the plurality of nanocores, and pattern recognition output from the plurality of nanocores A functional safety processor core that receives a result of the operation to detect the presence of a recognition error, and performs an error tolerance function for the detected recognition error, wherein the functional safety processor core includes a first driving voltage and a first driving voltage a first processor driven by a clock, a second driving voltage, and a second driving clock driven by a second processor performing the same operation as the first processor, wherein the first driving voltage and the second driving Each of the voltages is supplied from a mutually independent voltage regulator, and each of the first driving clock and the second driving clock is supplied from mutually independent clock generators.

Description

Image recognition processor including functional safety processor core and method of operation thereof

The present invention relates to an electronic device, and more particularly, to an image recognition processor including a functional safety processor core that performs an error detection and error prevention function for a recognized pattern, and an operating method thereof.

Recently, in automobiles and various transportation systems, the development of an Advanced Driver Assistance System (ADAS) or a system requiring high intelligence and precision for autonomous driving is being actively developed. In particular, performance for analyzing and processing a large amount of data in real time is required in order to control the vehicle by analyzing the recognition of the external environment and input of recognized images, voices, or sensors. In such a system, since even a small recognition error can cause a major accident, the reliability of the processor system that performs the recognition of sensor data is particularly important.

As such, an apparatus for processing sensor data and recognizing the pattern must be designed to include a fault tolerance function in order to operate in harsh environments such as automobiles and airplanes. A pattern recognition system such as a deep neural network system uses a distributed computing technique using a large number of CPU cores. In particular, a technology capable of detecting a recognition error caused by various errors or defects and recovering the detected recognition error still requires a lot of research.

An object of the present invention is to provide an image recognition processor having a function of detecting an operation error of a processor for recognizing a pattern by processing sensor data in a semiconductor, and tolerating the detected error using the characteristics of the sensor data processing result. .

An image recognition processor according to an embodiment of the present invention includes a plurality of nanocores arranged in a row and a column for performing a pattern recognition operation using kernel coefficients on input features in response to commands, each of the plurality of nanocores a command memory providing the instruction to, a feature memory providing the input feature to each of the plurality of nanocores, a kernel memory providing the kernel coefficients to the plurality of nanocores, and from the plurality of nanocores A functional safety processor core that receives a result of the output pattern recognition operation, detects the presence of a recognition error, and performs an error tolerance function for the detected recognition error, wherein the functional safety processor core includes a first driving voltage and a first processor driven by a first driving clock, a second driving voltage, and a second processor driven by a second driving clock and performing the same operation as the first processor, wherein the first driving voltage and Each of the second driving voltages is supplied from a mutually independent voltage regulator, and each of the first driving clock and the second driving clock is supplied from mutually independent clock generators.

The image recognition processor according to the above-described embodiment of the present invention can reduce the possibility of a malfunction due to a temporary error or a permanent error occurring in the process of processing the sensor device or sensor data. Therefore, it is expected that the semiconductor malfunction due to the limit of the recognition rate of sensor data in the image recognition processor can be prevented.

1 is a block diagram schematically illustrating an image recognition processor according to an embodiment of the present invention.
2 is a diagram schematically illustrating an image recognition process performed in the core array of the present invention.
3 is a block diagram specifically showing the structure of an image recognition processor according to an embodiment of the present invention.
4 is a diagram schematically showing data movement in the nanocores of the present invention.
5 is a block diagram exemplarily showing the configuration of the functional safety processor core of FIG. 3 .
6 is a diagram exemplarily showing a grouping method of nanocores for detecting a recognition error.
7 is a flowchart illustrating a method for detecting a recognition error and allowing an error according to an embodiment of the present invention.

Hereinafter, in order to describe in detail enough that a person of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention, an embodiment of the present invention will be described with reference to the accompanying drawings. .

1 is a block diagram schematically showing the structure of an image recognition processor according to an embodiment of the present invention. Referring to FIG. 1 , the image recognition processor 100 of the present invention includes a core array 110 , a kernel memory 120 , an instruction memory 130 , a delay chain 140 , a feature memory 150 , and a functional safety processor core. 160 , and a bus 170 .

The core array 110 processes input data Fm provided from the feature memory 150 using commands and kernel information provided by the command memory 130 and the kernel memory 120 . The core array 110 processes, for example, when a sensed image is received, using an instruction and a kernel for identifying a pattern of the sensed image. For example, the core array 110 may perform a down-sampling operation such as a convolution operation or a pooling operation.

In particular, the core array 110 may include a plurality of nano-cores (hereinafter, NCs) arranged in rows and columns for processing a convolution operation or a pooling operation. The nanocores may process provided features in a parallel processing method or a sequential processing method. In the present invention, a plurality of nanocores may be controlled to process the same operation with a specific delay time. In addition, the result of the pattern recognition operation processed by the plurality of nanocores at regular time intervals may be provided to the functional safety processor core 160 via the feature memory 150 . The plurality of nanocores NCs included in the core array 110 may be divided into groups. In addition, the nanocores of each group may perform a pattern recognition operation on the same image or different images.

The kernel memory 120 provides a plurality of kernel information to the core array 110 in synchronization with a command provided to the core array 110 . The kernel memory 120 may provide various parameters necessary for kernel information, coefficients, bias addition, activation, pooling, and the like for a convolution operation performed in the core array 110 . The kernel memory 120 may provide, for example, a kernel coefficient in units of columns to the nanocores of the core array 110 . Then, each of the nanocores may perform a pattern recognition operation using the received kernel coefficients and, at the same time, transmit the received kernel coefficients to the nanocores located in the lower row.

The instruction memory 130 stores instructions to be executed by the plurality of nano-cores NC included in the core array 110 , respectively. And in synchronization with the input features, the instruction memory 130 will transfer the stored instruction to the delay chain 140 . Then, in the delay chain 140 , one instruction is delayed by a specific delay unit. Instructions having a specific delay time generated by the delay chain 140 may be provided to different nanocores. Any one instruction INS provided from the instruction memory 130 may be sequentially provided to the plurality of nanocores with a specific delay time. The unit in which the delayed instruction is provided may be a row unit of the nano-core.

The feature memory 150 stores an input image or data to be processed by the image recognition processor 100 . The input image to be recognized is stored in the feature memory 150 and transmitted to the core array 110 . For example, image data provided from an image sensor may be stored in the feature memory 150 . In addition, a feature map processed in one layer may be stored in the feature memory 150 .

The functional safety processor core 160 may receive the result of the operation performed in the core array 110 and detect whether a recognition error has occurred. For example, the functional safety processor core 160 may compare outputs of a plurality of nanocores NC using the same input feature, the same instruction, and the same kernel coefficient. For example, the nanocores NC may be divided into different groups, and each group may be controlled to perform the same pattern recognition operation. And the recognition results output from each group may be compared by the functional safety processor core 160 . If it is detected that there is a difference in the pattern recognition result value of each of the groups that have performed the same operation, the functional safety processor core 160 will determine that a recognition error exists. However, if the outputs of each of the nanocore groups that have performed the same operation are detected to be the same, the functional safety processor core 160 will determine that there is no recognition error. In addition, the functional safety processor core 160 may apply a fault tolerance function even if a recognition error exists when the degree of recognition error is not large.

In particular, the functional safety processor core 160 may include a plurality of processors that perform the same operation. The plurality of processors may receive independent clock signals and driving voltages, respectively. The functional safety processor core 160 using independent clock signals and driving voltages may be free from common cause faults such as power sources or clock sources. Through this configuration, the functional safety processor core 160 may provide a highly reliable error detection and error tolerance function.

Bus 170 provides a data transfer path between functional safety processor core 160 and kernel memory 120 , instruction memory 130 , and feature memory 150 .

In the above, the structure of the image recognition processor of the present invention has been briefly described. By the functional safety processor core 160 of the present invention, results of calculations performed by a plurality of nano-cores NC can be compared, and recognition errors can be identified. In addition, if the error tolerance function is applied by applying a specific margin according to the degree of the identified recognition error, the reliability of the image recognition processor of the present invention can be increased.

2 is a diagram schematically illustrating an example of a convolution operation performed by nanocores according to an embodiment of the present invention. Referring to FIG. 2 , an input feature 152 is generated as a feature map 154 by a convolution operation. In addition, the feature map 154 may be output as the feature maps 156 by a pooling operation.

The input features 152 are converted into an array-type feature map 154 by convolution operations with the kernel 121 . When values of elements generated through convolution of the input feature 152 and the location overlapping the kernel 121 are added, a feature value corresponding to a point in the feature map 154 is generated. If a convolution operation is performed while shifting the kernel 121 with respect to the positions of all input features 152 , one feature array may be generated. If such a convolution operation is performed using a plurality of kernels 122 (M pieces), a plurality (M pieces) of feature maps 154 may be generated.

Here, the input feature 152 processed through the convolution operation may be substantially differentiated into various components. For example, one input feature 152 may be divided into image data expressing color and contrast for each of the RGB components. In addition, the input feature 152 may be differentiated even with data having a variable size. A processing unit for performing a convolution operation using all the kernels 122 and M on these various image data is hereinafter referred to as a convolution loop. These convolution loops will be executed for a plurality of depths of the input feature 152 .

In addition, a convolution operation using a different kernel 124 for the feature map 154 may be followed. A feature map 154 may be generated through such a convolution operation on any one input feature 152 .

The nanocores NC included in the core array 110 of the present invention may process these operations in a parallel processing manner. Alternatively, the nanocores NC may be driven in such a way that one input image is processed in parallel by applying different kernels. Alternatively, the nanocores NCs may be divided into groups, and each group may process a pattern recognition operation for the same input feature (frame data) or different input features. In each case, the functional safety processor core 160 may compare the operation results and detect the presence of a recognition error or perform an error tolerance function of a predetermined margin.

3 is a block diagram specifically showing the configuration of an image recognition processor of the present invention. Referring to FIG. 3 , the core array 110 , the kernel memory 120 , the delay chain 140 , and the feature memory 150 may each include a plurality of sub-elements.

The core array 110 may include, for example, a plurality of nanocores NC_xy, 0 ≤ x, y ≤ 3 arranged in rows and columns. Each of the nanocores NC_xy may perform a convolution operation for pattern recognition using input commands, data, and kernel coefficients. The nano-cores NC_xy may process provided input features in a parallel processing method or a sequential processing method.

Any one of the nano-cores NC_xy may be provided from the instruction memory 130 or may receive a command from the adjacent nano-core NC_x(y-1) in the left direction. In addition, the nano-core NC_xy may receive an input feature from any one of the feature memories 152 , 154 , 156 , 158 or from the nano-core NC_x(y-1) adjacent to the left in the left direction. In addition, the nano-core NC_xy provides a kernel coefficient for operation from any one of the kernel memories 122 , 124 , 126 , and 128 or from the nano-core NC_(x-1)y adjacent in the upward direction. can receive

Also, the nano-core NC_xy may perform an operation corresponding to the provided instruction using the received input feature and kernel coefficients. Simultaneously performing the operation according to the instruction, the nano-core (NC_xy) transfers the input feature to the right nano-core (NC_x(y+1)), and transfers the kernel coefficient received from the upper side to the lower adjacent nano-core ( NC_(x+1)y).

In addition, the nano-core NC_xy transmits the result of the operation to one of the nano-core NC_x(y-1) adjacent to the left or the feature memories 152 , 154 , 156 , and 158 . For example, the nanocore NC_01 transfers the result of an operation using the input feature and the kernel coefficient to the nanocore NC_00 in the same row. In addition, the nano-core NC_01 may receive the operation result of the nano-core NC_02 and transmit it to the nano-core NC_00. In this way, the nano-core NC_00 transmits its own operation result and the operation result of the nano-cores NC_01 , NC_02 , and NC_03 included in the same row to the feature memory 152 , respectively.

According to the above-described method, the plurality of nanocores NC_xy included in one row may perform a convolution operation on one input feature using coefficients corresponding to different kernels. That is, the plurality of nanocores NC_xy may process a plurality of convolution operations using a plurality of different kernels for one input feature. In addition, the operation result of each of the nano-cores NC_xy may be compared or evaluated by any one of the feature memories 152 , 154 , 156 , 158 or the functional safety processor core 160 .

In the above-described manner, the nanocores NC_1y, 0≤y≤3 use the kernel coefficients provided via the upper nanocores NC_0y, 0≤y≤3 in response to a command from the delay circuit 144 . to process the input features provided from the feature memory 154 . That is, the nano-cores NC_1y, 0≤y≤3 perform an operation on the input feature provided from the feature memory 154 and transfer the operation to the feature memory 154 .

The nano-cores NC_2y and 0≤y≤3 constituting one row in the core array 110 transfer the operation result to the feature memory 156 . Similarly, the nano-cores NC_3y, 0≤y≤3 will also perform an operation using the input features and kernel coefficients from the feature memory 158 and transfer them to the feature memory 158 .

Each of the feature memories 152 , 154 , 156 , and 158 may receive the pattern recognition operation result transmitted from the nano-cores NC_xy and transmit it to the functional safety processor core 160 via the bus 170 . In addition, the feature memories 152 , 154 , 156 , and 158 may provide an input feature provided by the functional safety processor core 160 to the nanocores NC_xy .

The functional safety processor core 160 may determine whether there is a recognition error by comparing operation results of each of the plurality of nanocores NC_xy included in one row. Alternatively, the functional safety processor core 160 may determine whether a recognition error exists by comparing the operation results of the plurality of nanocores NC_xy in each row. In addition, the functional safety processor core 160 may apply a fault tolerance function when the error is less than or equal to a predefined reference value even if there is an error in the result of the pattern recognition operation.

In particular, the functional safety processor core 160 of the present invention may include a plurality of processors capable of performing the same operation. The plurality of processors may receive independent clock signals and driving voltages, respectively. The functional safety processor core 160 using independent clock signals and driving voltages may be free from common cause faults such as power sources or clock sources.

4 is a diagram schematically illustrating operations of nanocores in a row unit according to an embodiment of the present invention. Referring to FIG. 4 , detailed operations of the nanocores NC_0y and 0≤y≤3 that receive input features from the feature memory 152 will be briefly described.

The nano-core NC_00 may receive the command INS output from the delay circuit 142 . Then, the nano-core NC_00 receives the input feature F_IN_0 from the feature memory 152 along with the reception of the instruction INS, and receives the kernel coefficient Coff_0 from the kernel memory 122 . The nano-core NC_00 performs an operation using the input feature F_IN_0 and the kernel coefficient Coff_0 in response to the command INS. For example, the nanocore NC_00 may perform a convolution operation using a kernel corresponding to the kernel coefficient Coff_0 for the input feature F_IN_0. At the same time, the nano-core NC_00 will transfer the kernel coefficient Coff_0 transmitted from the kernel memory 122 to the nano-core NC_10 located in the same column. In addition, the nano-core NC_00 may transfer the input feature F_IN_0 transmitted from the feature memory 152 to the adjacent nano-core NC_01 in the row direction.

The nano-core NC_01 receives the input feature F_IN_0 and the instruction INS transmitted via the nano-core NC_00. In addition, the nano-core NC_01 receives the kernel coefficient Coff_1 from the kernel memory 124 . The nanocore NC_01 may perform a convolution operation using the input feature F_IN_0 and the kernel coefficient Coff_1 in response to the instruction INS. Also, the nano-core NC_01 may transfer the kernel coefficient Coff_1 transferred from the kernel memory 124 to the nano-core NC_11 located in the same column. In addition, the nano-core NC_01 transfers the input feature F_IN_0 and the instruction INS transmitted via the nano-core NC_00 to the adjacent nano-cores NC_02 in the row direction, respectively.

The nano-core NC_02 receives the input feature F_IN_0 and the instruction INS transmitted via the nano-core NC_01. The nano-core NC_02 receives the coefficient Coff_2 from the kernel memory 126 . The nano-core NC_02 may perform a convolution operation using the input feature F_IN_0 and the kernel coefficient Coff_2 in response to the instruction INS. Also, the nano-core NC_02 may transfer the kernel coefficient Coff_2 transmitted from the kernel memory 126 to the nano-core NC_12 located in the same column. In addition, the nano-core NC_02 transfers the input feature F_IN_0 and the instruction INS transmitted via the nano-core NC_01 to the adjacent nano-cores NC_03 in the row direction, respectively.

The nano-core NC_03 receives the input feature F_IN_0 and the instruction INS transmitted via the nano-core NC_02. The nano-core NC_03 receives the kernel coefficient Coff_3 from the kernel memory 128 . The nano-core NC_03 may perform a convolution operation using the input feature F_IN_0 and the kernel coefficient Coff_3 in response to the instruction INS. Also, the nano-core NC_03 will transfer the kernel coefficient Coff_3 transferred from the kernel memory 128 to the nano-core NC_13 located in the same column. Since the nano-core NC_03 is the last core on the right side of the row, the transfer operation of the instruction INS or the input feature F_IN_0 may not be performed.

The nano-core NC_03 will transmit the operation result F_OUT_03 to the left nano-core NC_02. The nano-core NC_02 transfers its own operation result F_OUT_02 and the operation result F_OUT_03 provided from the right nano-core NC_03 to the left nano-core NC_01. The nano-core NC_01 transfers its operation result F_OUT_01 and the operation results F_OUT_02 and F_OUT_03 provided from the right nano-cores NC_02 and NC_03 to the left nano-core NC_00. As a result, the nano-core NC_00 transmits its operation result F_OUT_00 and the operation results F_OUT_01 , F_OUT_02 , and F_OUT_03 provided from the right nano-cores NC_01 , NC_02 , and NC_03 to the feature memory 152 . The operation results F_OUT_00 , F_OUT_01 , F_OUT_02 , and F_OUT_03 transferred to the feature memory 152 will be transferred to the functional safety processor core 160 .

In the above, the input/output relationship between commands, input features, coefficients, and calculation results of the plurality of nanocores NC_00, NC_01, NC_02, and NC_03 receiving the input feature F_IN_0 from the feature memory 152 has been briefly described. Like the nanocores NC_00, NC_01, NC_02, and NC_03 constituting the first row of the core array 110 , the nanocores NC_10 , NC_11 , NC_12 , and NC_13 are driven in the same manner except for the input features different. can be The nanocores (NC_20, NC_21, NC_22, NC_23) and the nanocores (NC_30, NC_31, NC_32, NC_33) constituting one row differ only in the feature memory provided with the input feature, as well as the nanocores NC_00, NC_01, NC_02, NC_03) can be regarded as operating in the same way. Of course, it will be well understood that the instruction transfer time by the delay circuits 142 , 144 , 146 , 148 or the instruction, input feature, and coefficient transfer timing generated in each core may be different.

5 is a block diagram exemplarily showing a detailed configuration of the functional safety processor core of FIG. 3 . Referring to FIG. 5 , the functional safety processor core 160 may be provided in a multi-core structure. That is, the functional safety processor core 160 may include a first processor 162 , a second processor 164 , an external fault manager 166 , and a recovery controller 168 . In addition, the functional safety processor core 160 may include a first phase locked loop 161 , a second phase locked loop 165 , a first LDO 163 (Low Dropout), and a second LDO 167 .

The first phase-locked loop 161 and the second phase-locked loop 165 are provided as separate clock generation circuits. The first phase locked loop 161 may provide the first driving clock CLK1 to the first processor 162 . The second phase locked loop 165 may provide the second driving clock CLK2 to the second processor 164 . Each of the first and second driving clocks CLK1 and CLK2 are clock signals provided from independent sources from which mutual interference or influence is blocked. Accordingly, even if the first driving clock CLK1 is exposed to noise or interference, the second driving clock CLK2 may maintain a normal frequency or phase. It will be well understood that the first phase-locked loop 161 and the second phase-locked loop 165 may be implemented with various circuits that generate a clock other than the phase-locked loop (PLL) method.

The first LDO 163 and the second LDO 167 may be provided as mutually independent power supply circuits. The first LDO 163 generates a first driving voltage VDD1 and provides it to the first processor 162 . The second LDO 167 generates a second driving voltage VDD2 and provides it to the second processor 164 . Accordingly, the first and second driving voltages VDD1 and VDD2 are independent power supply voltages from which mutual interference or influence is blocked. For example, each of the first LDO 163 and the second LDO 167 may be provided as voltage regulators operating independently of each other. Alternatively, each of the first LDO 163 and the second LDO 167 may be provided as a power management integrated circuit (PMIC) that operates independently of each other. Accordingly, even if the first driving voltage VDD1 fluctuates due to noise or interference, the second driving voltage VDD2 may maintain a normal level. Here, it will be well understood that each of the first and second driving voltages VDD1 and VDD2 may be provided at the same level or different levels.

According to the first processor 162 and the second processor 164 using the clock signal and the driving voltage independent of each other, it is possible to be free from common cause faults such as the driving voltage or the clock signal.

The first processor 162 and the second processor 164 may each process the same task. The first processor 162 and the second processor 164 compare results of operations performed on the core array 110 (refer to FIG. 1 ) transmitted through the feature memory 150 , respectively. The operation result transmitted through the feature memory 150 is equally transmitted to the first processor 162 and the second processor 164 . Then, the first processor 162 and the second processor 164 may determine whether there is an error in the core array 110 by comparing independently transmitted operation results.

In addition, the external defect manager 166 may determine whether a defect or an error exists by comparing the error detection results of the first processor 162 and the second processor 164 with each other. The first processor 162 and the second processor 164 may transmit the result to the external defect manager 166 while performing the same task. Then, the external fault manager 166 may detect a transient fault or a permanent fault by comparing the processing results of the first processor 162 and the second processor 164 for each specific checkpoint. .

The repair controller 168 performs various control operations for repairing the detected error with reference to the detection result of the internal defect managers 162_7 , 162_9 , 164_7 , 164_9 or the external defect manager 166 . The repair controller 168 may correct or repair an error detected based on error trap information from at least one of the internal defect managers 162_7, 162_9, 164_7, and 164_9, for example. The repair controller 168 may perform an error or defect repair operation based on error trap information provided from the external defect manager 166 .

The first processor 162 includes a first processor core 162_1 , a first instruction cache 162_3 , a first data cache 162_5 , a first internal fault manager 162_7 , and a second internal fault manager 162_9 . may include The first processor core 162_1 performs various operations based on instructions and data provided through the first instruction cache 162_3 and the first data cache 162_5 inside the first processor 162 . In particular, the first internal defect manager 162_7 may detect and repair errors or defects occurring in the first instruction cache 162_3 . The second internal defect manager 162_9 may detect or repair an error or defect occurring in the first data cache 162_5 .

The second processor 164 includes a second processor core 164_1, a fourth instruction cache 164_3, a second data cache 164_5, a third internal fault manager 164_7, and a fourth internal fault manager 164_9. may include The second processor core 164_1 performs various operations based on instructions and data provided through the second instruction cache 164_3 and the second data cache 164_5 inside the second processor 164 . In particular, the third internal defect manager 164_7 may detect and repair errors or defects occurring in the second instruction cache 164_3 . The fourth internal defect manager 164_9 may detect or repair an error or defect occurring in the second data cache 164_5 .

The functional safety processor core 160 having the above-described configuration and function may receive the result of the operation processed by the plurality of nanocores to detect whether a recognition error has occurred. In particular, the first processor 162 and the second processor 164 are driven by independent clock signals CLK1 and CLK2 and driving voltages VDD1 and VDD2, respectively. Accordingly, the first processor 162 and the second processor 164 may be free from common cause faults generated from driving voltages or clock signals, respectively. Accordingly, operational reliability of the functional safety processor core 160 may be provided.

6 is a block diagram exemplarily showing a unit of error detection performed by the functional safety processor core of FIG. 5 . Referring to FIG. 6 , the nanocores may be divided into a plurality of groups. Each group can be controlled to process the same frame. Alternatively, each group may be controlled to process different frames.

For example, the nanocores NC_00, NC_01, NC_02, and NC_03 positioned in the first row of the core array 110 are classified into a first group, and the nanocores NC_10, NC_11, NC_12, NC_13) may be classified into the second group. In addition, the nanocores NC_20, NC_21, NC_22, NC_23, NC_30, NC_31, NC_32, and NC_33 positioned in the third and fourth rows may be classified into a third group.

Based on the group classification described above, a pattern recognition operation for the first frame Frame_N may be assigned to the nanocores of the first group and the second group. That is, the input feature of the first frame Frame_Na may be transmitted to the first group of nanocores NC_00 , NC_01 , NC_02 , and NC_03 through the feature memory 152 . In addition, the input feature of the first frame Frame_Nb may be transferred to the second group of nanocores NC_10 , NC_11 , NC_12 , and NC_13 through the feature memory 154 . Here, the frames Frame_Na and Frame_Nb mean the same frame data but are allocated to different nanocore groups.

In addition, a pattern recognition operation for the second frame Frame_N+1 may be allocated to the third group of nanocores positioned in the third and fourth rows of the core array 110 . That is, in the third group of nanocores NC_20, NC_21, NC_22, NC_23, NC_30, NC_31, NC_32, NC_33, the input feature of the second frame Frame_N+1 different from the first frame Frame_Na or Frame_Nb is may be passed through feature memories 156 and 158, respectively.

The functional safety processor core 160 may determine whether a pattern recognition operation is in error by comparing operation results output by the nanocores of the first group and the second group. In addition, when an error occurs, a difference value from the previous frame Frame_N-1 is calculated, and any one of the operation results of each of the first frames Frame_Na or Frame_Nb may be selected as a result of pattern recognition.

7 is a flowchart schematically illustrating an operation of a functional safety processor core according to an embodiment of the present invention. Referring to FIG. 7 , a method for the functional safety processor core 160 to process one frame Frame_N is illustrated. The functional safety processor core 160 allocates pattern recognition of the current frame image Frame_N to the nanocores of the first group and the second group, respectively. In addition, the functional safety processor core 160 may determine whether an error exists and select an optimal result by comparing the processing result values of the pattern recognition.

In step S110 , the functional safety processor core 160 equally allocates the current frame image Frame_N to the plurality of nanocore groups. For example, the functional safety processor core 160 may simultaneously allocate the current frame image Frame_N to two nanocore groups. In addition, the functional safety processor core 160 will receive the processing result for the frame image Frame_N allocated from the two nano-core groups.

In step S120, the functional safety processor core 160 compares the processing results transmitted from each of the two nano-core groups. That is, the functional safety processor core 160 processes the processing result R (Frame_Na) of the frame image Frame_N of the first group of nanocores and the processing result R (Frame_Nb) of the frame image (Frame_N) of the second group of nanocores. ) will be compared. If the processing results are the same (yes direction), the procedure moves to step S140. On the other hand, if the processing result is different (No direction), the procedure moves to step S130.

In step S130, the functional safety processor core 160 determines the difference value [R(Frame_Na) between the processing result R(Frame_Na) of the first group for the current frame image (Frame_N) and the processing result R(Frame_N-1) for the previous frame )-R(Frame_N-1)] is calculated. And the functional safety processor core 160 compares the calculated difference value with the reference value Ref. If the difference value [R(Frame_Na)-R(Frame_N-1)] is not smaller than the reference value Ref (No direction), the procedure moves to step S150. On the other hand, if the difference value [R(Frame_Na)-R(Frame_N-1)] is not smaller than the reference value Ref (yes direction), the procedure moves to step S140.

In step S140 , the functional safety processor core 160 selects the processing result R(Frame_Na) of the first group as a result of the image recognition operation. Then, the procedure moves to step S170 to proceed with the process for processing the next frame (Frame_N+1).

In step S150, the functional safety processor core 160 determines the difference between the processing result R(Frame_Nb) of the second group and the processing result R(Frame_N-1) for the previous frame [R(Frame_Nb)-R(Frame_N-1)] ] is calculated. And the functional safety processor core 160 compares the calculated difference value with the reference value Ref. If the difference value [R(Frame_Nb)-R(Frame_N-1)] is not smaller than the reference value Ref (No direction), the procedure moves to step S180. On the other hand, if the difference value [R(Frame_Nb)-R(Frame_N-1)] is not smaller than the reference value Ref (yes direction), the procedure moves to step S160.

In step S160 , the functional safety processor core 160 selects the processing result R(Frame_Nb) of the second group as a result of the image recognition operation. Then, the procedure moves to step S170 to proceed with the process for processing the next frame (Frame_N+1).

In step S180 , the functional safety processor core 160 determines that an error has occurred in the image recognition operation for the current frame Frame_N. In addition, the functional safety processor core 160 may perform an additional error recovery operation according to the determination result.

In the above, the error detection operation by the functional safety processor core 160 of the present invention has been briefly described. However, it will be well understood that the above-described examples are merely examples for explaining the advantages of the present invention, and various changes are possible without departing from the spirit of the present invention.

The contents described above are specific examples for carrying out the present invention. The present invention will include not only the above-described embodiments, but also simple design changes or easily changeable embodiments. In addition, the present invention will include techniques that can be easily modified and implemented in the future using the above-described embodiments.

Claims (15)

a plurality of nanocores arranged in rows and columns for performing a pattern recognition operation using kernel coefficients on input features in response to each instruction;
an instruction memory providing the instruction to each of the plurality of nano-cores;
a feature memory providing the input feature to each of the plurality of nano-cores;
a kernel memory providing the kernel coefficients to the plurality of nanocores; and
A functional safety processor core that receives the result of the pattern recognition operation output from the plurality of nano-cores, detects the presence of a recognition error, and performs an error tolerance function for the detected recognition error,
The functional safety processor core comprises:
a first processor driven by a first driving voltage and a first driving clock; and
a second processor driven by a second driving voltage and a second driving clock and performing the same operation as the first processor;
The first driving voltage and the second driving voltage are respectively supplied from mutually independent voltage regulators, and the first driving clock and the second driving clock are each supplied from mutually independent clock generators. The method of claim 1,
The functional safety processor core comprises:
a first LDO providing the first driving voltage; and
and a second LDO providing the second driving voltage. The method of claim 1,
The functional safety processor core comprises:
a first phase locked loop providing the first driving clock; and
and a second phase locked loop providing the second driving clock. The method of claim 1,
The functional safety processor core comprises:
an external defect manager that compares the processing results of the first processor and the second processor to detect whether an error or a defect exists; and
The image recognition processor further comprising a recovery controller for recovering the detected error or defect. The method of claim 1,
The first processor includes:
a first processor core to perform the requested operation;
a first instruction cache providing instructions input to the first processor core;
a first data cache providing cache data to the first processor core;
a first internal fault manager for managing errors in the first instruction cache; and
and a second internal defect manager for managing errors in the first data cache. The method of claim 1,
The second processor includes:
a second processor core to perform the requested operation;
a second instruction cache providing instructions input to the second processor core;
a second data cache providing cache data to the second processor core;
a third internal fault manager for managing errors in the second instruction cache; and
and a fourth internal defect manager for managing errors in the second data cache. The method of claim 1,
The functional safety processor core allocates an input feature corresponding to a current frame to each of the first group and the second group among the plurality of nanocores, and performs an operation of each of the first group and the second group An image recognition processor that compares the results. 8. The method of claim 7,
The functional safety processor core selects either the first recognition result or the second recognition result as an output value when the first recognition result output from the first group and the second recognition result output from the second group are the same image recognition processor. 9. The method of claim 8,
When the first recognition result and the second recognition result are different from the first recognition result, the functional safety processor core is configured to refer to a third recognition result corresponding to a result of a recognition operation for a previous frame, and the first recognition result or the second recognition result An image recognition processor that selects any one as an output value. 10. The method of claim 9,
The functional safety processor core selects one of the first recognition result and the second recognition result with a smaller difference between the third recognition result and the image recognition processor. 11. The method of claim 10,
The functional safety processor core determines a recognition error when the difference between the first recognition result and the third recognition result and the difference between the second recognition result and the third recognition result are not smaller than a reference value, respectively. recognition processor. A method of operating an image recognition processor including a plurality of nano-cores, the method comprising:
allocating a current frame image to a first group of nanocores and a second group of nanocores from among the plurality of nanocores;
comparing a first recognition result output by the first group of nanocores with a second recognition result output by the second group of nanocores;
selecting the first recognition result as a recognition result of the current frame image when the first recognition result is the same as the second recognition result; and
When the first recognition result and the second recognition result are different from each other, an error is allowed by referring to a third recognition result corresponding to a recognition result value for a previous frame, and a difference value between the first recognition result and the second recognition result or determining as a recognition error. 13. The method of claim 12,
The step of determining that the error is accepted or the recognition error is:
and comparing a first difference value between the first recognition result and the third recognition result and a second difference value between the second recognition result and the third recognition result with a reference value. 14. The method of claim 13,
When the first difference value and the second difference value are respectively smaller than the reference value, an operation method of selecting any one of the first recognition result and the second recognition result having a smaller difference between the reference value as an error tolerance value . 14. The method of claim 13,
When the first difference value and the second difference value are not smaller than the reference value, respectively, it is determined as a recognition error.

Images