WO2020152716A1

WO2020152716A1 - System-on-chip interfaced with sensor for processing sensor output signals

Info

Publication number: WO2020152716A1
Application number: PCT/IN2020/050077
Authority: WO
Inventors: Ankur Anchlia
Original assignee: Ankur Anchlia
Priority date: 2019-01-25
Filing date: 2020-01-24
Publication date: 2020-07-30

Abstract

A system-on-chip (SoC) is interfaced with a sensor for driving the sensor and processing an output signal generated by the sensor. The sensor is an infrared imaging sensor or a light detection and ranging sensor. The SoC includes driving circuitry that drives the sensor by providing a set of supply voltages, a set of reference voltages, a set of clock signals, and a set of timing signals, and processing circuitry that executes a set of processing operations on the output signal to generate a processed output signal. The SoC further includes feedback circuitry that generates a set of control signals based on the processed output signal and environmental data indicative of environmental conditions associated with the sensor. The set of control signals are provided to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor.

Description

SYSTEM-ON-CHIP INTERFACED WITH SENSOR FOR PROCESSING SENSOR

OUTPUT SIGNALS

FIELD OF THE INVENTION

[0001] The present invention relates generally to electronic circuits, and more particularly, to system-on-chips (SoCs) that are interfaced with sensors for driving the sensors and processing output signals generated by the sensors.

BACKGROUND

[0002] Now-a-days, infrared (IR) imaging sensors and light detection and ranging (LiDAR) sensors (collectively referred to as“sensors”) are used in a wide range of applications. For example, the LiDAR sensors are utilized in automobile applications for facilitating driver assistance, whereas the IR imaging sensors are utilized in medical and night vision applications for capturing IR images of a scene. Typically, such sensors are interfaced with Field Programmable Gate Arrays (FPGAs) that process output signals generated by the sensors, and with various discrete printed circuit boards (PCBs) that drive the sensors. The use of the FPGAs and the PCBs results in an increase in size, weight, and cost of a camera system that houses the FPGAs, the PCBs, and the sensors, and an increase in power consumed by the camera system.

[0003] A conventional solution to solve the above-mentioned problems is to use a system-on-chip (SoC), in place of the FPGAs and the PCBs, for driving the sensors and for processing the output signals generated by the sensors. The SoC includes driving circuitry for driving the sensors and processing circuitry for processing the output signals generated by the sensors. This results in a decrease in the size, weight, and cost of the camera system, and a decrease in the power consumed by the camera system. However, the use of the SoC has shortcomings of its own. The driving and processing circuitries associated with IR imaging sensors and LiDAR sensors vary widely. Further, each of the IR imaging and LiDAR sensors has various operating characteristics such as resolutions, formats, operational modes, or the like, and the driving and processing circuitries associated with various combinations of operating characteristics vary widely. Thus, it is difficult for a single conventional SoC to interface with both the IR imaging and LiDAR sensors having various operating

characteristics. As a result, different SoCs are required to drive the sensors of different types and operating characteristics. Further, a processed output signal generated by the processing circuitry of the SoC require additional processing that is typically facilitated by additional circuitries (e.g., FPGA) on the camera system. The additional circuitries occupy significant area, consume significant power, and significantly adds to weight and cost of the camera system. Further, variations in environmental conditions associated with the sensors cause drastic effects in the operation of the sensors leading to irregularities in the output signals generated by the sensors. The conventional SoCs lack dynamism in countering such drastic effects. As a result, the irregularities in the output signals persist, and consequently lead to irregularities in the processed output signals generated by the conventional SoCs.

[0004] Thus, it would be advantageous to have an SoC that drives and processes output signals of both IR imaging and LiDAR sensors with various operating characteristics.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The following detailed description of the preferred embodiments of the present invention will be better understood when read in conjunction with the appended drawings.

The present invention is illustrated by way of example, and not limited by the accompanying figures, in which like references indicate similar elements.

[0006] FIG. l is a block diagram of a camera system in accordance with an embodiment of the present invention;

[0007] FIG. 2 is a block diagram of processing circuitry of the camera system of FIG. 1 in accordance with an embodiment of the present invention;

[0008] FIG. 3 is a block diagram of the processing circuitry in accordance with another embodiment of the present invention; and

[0009] FIG. 4 is a block diagram of the processing circuitry in accordance with yet another embodiment of the present invention.

DETAILED DESCRIPTION

[0010] The detailed description of the appended drawings is intended as a description of the currently preferred embodiments of the present invention, and is not intended to represent the only form in which the present invention may be practiced. It is to be understood that the same or equivalent functions may be accomplished by different embodiments that are intended to be encompassed within the spirit and scope of the present invention.

[0011] In one embodiment, the present invention provides a system-on-chip (SoC) that is interfaced with at least one sensor. The sensor is an infrared (IR) imaging sensor or a light detection and ranging (LiDAR) sensor. The SoC includes driving circuitry, processing circuitry, and feedback circuitry. The driving circuitry is configured to provide at least one of a set of supply voltages, a set of reference voltages, a set of clock signals, and a set of timing signals to the sensor for driving the sensor. The processing circuitry is configured to receive at least one output signal generated by the sensor, and generate a processed output signal. The feedback circuitry is connected to the processing circuitry and the driving circuitry. The feedback circuitry is configured to receive the processed output signal and environmental data indicative of environmental conditions associated with the sensor, and generate a first set of control signals. The feedback circuitry is further configured to provide the first set of control signals to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor.

[0012] In another embodiment, the present invention provides a camera system. The camera system includes at least one sensor, monitoring circuitry that is configured to generate environmental data indicative of environmental conditions associated with the sensor, and a system-on-chip (SoC) that is interfaced with the sensor and the monitoring circuitry. The sensor is an infrared (IR) imaging sensor or a light detection and ranging (LiDAR) sensor.

The SoC includes driving circuitry, processing circuitry, and feedback circuitry. The driving circuitry is configured to provide at least one of a set of supply voltages, a set of reference voltages, a set of clock signals, and a set of timing signals to the sensor for driving the sensor. The processing circuitry is configured to receive at least one output signal generated by the sensor, and generate a processed output signal. The feedback circuitry is connected to the processing circuitry, the driving circuitry, and the monitoring circuitry. The feedback circuitry is configured to receive, from the processing circuitry and the monitoring circuitry, the processed output signal and the environmental data, respectively, and generate a first set of control signals based on the processed output signal and the environmental data. The feedback circuitry is further configured to provide the first set of control signals to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor.

[0013] Various embodiments of the present invention provide a camera system that includes at least one sensor, monitoring circuitry that generates environmental data indicative of environmental conditions associated with the sensor, and a system-on-chip (SoC) interfaced with the sensor and the monitoring circuitry. The sensor is an infrared (IR) imaging sensor or a light detection and ranging (LiDAR) sensor. The SoC includes driving circuitry that provides at least one of a set of supply voltages, a set of reference voltages, a set of clock signals, and a set of timing signals to the sensor for driving the sensor, and processing circuitry that processes at least one output signal generated by the sensor to generate a processed output signal. The driving circuitry includes a power supply generator, a reference voltage generator, a clock generator, and a timing signal generator for generating the set of supply voltages, the set of reference voltages, the set of clock signals, and the set of timing signals, respectively. The processing circuitry is sensor-specific, and a structure of the processing circuitry varies based on the sensor and a format of the output signal generated by the sensor.

[0014] The SoC further includes feedback circuitry that is connected to the processing circuitry, the driving circuitry, and the monitoring circuitry. The feedback circuitry receives, from the processing circuitry and the monitoring circuitry, the processed output signal and the environmental data, respectively, and generates first and second sets of control signals. The first set of control signals is provided to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor. Further, the second set of control signals is provided to the processing circuitry for dynamically configuring one or more operational parameters of the processing circuitry, thereby controlling the generation of the processed output signal. The SoC further includes a system controller that is connected to the driving circuitry, the processing circuitry, and the feedback circuitry for controlling the respective operations.

[0015] The SoC of the present invention thus is capable of being easily interfaced with both the IR imaging and LiDAR sensors having various operating characteristics such as resolutions, formats, operational modes, or the like. Thus, a need to use different SoCs for driving different types of sensors having different operating characteristics is eliminated. Further, the system controller executes additional processing operations on the processed output signal. Hence, a need for additional processing circuitries in the camera system is eliminated, thereby reducing size, weight, and cost of the camera system and power consumed by the camera system. Further, the use of the feedback circuitry ensures that the operation of the sensor is unaffected by variations in the environmental conditions associated with the sensor. Thus, the output signal, and consequently the processed output signal, are devoid of any irregularities caused due to the variations in the environmental conditions. The processed output signal generated by the SoC of the present invention is thus more accurate as compared to processed output signals generated by conventional SoCs.

[0016] FIG. 1 is a block diagram of a camera system 100 in accordance with an embodiment of the present invention. The camera system 100 may be utilized in automobile applications, medical applications, night vision applications, or the like. The camera system 100 includes a sensor 102, monitoring circuitry 104, and a system-on-chip (SoC) 106 interfaced with the sensor 102 and the monitoring circuitry 104.

[0017] The sensor 102 may be an infrared (IR) imaging sensor, a time-of-flight (ToF) light detection and ranging (LiDAR) sensor, or a frequency-modulated continuous-wave (FMCW) LiDAR sensor. Each of the IR imaging sensor, the ToF LiDAR sensor, and the FMCW LiDAR sensor may further be categorized based on an operational mode, a format, a resolution, or the like of the corresponding sensor. Examples of the operational mode of the sensor 102 include a global shutter mode, a rolling shutter high-resolution mode, a rolling shutter low-resolution binning mode, or the like, whereas examples of the format of the sensor 102 include a line format, an array format, or the like. Further, the resolution of the sensor 102 corresponds to a number of pixels in output signals generated by the sensor 102. Thus, in one example, the sensor 102 may be an IR imaging sensor that has a line format and a resolution of 1x1024 pixels, and that operates in a global shutter mode, and in another example, the sensor 102 may be an IR imaging sensor that has an array format and a resolution of 640x512 pixels, and that operates in a global shutter mode. The operational mode, format, resolution, or the like define operating characteristics of the sensor 102.

[0018] The sensor 102 receives multiple driving signals that drive the sensor 102 (z.e., controls an operation of the sensor 102). Based on the driving signals, the sensor 102 captures information associated with a scene in front of the sensor 102, and generates an output signal OS indicative of depth information of an object in the scene, image or video data associated with the captured scene, or the like. For example, when the sensor 102 is an IR imaging sensor, the output signal OS may be indicative of an IR image of the captured scene.

Similarly, when the sensor 102 is a LiDAR sensor (e.g., a ToF LiDAR sensor or an FMCW LiDAR sensor), the output signal OS may be indicative of the depth information of an object in the captured scene.

[0019] The monitoring circuitry 104 monitors environmental conditions associated with the sensor 102, and generates environmental data indicative of the environmental conditions. For example, the monitoring circuitry 104 may monitor a sensor temperature, an ambient temperature in a vicinity of the sensor 102, motion of a platform (not shown) on which the sensor 102 is installed, or the like. In an embodiment, the monitoring circuitry 104 includes a first temperature sensor (not shown) that measures the sensor temperature and generates first temperature data indicative of the measured sensor temperature. The monitoring circuitry 104 may further include a second temperature sensor (not shown) that measures the ambient temperature in the vicinity of the sensor 102, and generates second temperature data indicative of the measured ambient temperature. The monitoring circuitry 104 may further include a gyroscope (not shown) that monitors the motion of the platform on which the sensor 102 is installed, and generates motion data. For example, when the sensor 102 is a LiDAR sensor utilized in the automobile applications, the motion of the platform may correspond to motion of an automobile. The first temperature data, the second temperature data, and the motion data are collectively referred to as the environmental data.

[0020] Although FIG. 1 describes that the monitoring circuitry 104 includes the first and second temperature sensors and the gyroscope, the scope of the present invention is not limited to it. In various other embodiments, the monitoring circuitry 104 may include various other sensors or monitors for monitoring various other environmental parameters associated with the sensor 102, without deviating from the scope of the present invention.

[0021] The SoC 106 is interfaced with the sensor 102 for driving the sensor 102 and for processing the output signal OS generated by the sensor 102. The SoC 106 thus generates and provides the driving signals to the sensor 102 for driving the sensor 102, and receives the output signal OS generated by the sensor 102 for processing. The SoC 106 is further connected to the monitoring circuitry 104 for receiving the environmental data. The SoC 106 includes a first memory 108, a second memory 110, a system controller 112, driving circuitry 114, processing circuitry 116, a third memory 118, a media interface 120, a communication interface 122, and feedback circuitry 124. The first memory 108, the second memory 110, the system controller 112, the driving circuitry 114, the processing circuitry 116, the third memory 118, the media interface 120, the communication interface 122, and the feedback circuitry 124 communicate with each other by way of a communication bus 126.

[0022] The first memory 108 includes suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for storing correction data associated with the processing of the output signal OS and configuration data associated with configuration of various components of the SoC 106. In an embodiment, the correction data correspond to pixel level correction data. The first memory 108 may be a non-volatile memory, and the correction and

configuration data may thus be stored in the first memory 108 during manufacturing of the first memory 108. The first memory 108 may be associated with a first memory controller 128 for controlling write and read operations associated with the first memory 108. Examples of the first memory 108 may include, but are not limited to, a read-only memory (ROM), a removable storage drive, a hard-disk drive (HDD), a flash memory, and a solid-state memory. [0023] The second memory 110 includes suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for storing the correction and configuration data upon power-up of the SoC 106. The second memory 110 may be associated with a second memory controller 130 for controlling write and read operations associated with the second memory 110. Thus, upon the power-up of the SoC 106, the correction and configuration data from the first memory 108 are transferred to the second memory 110, by the system controller 112 by way of the first and second memory controllers 128 and 130, to facilitate various real-time operations of the SoC 106. In an embodiment, the second memory 110 is a high-speed volatile memory. Examples of the second memory 110 may include, but are not limited to, a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like.

[0024] It will be apparent to a person skilled in the art that the scope of the present invention is not limited to realizing the first and second memories 108 and 110 in the SoC 106, as described herein. In various other embodiments, one or both of the first and second memories 108 and 110 may be realized external to the SoC 106, without departing from the scope of the present invention. In such scenarios, the first and second memory controllers 128 and 130 may remain internal to the SoC 106 and facilitate communication with the first and second memories 108 and 110, respectively.

[0025] The system controller 112 includes suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for controlling operations of the driving circuitry 114, the processing circuitry 116, the media interface 120, the communication interface 122, and the feedback circuitry 124. Upon powering-up the SoC 106, the system controller 112 transfers the correction and configuration data from the first memory 108 to the second memory 110. Further, the system controller 112 configures, based on the configuration data, the driving circuitry 114, the processing circuitry 116, the media interface 120, the communication interface 122, and the feedback circuitry 124. The configuration may include a read operation to read the configuration data from the second memory 110, and a write operation to write configuration data corresponding to the driving circuitry 114, the processing circuitry 116, the media interface 120, the communication interface 122, and the feedback circuitry 124 to corresponding configuration memories (not shown). Examples of the system controller 112 include a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or the like.

[0026] Although FIG. 1 illustrates that the system controller 112 is internal to the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the system controller 112 may be external to the SoC 106, without deviating from the scope of the present invention. In such a scenario, the system controller 112 may be implemented as an application specific integrated circuit (ASIC) processor, a Field Programmable Gate Array (FPGA), or the like.

[0027] The driving circuitry 114 generates the driving signals that drive the sensor 102. The driving circuitry 114 includes a power supply generator 132, a reference voltage generator 134, a clock generator 136, and a timing signal generator 138.

[0028] The power supply generator 132 generates, based on the configuration data stored in the corresponding configuration memory, a first supply voltage Vsl for driving the sensor 102. Additionally, the power supply generator 132 may generate a second supply voltage Vs2 to power various components of the SoC 106. The power supply generator 132 may include one or more linear regulators, one or more switching regulators, or a combination thereof to generate the first and second supply voltages Vsl and Vs2. Alternatively, the power supply generator 132 may include one or more charge pumps to generate the first and second supply voltages Vsl and Vs2. In an example, the first and second supply voltages Vsl and Vs2 may have voltage levels in a range of 0.9 volts (V) to 3.3V and load current capacities in a range of 0 milliamperes (mA) to 100mA.

[0029] Although FIG. 1 describes that the power supply generator 132 generates a single supply voltage (i.e., the first supply voltage Vsl) for driving the sensor 102 and a single supply voltage (i.e., the second supply voltage Vs2) for powering various components of the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the power supply generator 132 may generate multiple supply voltages for driving the sensor 102 and for powering various components of the SoC 106, without deviating from the scope of the present invention. In such a scenario, the supply voltages may have different voltage levels and load current capacities.

[0030] The reference voltage generator 134 generates, based on the configuration data stored in the corresponding configuration memory, a first reference voltage Vrl for driving the sensor 102. Additionally, the reference voltage generator 134 may generate a second reference voltage Vr2 for providing to various components of the SoC 106 to facilitate various operations of the SoC 106. The reference voltage generator 134 may include a bandgap circuit (not shown) for generating the first and second reference voltages Vrl and Vr2. In an example, the first and second reference voltages Vrl and Vr2 may have voltage levels in a range of 0. IV to 3 V and load current capacities in a range of 0mA to 50mA. [0031] Although FIG. 1 describes that the reference voltage generator 134 generates a single reference voltage (i.e., the first reference voltage Vrl) for driving the sensor 102 and a single reference voltage (i.e., the second reference voltage Vr2) for providing to various components of the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the reference voltage generator 134 may generate multiple reference voltages for driving the sensor 102 and for providing to various components of the SoC 106, without deviating from the scope of the present invention. In such a scenario, the reference voltages may have different voltage levels and load current capacities.

[0032] The clock generator 136 generates, based on the configuration data stored in the corresponding configuration memory, a first clock signal CLK1 for driving the sensor 102. Additionally, the clock generator 136 may generate a second clock signal CLK2 for providing to various components of the SoC 106 to maintain synchronization between the SoC 106 and the sensor 102. In one embodiment, the clock generator 136 may include a phase-locked loop (PLL) (not shown) for generating the first and second clock signals CLK1 and CLK2. The PLL generates the first and second clock signals CLK1 and CLK2 based on a reference clock signal generated by a reference oscillator (not shown) that may be internal or external to the SoC 106. In an example, frequencies of the first and second clock signals CLK1 and CLK2 may be in a range of 1 megahertz (MHz) to 700MHz.

[0033] Although FIG. 1 describes that the clock generator 136 generates a single clock signal (i.e., the first clock signal CLK1) for driving the sensor 102 and a single clock signal (i.e., the second clock signal CLK2) for providing to various components of the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the clock generator 136 may generate multiple clock signals for driving the sensor 102 and for providing to various components of the SoC 106, without deviating from the scope of the present invention. In such a scenario, the clock signals may have different frequencies. For example, a clock signal provided to the processing circuitry 116 may have a frequency in a range of 80MHz to 160MHz and a clock signal provided to the communication interface 122 may have a frequency in a range of 1MHz to 10MHz.

[0034] The timing signal generator 138 is connected to the clock generator 136 for receiving the first clock signal CLK1. Based on the first clock signal CLK1 and the configuration data stored in the corresponding configuration memory, the timing signal generator 138 generates a first timing signal T1 for driving the sensor 102. The first timing signal T1 may correspond to non-uniformity correction (NUC) data for facilitating execution of a real-time NUC operation by the sensor 102, a shutter control signal for controlling a shutter operation of the sensor 102 (e.g., the IR imaging sensor), a beam steering signal for steering a laser beam of the sensor 102 (e.g., the LiDAR sensor) across a field-of-view (FoV) of the sensor 102, or the like. Additionally, the timing signal generator 138 may receive the second clock signal CLK2 from the clock generator 136 and generate a second timing signal CLK2 for providing to various components of the SoC 106 to control timing of various components of the SoC 106.

[0035] Although FIG. 1 describes that the timing signal generator 138 generates a single timing signal (i.e., the first timing signal Tl) for driving the sensor 102 and a single timing signal (i.e., the second timing signal T2) for providing to various components of the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the timing signal generator 138 may generate multiple timing signals for driving the sensor 102 and for providing to various components of the SoC 106, without deviating from the scope of the present invention. In such a scenario, the timing signals may have different pulse widths, polarity, timing margins, repetition rate, synchronicity, or the like.

[0036] The first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl are collectively referred to as the driving signals that drive the sensor 102. The driving signals may be sensor-specific. In other words, the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving an IR imaging sensor may be different than the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving a LiDAR sensor. Similarly, the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving a ToF LiDAR sensor may be different than the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving an FMCW LiDAR sensor. The driving signals may further be generated based on the operating characteristics of the sensor 102. For example, the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving an IR imaging sensor having a first resolution may be different than the first supply voltage Vsl, the first reference voltage Vrl, the first clock signal CLK1, and the first timing signal Tl generated for driving an IR imaging sensor having a second resolution that is different than the first resolution.

[0037] Although FIG. 1 illustrates that the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138 are internal to the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138 may be external to the SoC 106, without deviating from the scope of the present invention. In such a scenario, the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138 may be implemented as discrete printed circuit boards (PCBs) that are interfaced with the sensor 102, or may be implemented in a Programmable Logic Device (PLD) or an FPGA interfaced with the sensor 102.

[0038] The processing circuitry 116 is connected to the sensor 102 for receiving the output signal OS, and is further connected to the second memory 110 for retrieving, by way of the second memory controller 130, the correction data stored therein. The correction data may then be stored in the third memory 118 for facilitating various operations of the processing circuitry 116. The processing circuitry 116 executes a set of processing operations on the output signal OS, based on the correction data stored in the third memory 118 and the configuration data stored in the corresponding configuration memory, to generate a processed output signal POS. The processed output signal POS may be indicative of image or video data associated with the captured scene and/or depth information of an object in the scene.

[0039] A structure of the processing circuitry 116 may vary based on the type of the sensor 102. For example, the structure of the processing circuitry 116 for processing the output signal OS generated by an IR imaging sensor may be different than the structure of the processing circuitry 116 for processing the output signal OS generated by a LiDAR sensor. Similarly, the structure of the processing circuitry 116 for processing the output signal OS generated by a ToF LiDAR sensor may be different than the structure of the processing circuitry 116 for processing the output signal OS generated by an FMCW LiDAR sensor. The structure of the processing circuitry 116 may further vary based on the operating

characteristics of the sensor 102. For example, the structure of the processing circuitry 116 for processing the output signal OS generated by an IR imaging sensor having the first resolution may be different than the structure of the processing circuitry 116 for processing the output signal OS generated by an IR imaging sensor having the second resolution. The structure of the processing circuitry 116 may further vary based on a format of the output signal OS. For example, the structure of the processing circuitry 116 for processing the output signal OS that is in an analog format may be different than the structure of the processing circuitry 116 for processing the output signal OS that is in a digital format. The processing circuitry 116 is explained in detail in FIGS. 2-4. [0040] It will be apparent to a person skilled in the art that the scope of the invention is not limited to realizing the processing circuitry 116 and the system controller 112 as separate entities. In various other embodiments, the functionalities of the processing circuitry 116 may be integrated into the system controller 112, without departing from the scope of the present invention.

[0041] The third memory 118 includes suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for storing the correction data retrieved from the second memory 110. The third memory 118 may correspond to a memory working in conjunction with the processing circuitry 116 and stores the correction data to facilitate the execution of the set of processing operations. The third memory 118 may be a volatile memory. Examples of the third memory 118 may include a DRAM, a SRAM, or the like.

[0042] The media interface 120 enables the processing circuitry 116 to communicate with a media device (not shown) that is external to the SoC 106. In an example, the media interface 120 enables the processing circuitry 116 to communicate the processed output signal POS to the media device. The media device may then display the processed output signal POS for visualization and inspection.

[0043] Although FIG. 1 illustrates that the processing circuitry 116 is connected to the media interface 120 by way of the communication bus 126 for communicating the processed output signal POS, the scope of the present invention is not limited to it. In various other embodiments, the media interface 120 may be connected to the processing circuitry 116 by way of a first dedicated interface (not shown) for receiving the processed output signal POS, without deviating from the scope of the present invention.

[0044] The communication interface 122 enables the SoC 106 to communicate with components that are external to the SoC 106 (e.g., a central processing unit (CPU) (not shown) of the camera system 100). By way of the communication interface 122, the CPU may store the configuration and correction data in the first memory 108, access and update the stored configuration and correction data, control operations of the SoC 106, or the like. The communication interface 122 further enables the processing circuitry 116 to

communicate the processed output signal POS to the CPU. The CPU may utilize the processed output signal POS for executing various real-time operations of the camera system 100. Examples of the communication interface 122 may include a serial-peripheral interface (SPI), an inter-IC communication interface (I2C), or the like.

[0045] Although FIG. 1 describes that the processed output signal POS is directly communicated to the media device or the CPU, the scope of the present invention is not limited to it. In various other embodiments, the processed output signal POS may be provided to the system controller 112 for further processing, prior to communicating to the media device and the CPU. In such a scenario, the system controller 112 may execute various machine learning and artificial intelligence operations on the processed output signal POS.

An output signal generated by the system controller 112 may then be communicated to the media device and the CPU by way of the media and communication interfaces 120 and 122, respectively.

[0046] The feedback circuitry 124 is connected to the processing circuitry 116 for receiving the processed output signal POS, and to the monitoring circuitry 104 for receiving the environmental data. Based on the processed output signal POS, the environmental data, and the configuration data stored in the corresponding configuration memory, the feedback circuitry 124 generates first through fifth control signals CS1-CS5 for dynamically configuring various operational parameters of the processing circuitry 116, the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138, respectively. In other words, the configuration data stored in the configuration memories of the processing circuitry 116, the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138 is dynamically updated based on the first through fifth control signals CS1-CS5, respectively. The first through fifth control signals CS1-CS5 may further be generated based on various imaging conditions associated with the scene. Examples of the imaging conditions include environmental conditions of the scene (such as temperature, illumination, or the like), direction of motion and speed of the one or more objects in the scene, or the like. In an embodiment, the imaging conditions may be determined by the processing circuitry 116 based on multiple output signals (e.g., the output signal OS) received from the sensor 102 over a period of time.

[0047] The feedback circuitry 124 controls the operation of the sensor 102 by

dynamically configuring, based on the second through fifth control signals CS2-CS5, the operational parameters of the power supply generator 132, the reference voltage generator 134, the clock generator 136, and the timing signal generator 138, respectively. Similarly, the feedback circuitry 124 controls the generation of the processed output signal POS by dynamically configuring the operational parameters of the processing circuitry 116 based on the first control signal CS1.

[0048] The dynamic configuration of the operational parameters of the driving circuitry 114 may result in changes in various characteristics of the driving signals. For example, the dynamic configuration of the operational parameters of the power supply generator 132 may result in a change in the voltage level of the first supply voltage Vsl. Similarly, the dynamic configuration of the operational parameters of the reference voltage generator 134 may result in a change in the voltage level of the first reference voltage Vrl. Further, the dynamic configuration of the operational parameters of the processing circuitry 116 may result in changes in various characteristics the processed output signal POS. Additionally, the feedback circuitry 124 may generate various other control signals (not shown) for

dynamically configuring various other components of the SoC 106. In an embodiment, the feedback circuitry 124 corresponds to a digital signal processor. In another embodiment, the feedback circuitry 124 corresponds to an artificial intelligence processor.

[0049] Although FIG. 1 illustrates that the processing circuitry 116 and the feedback circuitry 124 are connected by way of the communication bus 126, the scope of the present invention is not limited to it. In various other embodiments, the processing circuitry 116 and the feedback circuitry 124 may be connected by way of a second dedicated interface (not shown), without deviating from the scope of the present invention.

[0050] It will be apparent to a person skilled in the art that the scope of the present invention is not limited to realizing the feedback circuitry 124 and the system controller 112 as separate entities. In various other embodiments, the functionalities of the feedback circuitry 124 may be integrated into the system controller 112, without departing from the scope of the present invention.

[0051] Although FIG. 1 describes that the sensor 102 generates a single output signal (i.e., the output signal OS), it will be apparent to a person skilled in the art that the scope of the present invention is not limited to it. In various other embodiments, the sensor 102 may generate multiple output signals, without deviating from the scope of the present invention. In such a scenario, the SoC 106 may include multiple processing circuitries to facilitate simultaneous processing of the output signals. Further, each output signal may be processed by the SoC 106 in a manner similar to the processing of the output signal OS as described above.

[0052] It will be apparent to a person skilled in the art that although FIG. 1 describes that the camera system 100 includes a single sensor (i.e., the sensor 102), the scope of the present invention is not limited to it. In various other embodiments, the camera system 100 may include multiple sensors interfaced with the SoC 106, without deviating from the scope of the present invention. In such a scenario, each sensor is driven by the SoC 106 in a manner similar to the driving of the sensor 102 as described above. Further, the SoC 106 may include multiple processing circuitries to facilitate simultaneous processing of multiple output signals generated by multiple sensors. Each output signal may be processed by the SoC 106 in a manner similar to the processing of the output signal OS as described above.

[0053] FIG. 2 is a block diagram of the processing circuitry 116 in accordance with an embodiment of the present invention. The processing circuitry 116 illustrated in FIG. 2 is utilized for processing the output of an IR imaging sensor that is in an analog format. In such a scenario, the processing circuitry 116 includes first analog processing circuitry (APC) 202, a first analog-to-digital converter (ADC) 204, and first digital processing circuitry (DPC)

206.

[0054] The first APC 202 is connected to the sensor 102 for receiving the output signal OS. The first APC 202 executes a first analog processing operation on the output signal OS to generate a first intermediate analog signal IASI. Examples of the first analog processing operation may include a buffering operation, a black-level adjustment operation, a single-to- differential conversion operation, a correlated-double sampling operation, a range matching operation, or the like. The first APC 202 may include continuous-time processing circuitry or discrete-time processing circuitry to execute the first analog processing operation. It will be apparent to a person skilled in the art that the first APC 202 executes the first analog processing operation based on the configuration data stored in the corresponding

configuration memory.

[0055] The first ADC 204 is connected the first APC 202 for receiving the first intermediate analog signal IASI. The first ADC 204 converts the first intermediate analog signal IASI that is in an analog format to a first intermediate digital signal IDS1 that is in a digital format. Various operational parameters of the first ADC 204 (such as a resolution, an effective number of bits (ENOB), conversion time, or the like) vary based on the operating characteristics of the sensor 102 (/. ., the operating characteristics of the IR imaging sensor) and are configurable. The operational parameters of the first ADC 204 may be configured based on the configuration data stored in the corresponding configuration memory.

[0056] When a rate associated with the output signal OS is different than operational rates of the first APC 202 and the first ADC 204, the first APC 202 and the first ADC 204 may be time-multiplexed or the processing circuitry 116 may further include intermediate analog buffer memories (not shown) to ensure that the rate associated with the output signal OS is same as the operational rates of the first APC 202 and the first ADC 204.

[0057] The first DPC 206 is connected to the first ADC 204 for receiving the first intermediate digital signal IDSl, and may retrieve the correction data stored in the third memory 118. The first DPC 206 executes a first set of digital processing operations in sequence on the first intermediate digital signal IDS1 to generate the processed output signal POS. It will be apparent to a person skilled in the art that the first DPC 206 executes the first set of digital processing operations based on the configuration data stored in the

corresponding configuration memory.

[0058] The first set of digital processing operations includes a first pixel level operation, a first kernel level operation, and a first fixed function operation. The first DPC 206 includes first pixel processing circuitry (PPC) 208, kernel processing circuitry (KPC) 210, and first fixed function processing circuitry (FFPC) 212 for executing the first pixel level operation, the first kernel level operation, and the first fixed function operation, respectively. The first DPC 206 may further include an in-line buffer memory 214 connected to the KPC 210 for facilitating the first kernel level operation. Examples of the first DPC 206 may include an ASIC processor, a RISC processor, a CISC processor, or the like.

[0059] The first PPC 208 receives the first intermediate digital signal IDSl, and executes the first pixel level operation on the first intermediate digital signal IDSl to generate a second intermediate digital signal IDS2. Examples of the first pixel level operation may include an offset correction operation, a gain correction operation, an NUC correction operation, or the like. The first PPC 208 executes the first pixel level operation based on the correction data (i.e., the pixel level correction data) retrieved from the third memory 118. In an embodiment, the first PPC 208 may include a first local register (not shown) for storing the pixel level correction data.

[0060] The KPC 210 is connected to the first PPC 208 for receiving the second intermediate digital signal IDS2. The KPC 210 executes the first kernel level operation on the second intermediate digital signal IDS2 to generate a third intermediate digital signal IDS3. A kernel of the second intermediate digital signal IDS2 is generated in the in-line buffer memory 214 to facilitate the execution of the first kernel level operation. Examples of the first kernel level operation may include a kernel-based filtering operation, an image sharpening operation, a bad pixel replacement operation, or the like.

[0061] The in-line buffer memory 214 includes suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for storing multiple lines of the second intermediate digital signal IDS2 to generate the kernel. For example, the in-line buffer memory 214 may store five lines of the second intermediate digital signal IDS2 to facilitate generation of a 5x5 kernel. [0062] The first FFPC 212 is connected to the KPC 210 for receiving the third intermediate digital signal IDS3. The first FFPC 212 executes the first fixed function operation on the third intermediate digital signal IDS3 to generate the processed output signal POS. Examples of the first fixed function operation include a histogram equalization operation, an auto gain control operation, a non-linear spatial filtering operation, or the like.

[0063] The first intermediate digital signal IDSl, the second intermediate digital signal IDS2, the third intermediate digital signal IDS3, and the processed output signal POS are stored in the third memory 118. The first PPC 208 may then utilize at least one of the first intermediate digital signal IDSl, the second intermediate digital signal IDS2, the third intermediate digital signal IDS3, and the processed output signal POS as the correction data for executing the first pixel level operation, thereby providing a temporal pixel level correction of the first intermediate digital signal IDSl.

[0064] It will be apparent to a person skilled in the art that the structure of the first DPC 206 as illustrated in FIG. 2 (i.e., the first PPC 208, the KPC 210, and the first FFPC 212 connected in series) is one embodiment of the first DPC 206 and that the scope of the present invention is not limited to it. In various other embodiments, an order of connection between the first PPC 208, the KPC 210, and the first FFPC 212 may vary, without deviating from the scope of the present invention. Additionally, one or more of the first PPC 208, the KPC 210, and the first FFPC 212 may be excluded from the first DPC 206, without deviating from the scope of the present invention.

[0065] It will further be apparent to a person skilled in the art that the scope of the present invention is not limited to the first DPC 206 including a single PPC (i.e., the first PPC 208), a single KPC (i.e., the KPC 210), and a single FFPC (i.e., the first FFPC 212) that execute a single pixel level operation, a single kernel level operation, and a single fixed function operation, respectively. In various other embodiments, the first DPC 206 may include multiple first PPCs 208 connected in series, multiple KPCs 210 connected in series, and multiple FFPCs 212 connected in series for executing multiple pixel level, kernel level, and fixed function operations, respectively, without deviating from the scope of the present invention. Conversely, the first PPC 208, the KPC 210, and the first FFPC 212 may be configured to execute multiple pixel level, kernel level, and fixed function operations, respectively, without deviating from the scope of the present invention.

[0066] Although FIG. 2 describes the processing of a single output signal (i.e., the output signal OS) received by the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, when multiple output signals are received by the SoC 106, the processing circuitry 116 of FIG. 2 may include multiple first APCs 202, multiple first ADCs 204, and multiple first DPCs 206 to facilitate processing of multiple output signals, without deviating from the scope of the present invention.

[0067] The foregoing description of FIG. 2 describes the processing of an analog output signal OS generated by an IR imaging sensor. When the output signal OS generated by the IR imaging sensor is in a digital format, the operations performed by the processing circuitry 116 to generate the processed output signal POS is similar to the operations performed by the first DPC 206. In other words, the processing circuitry 116 corresponds to the first DPC 206 (z.e., the processing circuitry 116 may not include the first APC 202 and the first ADC 204), and the output signal OS corresponds to the first intermediate digital signal IDSl. In such a scenario, the first APC 202 and the first ADC 204 may be external to the SoC 106 and may be associated with the sensor 102 to generate the output signal OS in a digital format.

[0068] FIG. 3 is a block diagram of the processing circuitry 116 in accordance with another embodiment of the present invention. The processing circuitry 116 illustrated in FIG. 3 is utilized for processing the output (z.e., the output signal OS) of an FMCW LiDAR sensor that is in an analog format.

[0069] The FMCW LiDAR sensor includes a first laser (not shown) that emits a frequency modulated signal (z.e., a first laser signal) (not shown) across horizontal and vertical FoVs of the FMCW LiDAR sensor, and a coherent receiver (not shown) that receives a first reflected laser signal (not shown). The coherent receiver, in conjunction with a transimpedance amplifier (not shown) installed on the FMCW LiDAR sensor, generates the output signal OS in an analog format based on the first reflected laser signal and a reference laser signal (not shown) associated with the first laser. In such a scenario, the processing circuitry 116 includes a second APC 302, a second ADC 304, and a second DPC 306.

[0070] The second APC 302 is connected to the sensor 102 for receiving the output signal OS, and executes a second analog processing operation on the output signal OS to generate a second intermediate analog signal IAS2. Examples of the second analog processing operation may include a buffering operation, a black-level adjustment operation, a single-to-differential conversion operation, a correlated-double sampling operation, a range matching operation, or the like. The second APC 302 may include continuous-time processing circuitry or discrete-time processing circuitry to execute the second analog processing operation. It will be apparent to a person skilled in the art that the second APC 302 executes the second analog processing operation based on the configuration data stored in the corresponding configuration memory. [0071] The second ADC 304 is connected the second APC 302 for receiving the second intermediate analog signal IAS2, and converts the second intermediate analog signal IAS2 that is in an analog format to a fourth intermediate digital signal IDS4 that is in a digital format. In one embodiment, the second ADC 304 is structurally similar to the first ADC 204 of FIG. 2. In another embodiment, the first ADC 204 and the second ADC 304 are structurally different.

[0072] The second DPC 306 is connected to the second ADC 304 for receiving the fourth intermediate digital signal IDS4, and may retrieve the correction data stored in the third memory 118. The second DPC 306 executes a second set of digital processing operations in sequence on the fourth intermediate digital signal IDS4 to generate the processed output signal POS. It will be apparent to a person skilled in the art that the second DPC 306 executes the second set of digital processing operations based on the configuration data stored in the corresponding configuration memory.

[0073] The second set of digital processing operations include a second pixel level operation, a second fixed function operation, and a third pixel level operation. The second DPC 306 includes a second PPC 308, a second FFPC 310, and a third PPC 312 for executing the second pixel level operation, the second fixed function operation, and the third pixel level operation, respectively. Examples of the second DPC 306 may include an ASIC processor, a RISC processor, a CISC processor, or the like.

[0074] The second PPC 308 receives the fourth intermediate digital signal IDS4, and executes the second pixel level operation on the fourth intermediate digital signal IDS4 to generate a fifth intermediate digital signal IDS5. In an example, the second pixel level operation corresponds to an offset correction operation. The second PPC 308 executes the second pixel level operation based on the pixel level correction data retrieved from the third memory 118. In an embodiment, the second PPC 308 may include a second local register (not shown) for storing the pixel level correction data.

[0075] The second FFPC 310 is connected to the second PPC 308 for receiving the fifth intermediate digital signal IDS5, and executes the second fixed function operation on the fifth intermediate digital signal IDS5 to generate a sixth intermediate digital signal IDS6. In an example, the second fixed function operation corresponds to a fast Fourier transform operation. Further, the sixth intermediate digital signal IDS6 corresponds to a beat frequency corresponding to the depth information of the object in the scene captured by the FMCW LiDAR sensor. [0076] The sixth intermediate digital signal IDS6 {i.e., the calculated depth) may have static and temporal variations resulting from variations in the sensor and ambient

temperatures, a sensor fixed pattern noise, laser chirp variations, or the like. To correct the variations in the sixth intermediate digital signal IDS6, the third PPC 312 is utilized. The third PPC 312 is connected to the second FFPC 310 for receiving the sixth intermediate digital signal IDS6, and executes the third pixel level operation on the sixth intermediate digital signal IDS6 to generate the processed output signal POS. Examples of the third pixel level operation include an offset correction operation, a gain correction operation, an NUC correction operation, or the like. The third PPC 312 executes the third pixel level operation based on the pixel level correction data retrieved from the third memory 118. In an embodiment, the third PPC 312 may include a third local register (not shown) for storing the pixel level correction data.

[0077] The fourth intermediate digital signal IDS4, the fifth intermediate digital signal IDS5, the sixth intermediate digital signal IDS6, and the processed output signal POS are stored in the third memory 118. The second and third PPCs 308 and 312 may then utilize at least one of the fourth intermediate signal IDS4, the fifth intermediate digital signal IDS5, the sixth intermediate digital signal IDS6, and the processed output signal POS as the correction data for executing the second and third pixel level operations, thereby providing temporal pixel level corrections of the fourth and sixth intermediate digital signals IDS4 and IDS6, respectively.

[0078] It will be apparent to a person skilled in the art that the structure of the second DPC 306 as illustrated in FIG. 3 ( i.e ., the second PPC 308, the second FFPC 310, and the third PPC 312 connected in series) is one embodiment of the second DPC 306 and that the scope of the present invention is not limited to it. In various other embodiments, an order of connection between the second PPC 308, the second FFPC 310, and the third PPC 312 may vary, without deviating from the scope of the present invention. Similarly, a number of the second PPCs 308, the second FFPCs 310, and the third PPCs 312 may vary, without deviating from the scope of the present invention.

[0079] Although FIG. 3 describes the processing of a single output signal {i.e., the output signal OS) received by the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, when multiple output signals are received by the SoC 106, the processing circuitry 116 of FIG. 3 may include multiple second APCs 302, multiple second ADCs 304, and multiple second DPCs 306 to facilitate processing of multiple output signals, without deviating from the scope of the present invention. [0080] The processing circuitry 116 of FIG. 3 may be utilized in applications

implementing chip-scale FMCW LiDAR sensors (e.g., automobile applications). As a chip- scale FMCW LiDAR sensor has a finite FOV (i.e., less than 360°), multiple chip-scale LiDAR sensors covering different directions are installed to achieve a 360° FoV. The processing circuitry 116 utilized in such a scenario receives multiple output signals (i.e., depth information of one or more objects in the scene) from multiple chip-scale FMCW LiDAR sensors, respectively, and generates the processed output signal POS. The second set of digital processing operations in such a scenario may further include FoV-stitching operations and edge-processing operations. Further, the processed output signal POS is provided to a CPU of the automobile for executing various real-time operations of the automobile.

[0081] The foregoing description of FIG. 3 describes the processing of an analog output signal OS generated by an FMCW LiDAR sensor. When the output signal OS generated by the FMCW LiDAR sensor is in a digital format, the operations performed by the processing circuitry 116 to generate the processed output signal POS is similar to the operations performed by the second DPC 306. In other words, the processing circuitry 116 corresponds to the second DPC 306 (i.e., the processing circuitry 116 may not include the second APC 302 and the second ADC 304), and the output signal OS corresponds to the fourth

intermediate digital signal IDS4. In such a scenario, the second APC 302 and the second ADC 304 may be external to the SoC 106 and may be associated with the sensor 102 to generate the output signal OS in a digital format.

[0082] FIG. 4 is a block diagram of the processing circuitry 116 in accordance with yet another embodiment of the present invention. The processing circuitry 116 illustrated in FIG. 4 is utilized for processing the output (i.e., the output signal OS) of a ToF LiDAR sensor that is in an analog format.

[0083] The ToF LiDAR sensor includes a second laser (not shown) and a Micro-Electro- Mechanical System (MEMS) mirror (not shown). The second laser impinges a second laser signal (not shown) on the rotating MEMS mirror and is directed, by the MEMS mirror, in various directions and at different angles in the FoV of the ToF LiDAR sensor. The ToF LiDAR sensor further includes a photodiode (not shown) that captures the second laser signal that is reflected from an object (i.e., a second reflected laser signal) (not shown). Examples of the photodiode may include a PIN diode, an avalanche photodiode with linear gain, a silicon photomultiplier of Geiger-mode photodiodes array, a single-photon avalanche diode, or the like. It will be apparent to a person skilled in the art that the scope of the present invention is not limited to the ToF LiDAR sensor including a single pair of laser and photodiode. In various other embodiments, the ToF LiDAR sensor may include multiple laser-photodiode pairs in order to reduce the time required to scan the FoV of the ToF LiDAR sensor.

[0084] The second reflected laser signal is the output signal OS of the ToF LiDAR sensor and is in an analog format. In such a scenario, the processing circuitry 116, as illustrated in FIG. 4, includes ToF processing circuitry 402 and a third DPC 404.

[0085] The ToF processing circuitry 402 is connected to the sensor 102 for receiving the output signal OS, and executes a set of ToF processing operations in sequence on the output signal OS to generate a ToF digital signal TDS. The set of ToF processing operations includes a current to voltage conversion operation, a threshold voltage comparison operation, and a time-to-digital-conversion operation. In an embodiment, the ToF processing circuitry 402 may include a current-to-voltage converter (not shown) that receives the output signal OS and converts the output signal OS to a first voltage signal (not shown). The ToF processing circuitry 402 may further include a threshold voltage comparator (not shown) that is connected to the current-to-voltage converter for receiving the first voltage signal, and compares the first voltage signal with a set of threshold voltages to generate a comparator signal (not shown). The ToF processing circuitry 402 may further include a time-to-digital converter (not shown) connected to the threshold voltage comparator for receiving the comparator signal, and generates, based on the comparator signal, the ToF digital signal TDS. It will be apparent to a person skilled in the art that the ToF processing circuitry 402 executes the set of ToF processing operations based on the configuration data stored in the

corresponding configuration memory.

[0086] The ToF digital signal TDS may have variations arising from fixed offsets in the threshold voltage comparator, the time-to-digital converter, or the like. To correct the variations in the ToF digital signal TDS, the third DPC 404 may be utilized. The third DPC 404 is connected to the ToF processing circuitry 402 for receiving the ToF digital signal TDS, and executes a fourth pixel level operation on the ToF digital signal TDS to generate the processed output signal POS. It will be apparent to a person skilled in the art that the third DPC 404 executes the fourth pixel level operation based on the configuration data stored in the corresponding configuration memory. Examples of the fourth pixel level operation may include an offset correction operation, a gain correction operation, an NUC correction operation, or the like. The third DPC 404 executes the fourth pixel level operation based on the pixel level correction data retrieved by the third DPC 404 from the third memory 118. In an embodiment, the third DPC 404 may include a fourth local register (not shown) for storing the pixel level correction data.

[0087] The ToF digital signal TDS and the processed output signal POS are stored in the third memory 118. The third DPC 404 may utilize at least one of the ToF digital signal TDS and the processed output signal POS as the correction data for executing the fourth pixel level operation, thereby providing a temporal pixel level correction of the ToF digital signal TDS.

[0088] Although FIG. 4 describes the processing of a single output signal (i.e., the output signal OS) received by the SoC 106, the scope of the present invention is not limited to it. In various other embodiments, when multiple output signals are received by the SoC 106, the processing circuitry 116 of FIG. 4 may include multiple ToF processing circuitries 402 and multiple third DPCs 404 to facilitate processing of multiple output signals, without deviating from the scope of the present invention.

[0089] The foregoing description of FIG. 4 describes the processing of an analog output signal OS generated by a ToF LiDAR sensor. When the output signal OS generated by the ToF LiDAR sensor is in a digital format, the structure of and operations performed by the processing circuitry 116 may vary, without deviating from the scope of the present invention. In an embodiment, the ToF processing circuitry 402 may execute a time-to-digital conversion operation, instead of the set of ToF processing operations in sequence, on the output signal OS to generate the ToF digital signal TDS.

[0090] When the ToF LiDAR sensor is a phase modulated continuous wave two- dimensional array sensor, the structure of and the operations performed by the processing circuitry 116 to process the output signal OS generated in an analog or digital format may be similar to the corresponding structure of and the operations performed by the processing circuitry 116 of FIG. 2.

[0091] The SoC 106 of the present invention thus is capable of being easily interfaced (for driving and output signal processing) with the sensor 102 that may be an IR imaging sensor, a ToF LiDAR sensor, or an FMCW LiDAR sensor having various operating characteristics. The configuration data stored in the first memory 108 corresponds to the type and operating characteristics of the sensor 102. Thus, the same SoC 106 may be utilized for interfacing with both the IR imaging sensor and the LiDAR sensor by modifying the configuration data stored in the first memory 108. The use of the SoC 106 thus eliminates a need to use different SoCs for driving different types of sensors having different operating characteristics and processing output signals generated by such sensors. Further, as the system controller 112 executes additional processing operations on the processed output signal POS generated by the processing circuitry 116, a need for additional processing circuitries (not shown) in the camera system 100 is eliminated, thereby reducing size, weight, and cost of the camera system 100 and power consumed by the camera system 100. Further, the use of the feedback circuitry 124 ensures that the SoC 106 provides a dynamic approach to counter variations in the environmental conditions associated with the sensor 102, and that the operation of the sensor 102 is unaffected by the variations. Thus, the output signal OS, and consequently the processed output signal POS, are devoid of irregularities caused due to the variations in the environmental conditions. Additionally, the input to a DPC of the processing circuitry 116 (e.g., the first through third DPCs 206, 306, and 404) and output of each component of the DPC are stored in the third memory 118 and utilized as the correction data for pixel level operations (e.g., the first through fourth pixel level operations). The processed output signal POS generated by the processing circuitry 116 is thus more accurate as compared to processed output signals generated by processing circuitries of conventional SoCs.

[0092] While various embodiments of the present invention have been illustrated and described, it will be clear that the present invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the present invention, as described in the claims.

Claims

1. A system-on-chip (SoC) interfaced with at least one sensor, wherein the sensor is an infrared (IR) imaging sensor or a light detection and ranging (LiDAR) sensor, the SoC comprising:

driving circuitry that is configured to provide at least one of a set of supply voltages, a set of reference voltages, a set of clock signals, and a set of timing signals to the sensor for driving the sensor;

processing circuitry that is configured to receive at least one output signal generated by the sensor, and generate a processed output signal; and

feedback circuitry that is connected to the processing circuitry and the driving circuitry, wherein the feedback circuitry is configured to receive the processed output signal and environmental data indicative of environmental conditions associated with the sensor, and generate a first set of control signals, and wherein the feedback circuitry is further configured to provide the first set of control signals to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor.

2. The SoC of claim 1, wherein the driving circuitry includes:

a power supply generator that is configured to generate the set of supply voltages; a reference voltage generator that is configured to generate the set of reference voltages; and

a clock generator that is configured to generate the set of clock signals.

3. The SoC of claim 2, wherein the driving circuitry further includes a timing signal generator that is connected to the clock generator, and is configured to:

receive, from the clock generator, at least one clock signal of the set of clock signals; and

generate, based on the received clock signal, the set of timing signals.

4. The SoC of claim 1, wherein the feedback circuitry is further configured to:

generate, based on the processed output signal and the environmental data, a second set of control signals; and provide the second set of control signals to the processing circuitry for dynamically configuring one or more operational parameters of the processing circuitry.

5. The SoC of claim 1, wherein when the output signal is in an analog format, the processing circuitry includes:

analog processing circuitry that is configured to receive the output signal, and generate an intermediate analog signal by executing a set of analog processing operations on the output signal;

an analog-to-digital converter (ADC) that is connected to the analog processing circuitry and is configured to:

receive, from the analog processing circuitry, the intermediate analog signal; and convert the intermediate analog signal to an intermediate digital signal; and digital processing circuitry that is connected to the ADC and is configured to:

receive, from the ADC, the intermediate digital signal; and

generate the processed output signal by executing a set of digital processing operations on the intermediate digital signal.

6. The SoC of claim 5, wherein the set of analog processing operations includes at least one of a buffering operation, a black-level adjustment operation, a single-to-differential conversion operation, a correlated-double sampling operation, and a range matching operation.

7. The SoC of claim 5, wherein the set of digital processing operations includes at least one of a set of pixel level operations, a set of kernel level operations, and a set of fixed function operations, and wherein the set of fixed function operations includes at least one of an histogram equalization operation, an auto gain control operation, a non-linear spatial filtering operation, and a fast Fourier transform operation.

8. The SoC of claim 1, wherein when the output signal is in an analog format, the processing circuitry includes:

time-of-flight (ToF) processing circuitry that is configured to receive the output signal, and generate a ToF digital signal by executing a set of ToF processing operations on the output signal, wherein the set of ToF processing operations includes at least one of a current to voltage conversion operation, a threshold voltage comparison operation, and a time-to- digital conversion operation; and

digital processing circuitry that is connected to the ToF processing circuitry and is configured to:

receive, from the ToF processing circuitry, the ToF digital signal; and generate the processed output signal by executing a set of pixel level operations on the ToF digital signal.

9. The SoC of claim 1, wherein when the output signal is in a digital format, the processing circuitry includes:

time-of-flight (ToF) processing circuitry that is configured to receive the output signal, and generate a ToF digital signal by executing a set of time-to-digital conversion operations on the output signal; and

10. The SoC of claim 1, wherein when the output signal is in a digital format, the processing circuitry generates the processed output signal by executing a set of digital processing operations on the output signal, wherein the set of digital processing operations includes at least one of a set of pixel level operations, a set of kernel level operations, and a set of fixed function operations, and wherein the set of fixed function operations includes at least one of an histogram equalization operation, an auto gain control operation, a non-linear spatial filtering operation, and a fast Fourier transform operation.

11. The SoC of claim 1, further comprising a system controller that is connected to the driving circuitry, the processing circuitry, and the feedback circuitry for controlling operations of the driving circuitry, the processing circuitry, and the feedback circuitry, respectively.

12. A camera system, comprising: at least one sensor, wherein the sensor is an infrared (IR) imaging sensor or a light detection and ranging (LiDAR) sensor;

monitoring circuitry that is configured to generate environmental data indicative of environmental conditions associated with the sensor; and

a system-on-chip (SoC) interfaced with the sensor and the monitoring circuitry, the SoC comprising:

feedback circuitry that is connected to the processing circuitry, the driving circuitry, and the monitoring circuitry, wherein the feedback circuitry is configured to:

receive, from the processing circuitry and the monitoring circuitry, the processed output signal and the environmental data, respectively;

generate, based on the processed output signal and the environmental data, a first set of control signals; and

provide the first set of control signals to the driving circuitry for dynamically configuring one or more operational parameters of the driving circuitry, thereby controlling an operation of the sensor.

13. The camera system of claim 12, wherein the driving circuitry includes:

a clock generator that is configured to generate the set of clock signals.

14. The camera system of claim 13, wherein the driving circuitry further includes a timing signal generator that is connected to the clock generator, and is configured to:

generate, based on the received clock signal, the set of timing signals.

15. The camera system of claim 12, wherein when the output signal is in an analog format, the processing circuitry includes:

receive, from the ADC, the intermediate digital signal; and

16. The camera system of claim 15, wherein the set of analog processing operations includes at least one of a buffering operation, a black-level adjustment operation, a single-to- differential conversion operation, a correlated-double sampling operation, and a range matching operation, wherein the set of digital processing operations includes at least one of a set of pixel level operations, a set of kernel level operations, and a set of fixed function operations, and wherein the set of fixed function operations includes at least one of an histogram equalization operation, an auto gain control operation, a non-linear spatial filtering operation, and a fast Fourier transform operation.

17. The camera system of claim 12, wherein when the output signal is in an analog format, the processing circuitry includes:

18. The camera system of claim 12, wherein when the output signal is in a digital format, the processing circuitry includes:

19. The camera system of claim 12, wherein when the output signal is in a digital format, the processing circuitry generates the processed output signal by executing a set of digital processing operations on the output signal, wherein the set of digital processing operations includes at least one of a set of pixel level operations, a set of kernel level operations, and a set of fixed function operations, and wherein the set of fixed function operations includes at least one of an histogram equalization operation, an auto gain control operation, a non-linear spatial filtering operation, and a fast Fourier transform operation.

20. The camera system of claim 12, wherein the feedback circuitry is further configured to: generate, based on the processed output signal and the environmental data, a second set of control signals; and

provide the second set of control signals to the processing circuitry for dynamically configuring one or more operational parameters of the processing circuitry.