CN115299039A - 智能传感器 - Google Patents

智能传感器 Download PDF

Info

Publication number
CN115299039A
CN115299039A CN202180022776.1A CN202180022776A CN115299039A CN 115299039 A CN115299039 A CN 115299039A CN 202180022776 A CN202180022776 A CN 202180022776A CN 115299039 A CN115299039 A CN 115299039A
Authority
CN
China
Prior art keywords
memory
sensor
layer
model
examples
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202180022776.1A
Other languages
English (en)
Chinese (zh)
Inventor
刘新桥
芭芭拉·达萨尔沃
汉斯·雷瑟霍夫
李子云
阿西夫·伊蒂亚兹·可汗
赛义德·沙基卜·萨尔瓦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Meta Platforms Technologies LLC
Original Assignee
Meta Platforms Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/313,884 external-priority patent/US20210264679A1/en
Application filed by Meta Platforms Technologies LLC filed Critical Meta Platforms Technologies LLC
Publication of CN115299039A publication Critical patent/CN115299039A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G06N3/065—Analogue means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/79—Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Medical Informatics (AREA)
  • Vascular Medicine (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Neurology (AREA)
  • Image Analysis (AREA)
  • Hall/Mr Elements (AREA)
  • Solid State Image Pick-Up Elements (AREA)
  • Mram Or Spin Memory Techniques (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)
  • Semiconductor Memories (AREA)
CN202180022776.1A 2020-05-07 2021-05-07 智能传感器 Pending CN115299039A (zh)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
US202063021476P 2020-05-07 2020-05-07
US63/021,476 2020-05-07
US202063038636P 2020-06-12 2020-06-12
US63/038,636 2020-06-12
US17/313,884 2021-05-06
US17/313,884 US20210264679A1 (en) 2017-07-25 2021-05-06 Smart sensor
PCT/US2021/031201 WO2021226411A1 (en) 2020-05-07 2021-05-07 Smart sensor

Publications (1)

Publication Number Publication Date
CN115299039A true CN115299039A (zh) 2022-11-04

Family

ID=76197599

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180022776.1A Pending CN115299039A (zh) 2020-05-07 2021-05-07 智能传感器

Country Status (5)

Country Link
EP (1) EP4147445A1 (ja)
JP (1) JP2023525950A (ja)
CN (1) CN115299039A (ja)
TW (1) TW202146979A (ja)
WO (1) WO2021226411A1 (ja)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115291388A (zh) * 2017-07-25 2022-11-04 元平台技术有限公司 传感器组件和头戴式显示器
CN116156298A (zh) * 2023-04-11 2023-05-23 安徽医科大学 基于感存算一体化的内窥镜高清视频处理系统及方法

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11706546B2 (en) * 2021-06-01 2023-07-18 Sony Semiconductor Solutions Corporation Image sensor with integrated single object class detection deep neural network (DNN)
EP4221188A1 (en) * 2022-01-27 2023-08-02 VoxelSensors SRL Efficient image sensor
TWI845068B (zh) * 2022-03-29 2024-06-11 宏達國際電子股份有限公司 頭戴式顯示裝置
TWI873926B (zh) * 2022-06-30 2025-02-21 宏達國際電子股份有限公司 影像感測裝置及頭戴式顯示裝置
TWI848311B (zh) * 2022-06-30 2024-07-11 宏達國際電子股份有限公司 影像感測裝置及頭戴式顯示裝置
US12028505B2 (en) 2022-06-30 2024-07-02 Htc Corporation Image sensing device and head-mounted display
US11645032B1 (en) 2022-09-14 2023-05-09 International Business Machines Corporation Smart multi-screen wall arrangement in a virtual reality environment
SE546661C2 (en) * 2022-09-28 2025-01-14 Tobii Ab Image acquisition with dynamic resolution

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109298528A (zh) * 2017-07-25 2019-02-01 脸谱科技有限责任公司 传感器组件和头戴式显示器
CN110572593A (zh) * 2019-08-19 2019-12-13 上海集成电路研发中心有限公司 一种3d堆叠式图像传感器
US20200053299A1 (en) * 2018-08-10 2020-02-13 International Business Machines Corporation Image sensor with analog sample and hold circuit control for analog neural networks

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI511257B (zh) * 2013-07-30 2015-12-01 Univ Nat Chiao Tung 半導體元件之內連接結構
US9940534B1 (en) * 2016-10-10 2018-04-10 Gyrfalcon Technology, Inc. Digital integrated circuit for extracting features out of an input image based on cellular neural networks
IL259285B2 (en) * 2018-05-10 2023-07-01 Inspekto A M V Ltd A system and method for detecting defects on objects in an image
TW202038456A (zh) * 2018-10-26 2020-10-16 日商索尼半導體解決方案公司 固態攝像元件、固態攝像元件封裝及電子機器
JP7528064B2 (ja) * 2019-04-29 2024-08-05 株式会社半導体エネルギー研究所 撮像装置および電子機器

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109298528A (zh) * 2017-07-25 2019-02-01 脸谱科技有限责任公司 传感器组件和头戴式显示器
US20200053299A1 (en) * 2018-08-10 2020-02-13 International Business Machines Corporation Image sensor with analog sample and hold circuit control for analog neural networks
CN110572593A (zh) * 2019-08-19 2019-12-13 上海集成电路研发中心有限公司 一种3d堆叠式图像传感器

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MILLET LAURENT ET AL.: "A 5500-Frames/s 85-GOPS/W 3-D Stacked BSI Vision Chip Based on Parallel in-focal-plane Acquisition and Processing", IEEE JOURNAL OF SOLID-STATE CIRCUITS, vol. 54, no. 4, 1 April 2019 (2019-04-01), pages 1096 - 1105, XP011716786, DOI: 10.1109/JSSC.2018.2886325 *
SEBASTIAN ABU ET AL.: "Memory Devices and Applications for In-Memory Computing", NATURE NANOTECHNOLOGY, vol. 15, no. 7, 30 March 2020 (2020-03-30), pages 529 - 544, XP037194929, DOI: 10.1038/s41565-020-0655-z *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115291388A (zh) * 2017-07-25 2022-11-04 元平台技术有限公司 传感器组件和头戴式显示器
CN115291388B (zh) * 2017-07-25 2025-05-13 元平台技术有限公司 传感器组件和头戴式显示器
CN116156298A (zh) * 2023-04-11 2023-05-23 安徽医科大学 基于感存算一体化的内窥镜高清视频处理系统及方法

Also Published As

Publication number Publication date
JP2023525950A (ja) 2023-06-20
WO2021226411A1 (en) 2021-11-11
EP4147445A1 (en) 2023-03-15
TW202146979A (zh) 2021-12-16

Similar Documents

Publication Publication Date Title
US20210264679A1 (en) Smart sensor
US11568609B1 (en) Image sensor having on-chip compute circuit
CN115299039A (zh) 智能传感器
US11315330B1 (en) Sensor system based on stacked sensor layers
US11948089B2 (en) Sparse image sensing and processing
US12108141B2 (en) Dynamically programmable image sensor
US11888002B2 (en) Dynamically programmable image sensor
Akinyelu et al. Convolutional neural network-based methods for eye gaze estimation: A survey
TWI810304B (zh) 用於產生影像圖框之設備及方法
CN113454984B (zh) 可编程像素阵列
US11488320B2 (en) Pose estimation method, pose estimation apparatus, and training method for pose estimation
US12244936B2 (en) On-sensor image processor utilizing contextual data
Tabrizchi et al. AppCiP: Energy-efficient approximate convolution-in-pixel scheme for neural network acceleration
US12075175B1 (en) Programmable smart sensor with adaptive readout
US20220405553A1 (en) Sparse image processing
US11935575B1 (en) Heterogeneous memory system
EP4360001A1 (en) Sparse image processing
US12141888B1 (en) Dynamic and hierarchical image sensing and processing
CN117581256A (zh) 稀疏图像处理
EP4470202A1 (en) On-sensor image processor utilizing contextual data
CN118511501A (zh) 利用上下文数据的传感器上图像处理器

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20221104