JP7707174B2 - 知覚システムを訓練して確認する方法及びシステム - Google Patents

知覚システムを訓練して確認する方法及びシステム Download PDF

Info

Publication number
JP7707174B2
JP7707174B2 JP2022543814A JP2022543814A JP7707174B2 JP 7707174 B2 JP7707174 B2 JP 7707174B2 JP 2022543814 A JP2022543814 A JP 2022543814A JP 2022543814 A JP2022543814 A JP 2022543814A JP 7707174 B2 JP7707174 B2 JP 7707174B2
Authority
JP
Japan
Prior art keywords
image
neural network
sensors
detection
classification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022543814A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022550487A5 (enExample
JP2022550487A (ja
Inventor
ユヴァル ネフマディ
エズラ シャハール ベン
シュムエル マンガン
マーク ワグナー
アンナ コーエン
イツィク アヴィタル
Original Assignee
ヴァヤヴィジョン センシング リミテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ヴァヤヴィジョン センシング リミテッド filed Critical ヴァヤヴィジョン センシング リミテッド
Publication of JP2022550487A publication Critical patent/JP2022550487A/ja
Publication of JP2022550487A5 publication Critical patent/JP2022550487A5/ja
Application granted granted Critical
Publication of JP7707174B2 publication Critical patent/JP7707174B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features
    • G06V10/7784—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors
    • G06V10/7792—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors the supervisor being an automated module, e.g. "intelligent oracle"
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Rehabilitation Tools (AREA)
  • Walking Sticks, Umbrellas, And Fans (AREA)
JP2022543814A 2019-09-22 2020-09-22 知覚システムを訓練して確認する方法及びシステム Active JP7707174B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962903846P 2019-09-22 2019-09-22
US62/903,846 2019-09-22
PCT/IL2020/051028 WO2021053680A2 (en) 2019-09-22 2020-09-22 Methods and systems for training and validating a perception system

Publications (3)

Publication Number Publication Date
JP2022550487A JP2022550487A (ja) 2022-12-01
JP2022550487A5 JP2022550487A5 (enExample) 2023-09-14
JP7707174B2 true JP7707174B2 (ja) 2025-07-14

Family

ID=74884380

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022543814A Active JP7707174B2 (ja) 2019-09-22 2020-09-22 知覚システムを訓練して確認する方法及びシステム

Country Status (7)

Country Link
US (2) US12093834B2 (enExample)
EP (1) EP4031835A4 (enExample)
JP (1) JP7707174B2 (enExample)
KR (1) KR20220093312A (enExample)
CN (1) CN114787880A (enExample)
CA (2) CA3155593C (enExample)
WO (1) WO2021053680A2 (enExample)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12487564B2 (en) 2024-12-19 2025-12-02 Digital Global Systems, Inc. Systems and methods of sensor data fusion
US12314346B2 (en) 2024-12-19 2025-05-27 Digital Global Systems, Inc. Systems and methods of sensor data fusion
US12479105B2 (en) 2024-12-19 2025-11-25 Digital Global Systems, Inc Systems and methods of sensor data fusion

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018032074A (ja) 2016-08-22 2018-03-01 株式会社リコー 画像処理装置、撮像装置、移動体機器制御システム、画像処理方法、及びプログラム
US20190266418A1 (en) 2018-02-27 2019-08-29 Nvidia Corporation Real-time detection of lanes and boundaries by autonomous vehicles
WO2019178253A1 (en) 2018-03-13 2019-09-19 Magic Leap, Inc. Image-enhanced depth sensing using machine learning

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USRE46930E1 (en) 2007-12-21 2018-07-03 Leddartech Inc. Distance detection method and system
US8908159B2 (en) 2011-05-11 2014-12-09 Leddartech Inc. Multiple-field-of-view scannerless optical rangefinder in high ambient background light
JP5629642B2 (ja) 2011-05-19 2014-11-26 株式会社ソニー・コンピュータエンタテインメント 動画像撮影装置、情報処理システム、情報処理装置、および画像データ処理方法
CA2998175C (en) 2012-03-02 2020-11-10 Leddartech Inc. System and method for multipurpose traffic detection and characterization
US9269012B2 (en) 2013-08-22 2016-02-23 Amazon Technologies, Inc. Multi-tracker object tracking
EP3283843B1 (en) 2015-04-01 2024-01-10 Vayavision Sensing Ltd. Generating 3-dimensional maps of a scene using passive and active measurements
US20170140645A1 (en) * 2015-11-06 2017-05-18 The Board Of Regents Of The University Of Oklahoma Traffic monitoring system
EP4428578A3 (en) * 2017-01-03 2025-03-12 Innoviz Technologies Ltd. Lidar systems and methods for detection and classification of objects
CN110494715B (zh) 2017-02-06 2024-11-05 瓦亚视觉感知有限公司 计算机辅助驾驶
US10445928B2 (en) * 2017-02-11 2019-10-15 Vayavision Ltd. Method and system for generating multidimensional maps of a scene using a plurality of sensors of various types
KR102541559B1 (ko) * 2017-08-04 2023-06-08 삼성전자주식회사 관심 객체를 검출하는 방법 및 장치
US10356341B2 (en) 2017-10-13 2019-07-16 Fyusion, Inc. Skeleton-based effects and background replacement
US11292483B2 (en) 2018-02-28 2022-04-05 VayaVision Sensing, Ltd. Managing a change in a physical property of a vehicle due to an external object
US10628686B2 (en) * 2018-03-12 2020-04-21 Waymo Llc Neural networks for object detection and characterization
WO2019177562A1 (en) * 2018-03-15 2019-09-19 Harman International Industries, Incorporated Vehicle system and method for detecting objects and object distance
WO2019182974A2 (en) * 2018-03-21 2019-09-26 Nvidia Corporation Stereo depth estimation using deep neural networks
CN108875595A (zh) * 2018-05-29 2018-11-23 重庆大学 一种基于深度学习和多层特征融合的驾驶场景目标检测方法
CN109002807A (zh) * 2018-07-27 2018-12-14 重庆大学 一种基于ssd神经网络的驾驶场景车辆检测方法
KR102815456B1 (ko) * 2018-09-12 2025-05-30 삼성전자주식회사 영상 처리를 위한 학습 데이터 생성 방법, 영상 처리 방법, 및 그 장치들
US10460208B1 (en) * 2019-01-02 2019-10-29 Cognata Ltd. System and method for generating large simulation data sets for testing an autonomous driver
CA3125623C (en) 2020-07-21 2023-06-27 Leddartech Inc. Beam-steering device particularly for lidar systems
CA3125716C (en) 2020-07-21 2024-04-09 Leddartech Inc. Systems and methods for wide-angle lidar using non-uniform magnification optics
WO2022016274A1 (en) 2020-07-21 2022-01-27 Leddartech Inc. Beam-steering devices and methods for lidar applications

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018032074A (ja) 2016-08-22 2018-03-01 株式会社リコー 画像処理装置、撮像装置、移動体機器制御システム、画像処理方法、及びプログラム
US20190266418A1 (en) 2018-02-27 2019-08-29 Nvidia Corporation Real-time detection of lanes and boundaries by autonomous vehicles
WO2019178253A1 (en) 2018-03-13 2019-09-19 Magic Leap, Inc. Image-enhanced depth sensing using machine learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Eric Tzeng et al.，"Adversarial Discriminative Domain Adaptation"，2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)，米国，IEEE，2017年07月21日，pp.2962-2971
Fan Zhu et al.，"Boosted Cross-Domain Dictionary Learning for Visual Categorization"，IEEE Intelligent Systems，米国，IEEE，2016年03月18日，Vol.31, No.3，pp.6-18
Gledson Melotti et al.，"CNN-LIDAR pedestrian classification: combining range and reflectance data"，2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES)，米国，IEEE，2018年09月12日，pp.1-6
Wen Li et al.，"Visual Recognition in RGB Images and Videos by Learning from RGB-D Data"，IEEE Transactions on Pattern Analysis and Machine Intelligence，米国，IEEE，2017年08月02日，Vol.40, No.8，pp.2030-2036

Also Published As

Publication number Publication date
EP4031835A4 (en) 2023-10-04
WO2021053680A2 (en) 2021-03-25
CA3155593A1 (en) 2021-03-25
EP4031835A2 (en) 2022-07-27
US12093834B2 (en) 2024-09-17
CN114787880A (zh) 2022-07-22
CA3198976A1 (en) 2021-03-25
KR20220093312A (ko) 2022-07-05
WO2021053680A3 (en) 2021-06-17
US20220335729A1 (en) 2022-10-20
US20240419974A1 (en) 2024-12-19
CA3155593C (en) 2023-07-25
JP2022550487A (ja) 2022-12-01

Similar Documents

Publication Publication Date Title
CN114723955B (zh) 图像处理方法、装置、设备和计算机可读存储介质
US11074438B2 (en) Disentangling human dynamics for pedestrian locomotion forecasting with noisy supervision
US12008817B2 (en) Systems and methods for depth estimation in a vehicle
KR102270827B1 (ko) 360도 주변 물체 검출 및 인식 작업을 위한 다중 센서 데이터 기반의 융합 정보 생성 방법 및 장치
US20240419974A1 (en) Methods and systems for training and validating a perception system
US11810311B2 (en) Two-stage depth estimation machine learning algorithm and spherical warping layer for equi-rectangular projection stereo matching
Choi et al. Mask2map: Vectorized hd map construction using bird’s eye view segmentation masks
CN118226421B (zh) 基于反射率图的激光雷达-相机在线标定方法及系统
CN114782785A (zh) 多传感器信息融合方法及装置
US12100221B2 (en) Methods and electronic devices for detecting objects in surroundings of a self-driving car
US20250289369A1 (en) Efficient cloud-based dynamic multi-vehicle bev feature fusion for extended robust cooperative perception
CN120032339B (zh) 基于bev与全稀疏架构的自动驾驶场景的目标检测方法
CN114972425B (zh) 运动状态估计模型的训练方法、运动状态估计方法及装置
CN120472447A (zh) 一种结构化环境下多模态稀疏融合三维目标检测方法
KR20260049188A (ko) 융합 기반 인지를 위한 이미지 및 LiDAR 적응형 트랜스포머
US20240355042A1 (en) Fusing neural radiance fields by registration and blending
CN119274167A (zh) 一种基于时空融合的多模态自动驾驶三维目标跟踪方法
EP3992909A1 (en) Two-stage depth estimation machine learning algorithm and spherical warping layer for equi-rectangular projection stereo matching
CN120877253B (zh) 场景感知方法、训练方法、程序产品、介质和电子设备
CN116863284B (zh) 基于雷达-光学跨模态特征点融合的目标检测方法及装置
KR102721587B1 (ko) 센싱 정보 처리 장치 및 방법
Kälble et al. EvOcc: Accurate semantic occupancy for automated driving using evidence theory
HK40079164A (en) Methods and systems for training and validating a perception system
Kübel et al. Camera-radar sensor fusion using deep learning
Mustajbasic Learning-Based Sensor Fusion for 3D Scene Understanding in ADAS and AD

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230906

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230906

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240531

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240604

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20240815

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20241101

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20241119

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250319

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250602

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250702

R150 Certificate of patent or registration of utility model

Ref document number: 7707174

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150