JP7251401B2 - 周辺映像生成装置、周辺映像生成方法、およびプログラム - Google Patents

周辺映像生成装置、周辺映像生成方法、およびプログラム Download PDF

Info

Publication number
JP7251401B2
JP7251401B2 JP2019147985A JP2019147985A JP7251401B2 JP 7251401 B2 JP7251401 B2 JP 7251401B2 JP 2019147985 A JP2019147985 A JP 2019147985A JP 2019147985 A JP2019147985 A JP 2019147985A JP 7251401 B2 JP7251401 B2 JP 7251401B2
Authority
JP
Japan
Prior art keywords
image
peripheral
unit
video
shielded area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019147985A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021029037A5 (enExample
JP2021029037A (ja
Inventor
貴弘 小口
陽介 服部
雄介 関川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Denso Corp
Original Assignee
Denso Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Denso Corp filed Critical Denso Corp
Priority to JP2019147985A priority Critical patent/JP7251401B2/ja
Priority to CN202080056408.4A priority patent/CN114206674B/zh
Priority to PCT/JP2020/028542 priority patent/WO2021029203A1/ja
Priority to DE112020003788.6T priority patent/DE112020003788T5/de
Publication of JP2021029037A publication Critical patent/JP2021029037A/ja
Publication of JP2021029037A5 publication Critical patent/JP2021029037A5/ja
Priority to US17/650,201 priority patent/US12450784B2/en
Application granted granted Critical
Publication of JP7251401B2 publication Critical patent/JP7251401B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Traffic Control Systems (AREA)
JP2019147985A 2019-08-09 2019-08-09 周辺映像生成装置、周辺映像生成方法、およびプログラム Active JP7251401B2 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2019147985A JP7251401B2 (ja) 2019-08-09 2019-08-09 周辺映像生成装置、周辺映像生成方法、およびプログラム
CN202080056408.4A CN114206674B (zh) 2019-08-09 2020-07-22 周边影像生成装置、周边影像生成方法以及存储介质
PCT/JP2020/028542 WO2021029203A1 (ja) 2019-08-09 2020-07-22 周辺映像生成装置、周辺映像生成方法、およびプログラム
DE112020003788.6T DE112020003788T5 (de) 2019-08-09 2020-07-22 Peripheres-Video-Erzeugungsvorrichtung, Verfahren zur Erzeugung peripherer Videos, und Programm
US17/650,201 US12450784B2 (en) 2019-08-09 2022-02-07 Peripheral video generation device, peripheral video generation method, and storage medium storing program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2019147985A JP7251401B2 (ja) 2019-08-09 2019-08-09 周辺映像生成装置、周辺映像生成方法、およびプログラム

Publications (3)

Publication Number Publication Date
JP2021029037A JP2021029037A (ja) 2021-02-25
JP2021029037A5 JP2021029037A5 (enExample) 2021-11-11
JP7251401B2 true JP7251401B2 (ja) 2023-04-04

Family

ID=74569578

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2019147985A Active JP7251401B2 (ja) 2019-08-09 2019-08-09 周辺映像生成装置、周辺映像生成方法、およびプログラム

Country Status (5)

Country Link
US (1) US12450784B2 (enExample)
JP (1) JP7251401B2 (enExample)
CN (1) CN114206674B (enExample)
DE (1) DE112020003788T5 (enExample)
WO (1) WO2021029203A1 (enExample)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11816585B2 (en) * 2018-12-03 2023-11-14 Tesla, Inc. Machine learning models operating at different frequencies for autonomous vehicles
DE102020101637A1 (de) * 2020-01-24 2021-07-29 Bayerische Motoren Werke Aktiengesellschaft Erzeugen einer Draufsicht auf ein Kraftfahrzeug
JPWO2023089834A1 (enExample) 2021-11-22 2023-05-25

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001034898A (ja) 1999-07-21 2001-02-09 Toyota Central Res & Dev Lab Inc 探査不能領域推定装置及び運転支援システム
JP2010072836A (ja) 2008-09-17 2010-04-02 Toyota Motor Corp 周辺監視装置
JP2018122627A (ja) 2017-01-30 2018-08-09 株式会社デンソー 車両制御装置
JP2018142297A (ja) 2017-02-27 2018-09-13 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 情報処理装置およびプログラム

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4600760B2 (ja) * 2005-06-27 2010-12-15 アイシン精機株式会社 障害物検出装置
KR100966288B1 (ko) 2009-01-06 2010-06-28 주식회사 이미지넥스트 주변 영상 생성 방법 및 장치
CN105667518B (zh) * 2016-02-25 2018-07-24 福州华鹰重工机械有限公司 车道检测的方法及装置
US20180178840A1 (en) * 2016-12-28 2018-06-28 Automotive Research & Testing Center Automatic vehicle parking assistance correcting system with instant environmental detection and correcting method thereof
GB2559760B (en) * 2017-02-16 2019-08-28 Jaguar Land Rover Ltd Apparatus and method for displaying information
CN108501949B (zh) 2017-02-27 2022-11-22 松下电器（美国）知识产权公司 信息处理装置以及记录介质
JP7003730B2 (ja) 2018-02-27 2022-01-21 トヨタ自動車株式会社 金属積層造形方法
US10901416B2 (en) * 2018-07-19 2021-01-26 Honda Motor Co., Ltd. Scene creation system for autonomous vehicles and methods thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001034898A (ja) 1999-07-21 2001-02-09 Toyota Central Res & Dev Lab Inc 探査不能領域推定装置及び運転支援システム
JP2010072836A (ja) 2008-09-17 2010-04-02 Toyota Motor Corp 周辺監視装置
JP2018122627A (ja) 2017-01-30 2018-08-09 株式会社デンソー 車両制御装置
JP2018142297A (ja) 2017-02-27 2018-09-13 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 情報処理装置およびプログラム

Also Published As

Publication number Publication date
US20220156985A1 (en) 2022-05-19
CN114206674A (zh) 2022-03-18
DE112020003788T5 (de) 2022-06-30
US12450784B2 (en) 2025-10-21
CN114206674B (zh) 2025-01-14
WO2021029203A1 (ja) 2021-02-18
JP2021029037A (ja) 2021-02-25

Similar Documents

Publication Publication Date Title
KR102330744B1 (ko) 신경망을 이용한 장면 이해 및 생성
US6570566B1 (en) Image processing apparatus, image processing method, and program providing medium
JP6310652B2 (ja) 映像表示システム、映像合成装置及び映像合成方法
JP4966431B2 (ja) 画像処理装置
JP7251401B2 (ja) 周辺映像生成装置、周辺映像生成方法、およびプログラム
EP2739050A1 (en) Vehicle surroundings monitoring system
JP7461504B2 (ja) モデル生成方法、画像透視図の決定方法、装置、ディバイス及び媒体
JP2017520133A (ja) 車両周辺イメージ生成装置および方法
CN114793457B (zh) 用于改进确定深度图、相对姿势或语义分割的过程的设备和方法
Schick et al. Evaluation of video-based driver assistance systems with sensor data fusion by using virtual test driving
JP2021029037A5 (enExample)
JP2024022212A (ja) 画像処理装置、画像処理方法およびプログラム
WO2022244800A1 (ja) 画像処理アルゴリズム評価装置
KR101470198B1 (ko) 영상 합성 장치 및 방법
US12304395B2 (en) Camera monitoring system display including parallax manipulation
CN114600151A (zh) 用于深度致密化的域适应
CN117994777A (zh) 一种基于路侧相机的三维目标检测方法
CN119968654A (zh) 用于生成局外人视角图像的方法和设备以及训练神经网络的方法
Kikuchi et al. Automatic diminished reality-based virtual demolition method using semantic segmentation and generative adversarial network for landscape assessment
Eskandari et al. Observation-based diminished reality: a systematic literature review
CN117201705B (zh) 一种全景图像的获取方法、装置、电子设备及存储介质
JP7304922B2 (ja) 周辺監視システム
EP4506905A1 (en) Image processing method and apparatus, and model training method and apparatus
EP4276763A1 (en) Image generation for testing an electronic vehicle guidance system
JP2009282736A (ja) 画像処理装置、画像処理方法、及びプログラム

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20211004

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20211005

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20220823

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221020

A711 Notification of change in applicant

Free format text: JAPANESE INTERMEDIATE CODE: A711

Effective date: 20221104

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20221104

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20230221

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20230306

R151 Written notification of patent or utility model registration

Ref document number: 7251401

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151