WO2023276221A1 - Image processing system, image processing device, and image processing method - Google Patents

Image processing system, image processing device, and image processing method Download PDF

Info

Publication number
WO2023276221A1
WO2023276221A1 PCT/JP2022/004035 JP2022004035W WO2023276221A1 WO 2023276221 A1 WO2023276221 A1 WO 2023276221A1 JP 2022004035 W JP2022004035 W JP 2022004035W WO 2023276221 A1 WO2023276221 A1 WO 2023276221A1
Authority
WO
WIPO (PCT)
Prior art keywords
image processing
noise
image data
noise reduction
transmission line
Prior art date
Application number
PCT/JP2022/004035
Other languages
French (fr)
Japanese (ja)
Inventor
光一 信本
雄基 鶴田
裕貴 折出
Original Assignee
日立Astemo株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日立Astemo株式会社 filed Critical 日立Astemo株式会社
Priority to DE112022002245.0T priority Critical patent/DE112022002245T5/en
Priority to JP2023531363A priority patent/JPWO2023276221A1/ja
Publication of WO2023276221A1 publication Critical patent/WO2023276221A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • H04N5/213—Circuitry for suppressing or minimising impulsive noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B1/00—Details of transmission systems, not covered by a single one of groups H04B3/00 - H04B13/00; Details of transmission systems not characterised by the medium used for transmission
    • H04B1/06—Receivers
    • H04B1/10—Means associated with receiver for limiting or suppressing noise or interference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B1/00—Details of transmission systems, not covered by a single one of groups H04B3/00 - H04B13/00; Details of transmission systems not characterised by the medium used for transmission
    • H04B1/38—Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
    • H04B1/3822—Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving specially adapted for use in vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance

Definitions

  • the present invention relates to an image processing system, an image processing apparatus, and an image processing method for correcting image data from a camera transmitted via a transmission line.
  • the stereo camera device simultaneously measures visual information from images and distance information to an object, so it can detect various objects (people, cars, three-dimensional objects, road surfaces, road signs, signboard signs) around the car. etc.) can be grasped in detail, and it is said that it will also contribute to the improvement of safety during driving support.
  • Patent Document 1 Japanese Patent Laid-Open No. 2004-100000 describes reducing large unnecessary radiation noise generated when image data signals, clock signals, and image synchronization signals transmitted from two imaging units are transmitted to an image processing unit in an on-vehicle camera device. It is
  • the signals transmitted from the two imaging units are temporally shifted to reduce the amount of noise on the signal line, but no consideration is given to reducing communication errors due to external noise.
  • an object of the present invention is to provide an image processing system, an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on the image data transmission line.
  • the present invention includes an in-vehicle camera that acquires external information, an image processing device that is installed separately from the in-vehicle camera and processes image data acquired by the in-vehicle camera, and an in-vehicle camera and the image processing device.
  • an image processing system comprising a transmission line for transmitting image data by connecting the A noise source detection processing unit that determines whether or not an object registered in advance as a source is included in the image data; Accordingly, it has a noise reduction parameter determination unit that changes the parameters of the noise reduction filter.
  • an image processing system an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on transmission lines for image data.
  • FIG. 1 is a configuration block diagram of a conventional image processing system which is a premise of Embodiment 1;
  • FIG. 2 is a functional configuration block diagram of the image processing system in Embodiment 1.
  • FIG. FIG. 11 is a functional configuration block diagram of an image processing system according to Embodiment 2;
  • an image processing system using an in-vehicle camera installed in a car will be described as an example.
  • the image data acquired from the CMOS image sensor that constitutes the in-vehicle camera is transmitted through a transmission line using a communication IC such as Serdes (Serializer/Deserializer), which is a circuit that mutually converts serial data and parallel data. is assumed to be serially transmitted.
  • Serdes Serializer/Deserializer
  • FIG. 1 is a configuration block diagram imagining the hardware of a conventional image processing system, which is the premise of this embodiment.
  • an image processing system 100 includes a camera 101, an image processing device 102, and a transmission line 103 connecting them.
  • the camera 101 is composed of a CMOS image sensor 1011, a communication IC (Tx) 1012, and a connector section (Tx) 1013.
  • the image processing device 102 is composed of a connector section (Rx) 1021 , a communication IC (Rx) 1022 and a microcomputer 1024 .
  • the microcomputer 1024 is composed of a processor such as a general CPU (Central Processing Unit) and a storage device. Recognition processing of the target object is executed by software processing, and vehicle control is performed according to the recognition result.
  • the communication IC (Rx) 1022 which is the communication IC on the receiving side, is equipped with an equalizer such as a Continuous Time Linear Equalizer (hereinafter referred to as CTLE) and a Decision Feedback Equalizer (hereinafter referred to as DFE) to ensure communication quality.
  • CTLE Continuous Time Linear Equalizer
  • DFE Decision Feedback Equalizer
  • CTLE may also amplify noise components, so there is a tendency to use DFE in addition to CTLE in high-speed communication.
  • the DFE performs correction using feedback control, but if there is a radio tower or wireless communication area around the vehicle, or if wireless communication is performed nearby, it may receive large external noise in a short period of time. . If this external noise exceeds the correctable range of the equalizer, there is a possibility that the correction control by the equalizer will not keep up and a communication error will occur.
  • FIG. 2 is a functional configuration block diagram of the image processing system in this embodiment.
  • the same functions as in FIG. 1 are given the same reference numerals, and the description thereof is omitted. 2 differs from FIG. 1 in that it has a configuration using a stereo camera and has two systems of cameras, transmission lines, and noise reduction filters, and that it has a signal correction processing unit 1025 .
  • the image processing system 100 is an image processing system that performs three-dimensional object detection by processing parallax images using a stereo camera as means for detecting nearby vehicles. It should be noted that, for example, detection of a three-dimensional object using only a monocular camera or a combination of a monocular camera and radar can also be used. Also, three or more cameras may be used.
  • a stereo image is captured by camera 101 and camera 109 .
  • Image processor 102 receives image data via transmission lines 103 and 110 .
  • the captured image data is sent to the recognition processing unit 1026 through the noise reduction filters 104 and 111, and the distance information to the object is calculated by processing the parallax image and the object is recognized.
  • the noise reduction filters 104 and 111 that correct captured image data have their correction amounts controlled by the signal correction processing unit 1025 . That is, the captured image data from the noise reduction filters 104 and 111 are sent to the noise source detection processing unit 105 as external world information.
  • the noise source detection processing unit 105 refers to the external world information and the image of the object registered in the noise source information storage unit 108 stored in advance in the storage area 107, and the noise source and the noise source are included in the image data of the external world information. It is determined whether or not a target object is included. Then, when the object is included, the noise source detection processing unit 105 acquires detailed information of the noise source such as the frequency and noise level corresponding to the object registered in the noise source information storage unit 108 .
  • Examples of the external world information acquired by the camera include images of steel towers, lightning, mobile phone base stations, pantographs, radio communication taxis, relay vehicles, military vehicles, and signs such as high-voltage power transmission, and noise sources.
  • the detection processing unit 105 acquires information using them as noise sources.
  • the acquired detailed information of the noise source and the distance information to the noise source calculated by the recognition processing unit 1026 are sent to the noise reduction parameter determination unit 106 .
  • the noise reduction parameter determination unit 106 determines the correction amount of the noise reduction filter from the sent information, and corrects the parameters of the noise reduction filter. That is, the parameters of the noise reduction filter are corrected by feedforward control.
  • the signal correction processing unit 1025 first confirms the captured image, and when it confirms a noise source target in the image, starts correcting the filter. It should be noted that filter correction processing is not performed unless a noise source is confirmed in the captured image. Then, the frequency and noise level of the noise source are specified from the image of the external information, and the correction amount and correction timing of the noise reduction filter are determined by one or more of the frequency, noise level, and distance information to the noise source. to suppress communication errors caused by external noise.
  • the noise reduction filter is corrected using, for example, an equalizer in Serdes.
  • Serdes usually have two types of equalizers, CTLE and DFE, for filter correction.
  • the DFE noise correction parameter can set the amount of noise correction by changing the register setting. For example, by rewriting the register area for setting the noise correction amount using an interface provided by Serdes such as I2C (Inter-Integrated Circuit), the preset correction amount can be corrected in real time within a certain range.
  • the noise source is discriminated from the captured image, which is the external information, and the parameters of the noise reduction filter are changed according to the noise source, thereby suppressing the occurrence of communication errors and increasing the reliability. high communication processing becomes possible. Accordingly, it is possible to provide an image processing system, an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on the image data transmission line.
  • FIG. 3 is a functional configuration block diagram of the image processing system in this embodiment.
  • the same functions as in FIG. 2 are given the same reference numerals, and the description thereof is omitted.
  • FIG. 3 differs from FIG. 2 in that vehicle speed information 112 is provided as an input to noise reduction parameter determining section 106 .
  • the noise reduction parameter determination unit 106 performs processing such as shortening the interval between correction timings of the noise reduction filter when the vehicle speed is high according to the vehicle speed information.
  • the parameters of the noise reduction filter can be changed at more optimal correction timing of the noise reduction filter, and more reliable communication processing can be performed. It becomes possible.
  • the present invention is not limited to the above embodiments, and includes various modifications.
  • noise reduction in the transmission of image data has been described in the above embodiments
  • the present invention is not limited to image data and can also be applied to audio data.
  • the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations.
  • Noise source information storage unit 112 Vehicle speed information 1011: CMOS image sensor 1012: Communication IC (Tx) 1013: Connector unit (Tx) 1021: Connector unit (Rx) 1022: Communication IC (Rx), 1024: microcomputer, 1025: signal correction processing section, 1026 recognition processing section

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Image Processing (AREA)

Abstract

Provided are an image processing system, an image processing device, and an image processing method that make it possible to reduce communication errors caused by external noise over a transmission line for image data.　According to the present invention, an image processing system comprises an onboard camera that acquires external information, an image processing device that is provided separately from the onboard camera and processes image data acquired by the onboard camera, and a transmission line that connects the onboard camera and the image processing device and transmits the image data. The image processing device has: a noise reduction filter that reduces noise over the transmission line; and a processing processor. The processing processor has: a noise source detection processing unit that determines whether objects that have been pre-registered as noise sources for the transmission line are included in the image data; and a noise reduction parameter determination unit that, when a pre-registered noise source is included in the image data, modifies parameters for the noise reduction filter in accordance with the frequency or noise level of the noise source.

Description

画像処理システム、画像処理装置、及び画像処理方法Image processing system, image processing apparatus, and image processing method
　本発明は、伝送線を介して伝送されるカメラからの画像データを補正する画像処理システム、画像処理装置、及び画像処理方法に関する。 The present invention relates to an image processing system, an image processing apparatus, and an image processing method for correcting image data from a camera transmitted via a transmission line.
　近年、車載カメラ装置の普及により、安全運転や自動運転に向けた各種認識機能への要求が高まってきている。なかでも、ステレオカメラ装置は、画像による視覚的な情報と、対象物への距離情報を同時に計測するため、自動車周辺の様々な対象物（人、車、立体物、路面、路面標識、看板標識など）を詳細に把握でき、運転支援時の安全性の向上にも寄与するとされている。 In recent years, with the spread of in-vehicle camera devices, the demand for various recognition functions for safe driving and autonomous driving has increased. In particular, the stereo camera device simultaneously measures visual information from images and distance information to an object, so it can detect various objects (people, cars, three-dimensional objects, road surfaces, road signs, signboard signs) around the car. etc.) can be grasped in detail, and it is said that it will also contribute to the improvement of safety during driving support.
　しかしながら、カメラに使用するＣＭＯＳイメージセンサの解像度が増大するにつれ、画像処理を行うマイコンの発熱量が増加し、ＣＭＯＳイメージセンサと画像処理用マイコンを同一の筐体内に収めることが難しくなっている。そのため、カメラと画像処理用マイコンを搭載する画像処理基板を別体とし、カメラからの画像データを伝送線を介して伝送する構成が考えられる。その場合、伝送線上でのノイズが問題となる。 However, as the resolution of the CMOS image sensor used in the camera increases, the amount of heat generated by the microcomputer that performs image processing also increases, making it difficult to house the CMOS image sensor and image processing microcomputer in the same housing. For this reason, a configuration is conceivable in which a camera and an image processing board on which an image processing microcomputer is mounted are separately provided, and image data from the camera is transmitted through a transmission line. In that case, noise on the transmission line becomes a problem.
　本技術分野の背景技術として、例えば、特許文献１がある。特許文献１には、車載カメラ装置において、２つの撮像部から送信される画像データ信号やクロック信号、画像の同期信号を画像処理部へ伝送する時に発生する大きな不要輻射ノイズを低減することが記載されている。 As background art in this technical field, there is Patent Document 1, for example. Japanese Patent Laid-Open No. 2004-100000 describes reducing large unnecessary radiation noise generated when image data signals, clock signals, and image synchronization signals transmitted from two imaging units are transmitted to an image processing unit in an on-vehicle camera device. It is
特開２０１２－１８６５７４号公報JP 2012-186574 A
　特許文献１は、２つの撮像部から送信する信号を時間的にずらして信号線上のノイズ量を低減しているが、外部ノイズによる通信エラーの低減については考慮されていない。 In Patent Document 1, the signals transmitted from the two imaging units are temporally shifted to reduce the amount of noise on the signal line, but no consideration is given to reducing communication errors due to external noise.
　本発明は上記課題に鑑み、画像データの伝送線上での外部ノイズによる通信エラーの低減が可能な画像処理システム、画像処理装置、及び画像処理方法を提供することを目的とする。 In view of the above problems, an object of the present invention is to provide an image processing system, an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on the image data transmission line.
　本発明は、その一例を挙げるならば、外界情報を取得する車載カメラと、車載カメラとは別体で設置され車載カメラで取得した画像データを処理する画像処理装置と、車載カメラと画像処理装置をつなぎ画像データを伝送する伝送線からなる画像処理システムであって、画像処理装置は、伝送線でのノイズを低減するノイズ低減フィルタと、処理プロセッサを有し、処理プロセッサは、伝送線に対するノイズ源としてあらかじめ登録された対象物が画像データに含まれるか否かを判定するノイズ源検知処理部と、画像データにノイズ源が含まれる場合に、あらかじめ登録されたノイズ源の周波数またはノイズレベルに応じて、ノイズ低減フィルタのパラメータを変更するノイズ低減パラメータ決定部を有する。 To give an example, the present invention includes an in-vehicle camera that acquires external information, an image processing device that is installed separately from the in-vehicle camera and processes image data acquired by the in-vehicle camera, and an in-vehicle camera and the image processing device. an image processing system comprising a transmission line for transmitting image data by connecting the A noise source detection processing unit that determines whether or not an object registered in advance as a source is included in the image data; Accordingly, it has a noise reduction parameter determination unit that changes the parameters of the noise reduction filter.
　本発明によれば、画像データの伝送線上での外部ノイズによる通信エラーの低減が可能な画像処理システム、画像処理装置、及び画像処理方法を提供できる。 According to the present invention, it is possible to provide an image processing system, an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on transmission lines for image data.
実施例１の前提となる従来の画像処理システムの構成ブロック図である。1 is a configuration block diagram of a conventional image processing system which is a premise of Embodiment 1; FIG. 実施例１における画像処理システムの機能構成ブロック図である。2 is a functional configuration block diagram of the image processing system in Embodiment 1. FIG. 実施例２における画像処理システムの機能構成ブロック図である。FIG. 11 is a functional configuration block diagram of an image processing system according to Embodiment 2;
　以下、本発明を実施例について図面を参照して説明する。 Hereinafter, the present invention will be described with reference to the drawings in terms of embodiments.
　本実施例では、自動車に搭載する車載カメラによる画像処理システムを例にして説明する。そして、車載カメラを構成するＣＭＯＳイメージセンサから取得した画像データを、シリアルデータとパラレルデータを相互変換する回路であるＳｅｒｄｅｓ（Serializer/Deserializer）などの通信ＩＣを使用して、伝送線を介してデータをシリアル伝送する構成を前提に説明する。 In this embodiment, an image processing system using an in-vehicle camera installed in a car will be described as an example. Then, the image data acquired from the CMOS image sensor that constitutes the in-vehicle camera is transmitted through a transmission line using a communication IC such as Serdes (Serializer/Deserializer), which is a circuit that mutually converts serial data and parallel data. is assumed to be serially transmitted.
　図１は、本実施例の前提となる従来の画像処理システムのハードウェアをイメージした構成ブロック図である。図１において、画像処理システム１００は、カメラ１０１と画像処理装置１０２と、それらをつなぐ伝送線１０３で構成されている。 FIG. 1 is a configuration block diagram imagining the hardware of a conventional image processing system, which is the premise of this embodiment. In FIG. 1, an image processing system 100 includes a camera 101, an image processing device 102, and a transmission line 103 connecting them.
　カメラ１０１は、ＣＭＯＳイメージセンサ１０１１、通信ＩＣ（Ｔｘ）１０１２、コネクタ部（Ｔｘ）１０１３で構成される。 The camera 101 is composed of a CMOS image sensor 1011, a communication IC (Tx) 1012, and a connector section (Tx) 1013.
　また、画像処理装置１０２は、コネクタ部（Ｒｘ）１０２１、通信ＩＣ（Ｒｘ）１０２２、マイコン１０２４で構成される。なお、マイコン１０２４は、一般的なＣＰＵ（Central Processing Unit）等の処理プロセッサと記憶装置で構成され、処理プロセッサが、記憶装置からそれぞれの機能を実現するプログラムや情報を読み出して、例えばカメラで撮像した対象物の認識処理をソフトウェア処理することにより実行し、その認識結果に応じて車両制御を行う。 Also, the image processing device 102 is composed of a connector section (Rx) 1021 , a communication IC (Rx) 1022 and a microcomputer 1024 . The microcomputer 1024 is composed of a processor such as a general CPU (Central Processing Unit) and a storage device. Recognition processing of the target object is executed by software processing, and vehicle control is performed according to the recognition result.
　Ｓｅｒｄｅｓを使用した通信手法では、そのままでは、基板、コネクタ、伝送線をはじめとした通信経路上の損失などで信号が劣化し、正常な通信を行うことが難しい。そのため既存技術では、受信側の通信ＩＣである通信ＩＣ（Ｒｘ）１０２２に、通信品質の確保のためContinuous Time Linear Equalizer（以下、ＣＴＬＥ）、Decision Feedback Equalizer（以下、ＤＦＥ）などの等化器の機能が搭載されており、この機能を使用して信号波形が劣化した場合に波形の補正を行っている。なお、図１では、通信ＩＣ（Ｒｘ）１０２２内のＤＦＥをノイズ低減フィルタ１０４としている。 With the communication method using Serdes, if left as is, the signal will deteriorate due to losses on the communication path, including the board, connector, and transmission line, making normal communication difficult. Therefore, in the existing technology, the communication IC (Rx) 1022, which is the communication IC on the receiving side, is equipped with an equalizer such as a Continuous Time Linear Equalizer (hereinafter referred to as CTLE) and a Decision Feedback Equalizer (hereinafter referred to as DFE) to ensure communication quality. This function is used to correct the waveform when the signal waveform is degraded. Note that the DFE in the communication IC (Rx) 1022 is used as the noise reduction filter 104 in FIG.
　通常の補正であればＣＴＬＥで対応可能であるが、ＣＴＬＥはノイズ成分も増幅する可能性が有るため、高速通信ではＣＴＬＥに加えてＤＦＥを併用する傾向にある。ＤＦＥはフィードバック制御を用いた補正を行うが、車両の周囲に電波塔や無線通信エリアなどがある場合、また近傍で無線通信が行われた場合等、短時間に大きな外部ノイズを受ける場合がある。この外部ノイズが等化器の補正可能範囲を超えた場合、等化器による補正制御が間に合わず通信エラーが発生する可能性が有る。 Normal correction can be handled with CTLE, but CTLE may also amplify noise components, so there is a tendency to use DFE in addition to CTLE in high-speed communication. The DFE performs correction using feedback control, but if there is a radio tower or wireless communication area around the vehicle, or if wireless communication is performed nearby, it may receive large external noise in a short period of time. . If this external noise exceeds the correctable range of the equalizer, there is a possibility that the correction control by the equalizer will not keep up and a communication error will occur.
　そこで、本実施例では、外界情報からノイズ源を確認することで、ノイズ源に応じたノイズ低減フィルタのパラメータを変更することで信頼性の高い通信処理を可能とする。以下、本実施例の詳細について説明する。 Therefore, in this embodiment, by confirming the noise source from the external world information, highly reliable communication processing is made possible by changing the parameters of the noise reduction filter according to the noise source. The details of this embodiment will be described below.
　図２は本実施例における画像処理システムの機能構成ブロック図である。図２において、図１と同じ機能は同じ符号を付し、その説明は省略する。図２において、図１と異なる点は、ステレオカメラを用いた構成としておりカメラと伝送線とノイズ低減フィルタが２系統ある点と、信号補正処理部１０２５を有している点である。 FIG. 2 is a functional configuration block diagram of the image processing system in this embodiment. In FIG. 2, the same functions as in FIG. 1 are given the same reference numerals, and the description thereof is omitted. 2 differs from FIG. 1 in that it has a configuration using a stereo camera and has two systems of cameras, transmission lines, and noise reduction filters, and that it has a signal correction processing unit 1025 .
　図２において、画像処理システム１００は、近方の車両検知手段としてステレオカメラを用いた視差画像の処理による立体物検知を行う画像処理システムである。なお、例えば単眼カメラのみや、単眼カメラとレーダを組み合わせた立体物検知などによっても実施可能である。また、３つ以上の複数のカメラを用いてもよい。 In FIG. 2, the image processing system 100 is an image processing system that performs three-dimensional object detection by processing parallax images using a stereo camera as means for detecting nearby vehicles. It should be noted that, for example, detection of a three-dimensional object using only a monocular camera or a combination of a monocular camera and radar can also be used. Also, three or more cameras may be used.
　ステレオ画像は、カメラ１０１およびカメラ１０９によって撮像される。画像処理装置１０２は伝送線１０３および１１０を経由して画像データを受け取る。撮像された画像データは、ノイズ低減フィルタ１０４および１１１を通して認識処理部１０２６に送られ、視差画像の処理による対象物との距離情報の算出、及び対象物の認識処理を行う。 A stereo image is captured by camera 101 and camera 109 . Image processor 102 receives image data via transmission lines 103 and 110 . The captured image data is sent to the recognition processing unit 1026 through the noise reduction filters 104 and 111, and the distance information to the object is calculated by processing the parallax image and the object is recognized.
　一方、撮像された画像データの補正を行うノイズ低減フィルタ１０４および１１１は、信号補正処理部１０２５によって補正量を制御される。すなわち、ノイズ低減フィルタ１０４および１１１からの撮像された画像データは、外界情報としてノイズ源検知処理部１０５に送られる。ノイズ源検知処理部１０５では、外界情報と、あらかじめ記憶領域１０７に収められたノイズ源情報保存部１０８に登録された対象物の画像とを参照して、外界情報の画像データ内にノイズ源となる対象物が含まれるか否かを判定する。そして、対象物が含まれる場合、ノイズ源検知処理部１０５は、ノイズ源情報保存部１０８に登録された、その対象物に対応した周波数やノイズレベルなどのノイズ源の詳細情報を取得する。 On the other hand, the noise reduction filters 104 and 111 that correct captured image data have their correction amounts controlled by the signal correction processing unit 1025 . That is, the captured image data from the noise reduction filters 104 and 111 are sent to the noise source detection processing unit 105 as external world information. The noise source detection processing unit 105 refers to the external world information and the image of the object registered in the noise source information storage unit 108 stored in advance in the storage area 107, and the noise source and the noise source are included in the image data of the external world information. It is determined whether or not a target object is included. Then, when the object is included, the noise source detection processing unit 105 acquires detailed information of the noise source such as the frequency and noise level corresponding to the object registered in the noise source information storage unit 108 .
　カメラで取得する外界情報としては、例えば、鉄塔、雷、携帯電話基地局、パンタグラフや、無線通信を行うタクシー、中継車両、軍用車両、また、高圧送電などの標識等の画像であり、ノイズ源検知処理部１０５は、それらをノイズ源とする情報を取得する。 Examples of the external world information acquired by the camera include images of steel towers, lightning, mobile phone base stations, pantographs, radio communication taxis, relay vehicles, military vehicles, and signs such as high-voltage power transmission, and noise sources. The detection processing unit 105 acquires information using them as noise sources.
　取得したノイズ源の詳細情報と、認識処理部１０２６で算出したノイズ源までの距離情報は、ノイズ低減パラメータ決定部１０６に送られる。ノイズ低減パラメータ決定部１０６では、この送られた情報から、ノイズ低減フィルタの補正量を決定し、ノイズ低減フィルタのパラメータを補正する。すなわち、ノイズ低減フィルタのパラメータをフィードフォワード制御により補正する。 The acquired detailed information of the noise source and the distance information to the noise source calculated by the recognition processing unit 1026 are sent to the noise reduction parameter determination unit 106 . The noise reduction parameter determination unit 106 determines the correction amount of the noise reduction filter from the sent information, and corrects the parameters of the noise reduction filter. That is, the parameters of the noise reduction filter are corrected by feedforward control.
　このように、信号補正処理部１０２５は、まず撮像された画像を確認し、画像内にノイズ源となる対象を確認した場合、フィルタの補正を開始する。なお、撮像された画像にノイズ源が確認されなければフィルタの補正処理は実施しない。そして、ノイズ源の周波数やノイズレベルを外界情報の画像から特定し、周波数、ノイズレベル、ノイズ源までの距離情報の何れか１つ、または複数の情報によって、ノイズ低減フィルタの補正量と補正タイミングを変更するよう信号を送信し、外部ノイズによる通信エラーを抑制する。 In this way, the signal correction processing unit 1025 first confirms the captured image, and when it confirms a noise source target in the image, starts correcting the filter. It should be noted that filter correction processing is not performed unless a noise source is confirmed in the captured image. Then, the frequency and noise level of the noise source are specified from the image of the external information, and the correction amount and correction timing of the noise reduction filter are determined by one or more of the frequency, noise level, and distance information to the noise source. to suppress communication errors caused by external noise.
　なお、ノイズ低減フィルタの補正は、例えばＳｅｒｄｅｓ内のイコライザを用いて実施する。通常Ｓｅｒｄｅｓは、フィルタ補正用としてＣＴＬＥとＤＦＥの２種のイコライザをもつ。またＤＦＥのノイズ補正パラメータは、レジスタ設定を変更することで、ノイズの補正量を設定することができる。例えば、ノイズ補正量を設定するレジスタ領域を、Ｉ２Ｃ（Inter-Integrated Circuit）などのＳｅｒｄｅｓが備えるインターフェースを用いて書き換えることで、あらかじめ設定した補正量を一定の範囲でリアルタイムに補正することができる。 Note that the noise reduction filter is corrected using, for example, an equalizer in Serdes. Serdes usually have two types of equalizers, CTLE and DFE, for filter correction. The DFE noise correction parameter can set the amount of noise correction by changing the register setting. For example, by rewriting the register area for setting the noise correction amount using an interface provided by Serdes such as I2C (Inter-Integrated Circuit), the preset correction amount can be corrected in real time within a certain range.
　このように、本実施例によれば、外界情報である撮像した画像からノイズ源を判別し、ノイズ源に応じてノイズ低減フィルタのパラメータを変更することで通信エラーの発生を抑制し、信頼性の高い通信処理が可能となる。これにより、画像データの伝送線上での外部ノイズによる通信エラーの低減が可能な画像処理システム、画像処理装置、及び画像処理方法を提供できる。 As described above, according to the present embodiment, the noise source is discriminated from the captured image, which is the external information, and the parameters of the noise reduction filter are changed according to the noise source, thereby suppressing the occurrence of communication errors and increasing the reliability. high communication processing becomes possible. Accordingly, it is possible to provide an image processing system, an image processing apparatus, and an image processing method capable of reducing communication errors due to external noise on the image data transmission line.
　図３は本実施例における画像処理システムの機能構成ブロック図である。図３において、図２と同じ機能は同じ符号を付し、その説明は省略する。図３において、図２と異なる点は、ノイズ低減パラメータ決定部１０６への入力として、車速情報１１２を設けた点である。 FIG. 3 is a functional configuration block diagram of the image processing system in this embodiment. In FIG. 3, the same functions as in FIG. 2 are given the same reference numerals, and the description thereof is omitted. FIG. 3 differs from FIG. 2 in that vehicle speed information 112 is provided as an input to noise reduction parameter determining section 106 .
　車速が早いと単位時間当たりの移動距離が大きくなる。そのために、図３に示すように、ノイズ低減パラメータ決定部１０６において、車速情報により車速が早い場合は、ノイズ低減フィルタの補正タイミングの間隔を短くする等の処理を行う。 　The higher the vehicle speed, the longer the distance traveled per unit time. Therefore, as shown in FIG. 3, the noise reduction parameter determination unit 106 performs processing such as shortening the interval between correction timings of the noise reduction filter when the vehicle speed is high according to the vehicle speed information.
　このように、本実施例によれば、実施例１の効果に加えて、より最適なノイズ低減フィルタの補正タイミングでノイズ低減フィルタのパラメータを変更することができ、より信頼性の高い通信処理が可能となる。 As described above, according to the present embodiment, in addition to the effects of the first embodiment, the parameters of the noise reduction filter can be changed at more optimal correction timing of the noise reduction filter, and more reliable communication processing can be performed. It becomes possible.
　以上実施例について説明したが、本発明は上記した実施例に限定されるものではなく、様々な変形例が含まれる。例えば、上記した実施例では画像データの伝送におけるノイズ低減について説明したが、画像データに限定するものではなく、音声データについても適用可能である。また、上記した実施例は本発明を分かりやすく説明するために詳細に説明したものであり、必ずしも説明した全ての構成を備えるものに限定されるものではない。 Although the embodiments have been described above, the present invention is not limited to the above embodiments, and includes various modifications. For example, although noise reduction in the transmission of image data has been described in the above embodiments, the present invention is not limited to image data and can also be applied to audio data. Also, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations.
１００：画像処理システム、１０１、１０９：カメラ、１０２：画像処理装置、１０３、１１０：伝送線、１０４、１１１：ノイズ低減フィルタ、１０５：ノイズ源検知処理部、１０６：ノイズ低減パラメータ決定部、１０７：記憶領域、１０８：ノイズ源情報保存部、１１２：車速情報、１０１１：ＣＭＯＳイメージセンサ、１０１２：通信ＩＣ（Ｔｘ）、１０１３：コネクタ部（Ｔｘ）、１０２１：コネクタ部（Ｒｘ）、１０２２：通信ＩＣ（Ｒｘ）、１０２４：マイコン、１０２５：信号補正処理部、１０２６認識処理部 100: image processing system, 101, 109: camera, 102: image processing device, 103, 110: transmission line, 104, 111: noise reduction filter, 105: noise source detection processor, 106: noise reduction parameter determination unit, 107 108: Noise source information storage unit 112: Vehicle speed information 1011: CMOS image sensor 1012: Communication IC (Tx) 1013: Connector unit (Tx) 1021: Connector unit (Rx) 1022: Communication IC (Rx), 1024: microcomputer, 1025: signal correction processing section, 1026 recognition processing section

Claims (10)

  1. 　外界情報を取得する車載カメラと、前記車載カメラとは別体で設置され前記車載カメラで取得した画像データを処理する画像処理装置と、前記車載カメラと前記画像処理装置をつなぎ前記画像データを伝送する伝送線からなる画像処理システムであって、
    　前記画像処理装置は、前記伝送線でのノイズを低減するノイズ低減フィルタと、処理プロセッサを有し、
    　前記処理プロセッサは、
    　前記伝送線に対するノイズ源としてあらかじめ登録された対象物が前記画像データに含まれるか否かを判定するノイズ源検知処理部と、
    　前記画像データに前記ノイズ源が含まれる場合に、あらかじめ登録された前記ノイズ源の周波数またはノイズレベルに応じて、前記ノイズ低減フィルタのパラメータを変更するノイズ低減パラメータ決定部を有することを特徴とする画像処理システム。     an in-vehicle camera that acquires external information; an image processing device that is installed separately from the in-vehicle camera and processes image data acquired by the in-vehicle camera; and a transmission of the image data that connects the in-vehicle camera and the image processing device An image processing system comprising a transmission line that
    The image processing device has a noise reduction filter for reducing noise on the transmission line and a processor,
    The processing processor
    a noise source detection processing unit that determines whether an object registered in advance as a noise source for the transmission line is included in the image data;
    A noise reduction parameter determining unit that changes parameters of the noise reduction filter according to a pre-registered frequency or noise level of the noise source when the noise source is included in the image data. image processing system.
  2. 　請求項１に記載の画像処理システムにおいて、
    　前記車載カメラは複数であって、複数の前記車載カメラからのそれぞれの画像データを伝送する複数の伝送線を有し、
    　前記複数の伝送線からのそれぞれの画像データのノイズを低減する複数の前記ノイズ低減フィルタを有し、
    　前記処理プロセッサは、
    　複数の前記車載カメラからの視差画像の処理による対象物との距離情報を算出し、
    　前記ノイズ低減パラメータ決定部は、前記対象物との距離情報に応じて、複数の前記ノイズ低減フィルタのそれぞれのパラメータを補正することを特徴とする画像処理システム。     In the image processing system according to claim 1,
    a plurality of the vehicle-mounted cameras having a plurality of transmission lines for transmitting respective image data from the plurality of the vehicle-mounted cameras;
    a plurality of said noise reduction filters for reducing noise in respective image data from said plurality of transmission lines;
    The processing processor
    Calculating distance information with an object by processing parallax images from the plurality of in-vehicle cameras,
    The image processing system, wherein the noise reduction parameter determination unit corrects parameters of each of the plurality of noise reduction filters according to distance information from the object.
  3. 　請求項１に記載の画像処理システムにおいて、
    　前記ノイズ低減パラメータ決定部は、車速情報を用いて前記ノイズ低減フィルタのパラメータの補正タイミングを決定することを特徴とする画像処理システム。     In the image processing system according to claim 1,
    The image processing system according to claim 1, wherein the noise reduction parameter determination unit determines a correction timing of the parameters of the noise reduction filter using vehicle speed information.
  4. 　請求項１に記載の画像処理システムにおいて、
    　前記画像データの伝送は、Ｓｅｒｄｅｓを用いたシリアル伝送であって、
    　前記ノイズ低減フィルタは、通信ＩＣ内の等化器であることを特徴とする画像処理システム。     In the image processing system according to claim 1,
    The transmission of the image data is serial transmission using Serdes,
    The image processing system, wherein the noise reduction filter is an equalizer within a communication IC.
  5. 　外界情報を取得する車載カメラからの画像データを伝送線を介して入力し処理する画像処理装置であって、
    　前記伝送線でのノイズを低減するノイズ低減フィルタと、処理プロセッサを有し、
    　前記処理プロセッサは、
    　前記伝送線に対するノイズ源としてあらかじめ登録された対象物が前記画像データに含まれるか否かを判定するノイズ源検知処理部と、
    　前記画像データに前記ノイズ源が含まれる場合に、あらかじめ登録された前記ノイズ源の周波数またはノイズレベルに応じて、前記ノイズ低減フィルタのパラメータを変更するノイズ低減パラメータ決定部を有することを特徴とする画像処理装置。     An image processing device that inputs and processes image data from an in-vehicle camera that acquires external world information via a transmission line,
    a noise reduction filter for reducing noise in the transmission line and a processing processor;
    The processing processor
    a noise source detection processing unit that determines whether or not an object registered in advance as a noise source for the transmission line is included in the image data;
    A noise reduction parameter determining unit that changes parameters of the noise reduction filter according to a pre-registered frequency or noise level of the noise source when the noise source is included in the image data. Image processing device.
  6. 　請求項５に記載の画像処理装置において、
    　前記車載カメラは複数であって、それぞれの前記車載カメラからの画像データを伝送する複数の伝送線からのそれぞれの画像データのノイズを低減する複数の前記ノイズ低減フィルタを有し、
    　前記処理プロセッサは、
    　複数の前記車載カメラからの視差画像の処理による対象物との距離情報を算出し、
    　前記ノイズ低減パラメータ決定部は、前記対象物との距離情報に応じて、複数の前記ノイズ低減フィルタのそれぞれのパラメータを補正することを特徴とする画像処理装置。     In the image processing device according to claim 5,
    a plurality of the vehicle-mounted cameras, each having a plurality of noise reduction filters for reducing noise in image data from a plurality of transmission lines that transmit image data from the vehicle-mounted cameras;
    The processing processor
    Calculating distance information with an object by processing parallax images from the plurality of in-vehicle cameras,
    The image processing apparatus, wherein the noise reduction parameter determination unit corrects parameters of each of the plurality of noise reduction filters according to distance information from the object.
  7. 　請求項５に記載の画像処理装置において、
    　前記ノイズ低減パラメータ決定部は、車速情報を用いて前記ノイズ低減フィルタのパラメータの補正タイミングを決定することを特徴とする画像処理装置。     In the image processing device according to claim 5,
    The image processing apparatus, wherein the noise reduction parameter determination unit determines correction timing of the parameters of the noise reduction filter using vehicle speed information.
  8. 　外界情報を取得する車載カメラからの画像データを伝送線を介して入力し処理する画像処理方法であって、
    　前記伝送線に対するノイズ源としてあらかじめ登録された対象物が前記画像データに含まれるか否かを判定し、
    　前記画像データに前記ノイズ源が含まれる場合に、あらかじめ登録された前記ノイズ源の周波数またはノイズレベルに応じて、前記伝送線でのノイズを低減するノイズ低減フィルタのパラメータを変更することを特徴とする画像処理方法。     An image processing method for inputting and processing image data from an in-vehicle camera that acquires external world information via a transmission line,
    determining whether or not the image data includes an object registered in advance as a noise source for the transmission line;
    When the noise source is included in the image data, parameters of a noise reduction filter for reducing noise on the transmission line are changed according to a pre-registered frequency or noise level of the noise source. image processing method.
  9. 　請求項８に記載の画像処理方法において、
    　前記車載カメラおよび伝送線は複数であって、
    　複数の前記車載カメラからの視差画像の処理による対象物との距離情報を算出し、
    　前記画像データに前記ノイズ源が含まれる場合に、前記対象物との距離情報に応じて、複数の前記伝送線からのそれぞれの画像データのノイズを低減する複数の前記ノイズ低減フィルタのそれぞれのパラメータを補正することを特徴とする画像処理方法。     In the image processing method according to claim 8,
    The vehicle-mounted camera and the transmission line are plural,
    Calculating distance information with an object by processing parallax images from the plurality of in-vehicle cameras,
    Parameters for each of the plurality of noise reduction filters that reduce noise in each of the image data from the plurality of transmission lines according to distance information from the object when the image data includes the noise source. An image processing method characterized by correcting the
  10. 　請求項８に記載の画像処理方法において、
    　前記画像データに前記ノイズ源が含まれる場合に、車速情報を用いて前記ノイズ低減フィルタのパラメータの補正タイミングを決定することを特徴とする画像処理方法。     In the image processing method according to claim 8,
    An image processing method, wherein when the noise source is included in the image data, vehicle speed information is used to determine correction timing of parameters of the noise reduction filter.
PCT/JP2022/004035 2021-07-02 2022-02-02 Image processing system, image processing device, and image processing method WO2023276221A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE112022002245.0T DE112022002245T5 (en) 2021-07-02 2022-02-02 IMAGE PROCESSING SYSTEM, IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD
JP2023531363A JPWO2023276221A1 (en) 2021-07-02 2022-02-02

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021110584 2021-07-02
JP2021-110584 2021-07-02

Publications (1)

Publication Number Publication Date
WO2023276221A1 true WO2023276221A1 (en) 2023-01-05

Family

ID=84691063

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/004035 WO2023276221A1 (en) 2021-07-02 2022-02-02 Image processing system, image processing device, and image processing method

Country Status (3)

Country Link
JP (1) JPWO2023276221A1 (en)
DE (1) DE112022002245T5 (en)
WO (1) WO2023276221A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002351488A (en) * 2001-05-28 2002-12-06 Matsushita Electric Ind Co Ltd Noise canceller and on-vehicle system
JP2009194438A (en) * 2008-02-12 2009-08-27 Canon Inc Noise reduction device, and imaging apparatus
JP2010074899A (en) * 2008-09-17 2010-04-02 Meidensha Corp Device for measuring pantograph horizontal acceleration by image processing, and measuring method
JP2011050582A (en) * 2009-09-02 2011-03-17 Hitachi Medical Corp Noise reduction system for operation room
WO2012099190A1 (en) * 2011-01-21 2012-07-26 株式会社 日立ハイテクノロジーズ Charged particle beam device, and image analysis device
JP2020016838A (en) * 2018-07-27 2020-01-30 株式会社Ｊｖｃケンウッド Noise reduction device, noise reduction method, and noise reduction program
WO2020100529A1 (en) * 2018-11-14 2020-05-22 オムロン株式会社 Information processing device and control method for information processing device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5433610B2 (en) 2011-03-04 2014-03-05 日立オートモティブシステムズ株式会社 In-vehicle camera device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002351488A (en) * 2001-05-28 2002-12-06 Matsushita Electric Ind Co Ltd Noise canceller and on-vehicle system
JP2009194438A (en) * 2008-02-12 2009-08-27 Canon Inc Noise reduction device, and imaging apparatus
JP2010074899A (en) * 2008-09-17 2010-04-02 Meidensha Corp Device for measuring pantograph horizontal acceleration by image processing, and measuring method
JP2011050582A (en) * 2009-09-02 2011-03-17 Hitachi Medical Corp Noise reduction system for operation room
WO2012099190A1 (en) * 2011-01-21 2012-07-26 株式会社 日立ハイテクノロジーズ Charged particle beam device, and image analysis device
JP2020016838A (en) * 2018-07-27 2020-01-30 株式会社Ｊｖｃケンウッド Noise reduction device, noise reduction method, and noise reduction program
WO2020100529A1 (en) * 2018-11-14 2020-05-22 オムロン株式会社 Information processing device and control method for information processing device

Also Published As

Publication number Publication date
DE112022002245T5 (en) 2024-03-14
JPWO2023276221A1 (en) 2023-01-05

Similar Documents

Publication Publication Date Title
US10739812B2 (en) Communication system and communication method
CN107810506B (en) Remote viewing system with privacy protection
US20120290753A1 (en) Connection method for bus controllers and communication system
US9596225B2 (en) Out-of-vehicle device interface apparatus and method for protecting in-vehicle network
US9136945B2 (en) Vehicle communication system for visible light communication and optical networking and communication method thereof
CN111183707A (en) Wireless communication device
US20110137510A1 (en) Communication interface conversion device
WO2018025607A1 (en) On-board camera device, on-board camera adjustment device, and on-board camera adjustment system
WO2023276221A1 (en) Image processing system, image processing device, and image processing method
US20130329017A1 (en) Vehicle-mounted camera device
US20240104048A1 (en) Transmitting Apparatus, Receiving Apparatus, Parameter Adjustment Method, SerDes Circuit, and Electronic Device
US10343606B2 (en) Using parallel data lines for GPIO purposes
US20220060282A1 (en) Communication apparatus and communication method, and program
US20220303056A1 (en) Communication device, communication method, and program
KR20190124706A (en) Transmission device and communication system
JP2004208162A (en) System and device for transmitting camera video
US20020097789A1 (en) Transceiver with means for error management
US20210320688A1 (en) System and Method for Establishing an Intervehicle Communication
US20230155750A1 (en) Communication device and communication system
WO2018192585A1 (en) Transmission method and transmission device
KR20210134815A (en) A telematics communication unit and a server controlling it in a next-generation mobile communication system
JP7464801B2 (en) IMAGE PROCESSING APPARATUS AND IMAGE DATA TRANSMISSION METHOD
CN110554633A (en) Control system and automobile
JP7122547B2 (en) Video processing device and video processing system
US20210286753A1 (en) Image sensor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22832389

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023531363

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 112022002245

Country of ref document: DE