WO2020020446A1 - Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses - Google Patents

Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses Download PDF

Info

Publication number
WO2020020446A1
WO2020020446A1 PCT/EP2018/070074 EP2018070074W WO2020020446A1 WO 2020020446 A1 WO2020020446 A1 WO 2020020446A1 EP 2018070074 W EP2018070074 W EP 2018070074W WO 2020020446 A1 WO2020020446 A1 WO 2020020446A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
foggy
fog
fog density
model
Prior art date
Application number
PCT/EP2018/070074
Other languages
English (en)
Inventor
Hiroaki Shimizu
Dengxin DAI
Christos SAKARIDIS
Luc Van GOOL
Original Assignee
Toyota Motor Europe
Eth Zurich
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toyota Motor Europe, Eth Zurich filed Critical Toyota Motor Europe
Priority to US17/258,519 priority Critical patent/US11941815B2/en
Priority to PCT/EP2018/070074 priority patent/WO2020020446A1/fr
Priority to CN201880095853.4A priority patent/CN112513928A/zh
Publication of WO2020020446A1 publication Critical patent/WO2020020446A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present Invention relates to the field of image processing, and more precisely to the semantic segmentation of Images In which fog may appear.
  • Semantic segmentation is a method for determining the types of objects which are visible (or partially visible) in an image.
  • the image may be acquired by a camera mounted In a vehicle. Semantic segmentation of such an image allows distinguishing other cars, pedestrians, traffic lanes, etc. Therefore, semantic segmentation Is particularly useful for self-driving vehicles and for other types of automated systems.
  • Semantic segmentation methods typically use models such as neural networks or convolutional neural network to perform the segmentation. These models have to be trained.
  • Training a model typically comprises Inputting known Images to the model. For these images, a predetermined semantic segmentation Is already known (an operator may have prepared the predetermined semantic segmentations of each Image by annotating the images). The output of the model Is then evaluated in view of the predetermined semantic segmentation, and the parameters of the model are adjusted If the output of the model differs from the predetermined semantic segmentation of an image. It follows that in order to train a semantic segmentation model, a large number of images and predetermined semantic segmentations are necessary.
  • Fog is a typical example of adverse weather which degrades the visibility of a scene significantly, according to the density of the fog.
  • obtaining a semantic segmentation Is easier to perform on a dear Image and the semantic segmentation of a dear Image can be used to train a model to which the Image with synthetic fog has been Inputted.
  • the present invention overcomes one or more deficiencies of the prior art by proposing a method for training a model to be used for semantic segmentation of images, comprising:
  • c classifying a second plurality of Images using the classification model of step b according to the fog density in the images, so as to obtain a second plurality of foggy Images each having a fog density comprised within a first fog density threshold and a second fog density threshold, d - obtaining a third plurality of foggy Images each having a fog density of synthetic fog comprised within the first fog density threshold and the second fog density threshold,
  • step f applying the semantic segmentation model of step e to the second plurality of foggy Images to obtain semantic segmentations of the second plurality of foggy images
  • step e g - obtaining a fourth plurality of foggy images each having a fog density of synthetic fog comprised within a third fog density threshold and a fourth fog density threshold, the third and fourth fog density thresholds being both greater than the first and second fog density thresholds, h - training the semantic segmentation model of step e using:
  • step f the second plurality of foggy Images and the semantic segmentations of the plurality of foggy Images obtained In step f.
  • a predefined semantic segmentation is a semantic segmentation which may have been obtained through an operator's annotation.
  • a predefined semantic segmentation Is mentioned, it Is implied that the corresponding (foggy) Image can be used for training models or neural networks.
  • an operator may have prepared the annotations leading to the semantic segmentation, It is possible that these annotations are of very good quality.
  • training a model In step e using predefined semantic segmentations may provide good training of the semantic segmentation model.
  • the first and second density thresholds may correspond to light fog or at least lighter fog than the fog of the third and fourth fog density thresholds which correspond to dense fog or at least denser fog.
  • the person skilled In the art may choose the fog density thresholds according to the application to differentiate light fog from dense fog.
  • obtaining an Image having chosen fog density of synthetic fog may be performed according to the method disclosed In document Semantic foggy scene understanding with synthetic data" (International Journal of Computer Vision, Sakaridis, C., Dai, D., Van Gool, L, 2018).
  • This method allows using a parameter to select the density of the fog: hence, training the model of step b can use the parameter used for adding synthetic fog.
  • the images of the second plurality of image do not contain synthetic fog, they may be images of scenes which may contain real fog.
  • Using the trained model of step b it is possible to select only the images which contain real light fog. Because the model of step e has been trained using predefined semantic segmentations of images having synthetic light fog, and because the second plurality of foggy images only contains Images with light fog, the semantic segmentations obtained in step f are also of good quality.
  • step h the data Inputted to the model in step h is of good quality and allows adapting the model to perform semantic segmentation on denser fog.
  • synthetic fog and especially dense synthetic fog, may contain artefacts. It is therefore interesting to temper the training in step h using real fog (the second plurality of foggy images).
  • step h comprises inputting, to the semantic segmentation model of step e, a mix of Images from the second plurality of foggy Images and of Images from the fourth plurality of foggy Images so as to obtain a stream of Images to be fed to the model, wherein the stream of Images comprises a greater proportion of Images from the fourth plurality of foggy images than images from the second plurality of foggy images.
  • step h comprises minimizing the following value :
  • l is the number of image in the fourth plurality of foggy images
  • x" is an image of index i in the fourth plurality of foggy images
  • yt is the predefined semantic segmentation of image x
  • L(x,y) is the cross entropy loss function of x and y
  • f"(c") is the output of the semantic segmentation model
  • u is the number of images in the second plurality of foggy images
  • xj is an image of the second plurality of foggy Images
  • 3 ⁇ 4 is the semantic segmentation of image x" obtained in step f.
  • the loss L(x,y) is calculated after each processing of an image and the parameters of the models are adjusted to minimize this loss.
  • this training leads, in the end, to minimizing the above equation:
  • obtaining a foggy Image for the first, third or fourth plurality of foggy images comprises:
  • the transmittance map better reflects the transmittance of a real foggy image (obtained using a camera acquiring a picture of a foggy scene).
  • a (filtered) transmittance map determines the amount of scene radiance which may reach the camera (the viewpoint of the Image).
  • Transmittance maps are notably disclosed in document "Semantic foggy scene understanding with synthetic data".
  • the transmittance map may have the same resolution as the Image: a pixel of the Image has an associated transmittance value.
  • the depth map may also have the same resolution as the Image.
  • transmittance maps and radiance are terminologies which are known to the person skilled In the art and which are further described, by way of example, in documents "Single image haze removal using dark channel prior” (He, K., Sun, J., Tang, X., IEEE Transactions on Pattern Analysis and Machine Intelligence 33(12) (2011) 2341-2353) and “Bayesian defogging” (Nishlno, K., Kratz, L, Lombardi, S., International Journal of Computer Vision 98(3) (2012) 263-278).
  • filtering the transmittance map comprises applying a bilateral filter.
  • Bilateral filtering allows smoothing the transmittance while preserving the edges, and, in the present Invention, the edges between separate semantic objects.
  • the transmittance map is defined as:
  • l(q) is the depth of pixel q in the depth map.
  • the attenuation coefficient b may be chosen to reduce the visibility and increase the density of the fog according to the depth.
  • simulating the appearance of fog on the initial image using the filtered transmittance map comprises calculating the color value /(x) of each pixel as:
  • the depth map is obtained using a depth-by-stereo method and optionally an outlier suppression method.
  • the outlier suppression method may comprise a suppression of holes.
  • a depth-by-stereo method may comprise acquiring two images using two cameras which are separated horizontally by a distance.
  • the outlier suppression method may comprise a suppression of missing values in the depth map obtained from the depth-by stereo method. These holes are due to the failure of the Initial depth-by-stereo method due to occlusion: areas which are visible on one camera but ocduded in the other camera used for depth-by-stereo (this results in missing depth values).
  • the invention also proposes a system for training a model to be used for semantic segmentation of Images, comprising:
  • a module F for applying the semantic segmentation model of step e to the second plurality of foggy images to obtain semantic segmentations of the second plurality of foggy images
  • a module G for obtaining a fourth plurality of foggy Images each having a fog density of synthetic fog comprised within a third fog density threshold and a fourth fog density threshold, the third and fourth fog density thresholds being both greater than the first and second fog density thresholds,
  • This system may be configured to perform all the embodiments of the method as defined above.
  • the steps of the above methods are determined by computer program Instructions.
  • the Invention Is also directed to a computer program for executing the steps of the methods as described above when this program Is executed by a computer.
  • This program can use any programming language and take the form of source code, object code or a code Intermediate between source code and object code, such as a partially compiled form, or any other desirable form.
  • the Invention Is also directed to a computer-readable information medium containing instructions of a computer program as described above.
  • the Information medium can be any entity or device capable of storing the program.
  • the medium can include storage means such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or magnetic storage means, for example a diskette (floppy disk) or a hard disk.
  • the information medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute the method In question or to be used in its execution.
  • FIG. 1 is a block diagram of an exemplary method
  • FIG. 2 is a schematic diagram of an exemplary system.
  • a method for training a model Is shown on figure 1.
  • This model may be, Initially, a neural network or a convolutional neural network which may have been conceived to perform semantic segmentation on images. However, Initially, the model has not been trained to perform semantic segmentation on foggy Images.
  • the Images which may be processed by the model may be photographs taken by image sensors.
  • a plurality of objects may be visible on these images, preferably objects of different types which may or may not overlap.
  • the images show a scene which may be visible from a vehicle on a road, for example in a street.
  • a first plurality of foggy images having different densities of synthetic fog.
  • This step may be performed using a plurality of images which do not have fog on them.
  • the synthetic fog may be added as follows on each Initial Image of a plurality of Initial Images.
  • the Initial Image In an RGB Image.
  • a depth map has been obtained for this Image.
  • the depth map Is obtained using a depth-by-stereo method and an outlier suppression method.
  • two cameras may be used to acquire the Image, typically a left camera and a right camera (the actual image to be processed may be the left Image or the right Image).
  • outliers may be removed using a known method, for example the method disclosed In document "Semantic foggy scene understanding with synthetic data" (International Journal of Computer Vision, Sakaridis, C., Dal, D., Van Gool, L, 2018).
  • This semantic segmentation of the Image may have been prepared by an operator annotating each portion of the image (groups of pixels) so as to Indicate their type. For example, this results in a segmentation of the Image which indicates whether an object is a car, a pedestrian, the road, a lane, etc.
  • the segmentation may be mathematically represented as a function ft:
  • c Is the total number of semantic classes (or types) In the scene (or In the image).
  • a transmittance map is elaborated. This step may comprise adding "synthetic fog" using the following equation: with: i(q) being the transmittance of pixel q in the transmittance map,
  • l(q) is the depth of pixel q in the depth map.
  • the coefficient b is known from the field of meteorology.
  • the visibility of an object also known under the acronym “MOR: Meteorological optical Range” is defined as the maximum distance from the camera (or viewpoint) for which the transmission is superior or equal to 0.05. This implies that:
  • the attenuation coefficient may be chosen superior or equal to Z.OOexlO ⁇ m 1 , this value corresponding to light fog.
  • filtering the transmittance map may comprise using a bilateral filter.
  • Typical values for the parameters may be:
  • This simulation of the appearance of fog on the Image using the filtered transmittance map comprises calculating the color value /(x) of each pixel as:
  • /?(x) is a value which has three components in the Red-Green-Blue space, each value being an integer comprised between 0 and 255.
  • the atmospheric light is considered to be constant, for example the atmospheric light Is a constant value estimated from the Image.
  • the pixels with highest Intensity values can be taken as the atmospheric light
  • the image /(x) is also expressed In the Red-Green-Blue space and It Is to being called a foggy Image.
  • a first plurality of foggy images 100 may be obtained.
  • various values for the coefficient b for example a random value above 2.996xl0 '3 m '1 , various densities of synthetic fog are visible on the Images.
  • the first plurality of foggy Images 100 Is used to train a model for estimated fog density In step S02.
  • This model Is configured to receive as Input an Image, and to output a value (for example a coefficient b ) associated with the Image.
  • step S03 a second plurality of Images 101 Is classified using the model trained in step S02.
  • the second plurality of Images 101 comprises Images which comprise real fog.
  • these Images are photographs taken outside on which real fog may be present at different densities of fog.
  • a first fog density threshold and a second fog density threshold Prior to performing this classification, a first fog density threshold and a second fog density threshold have been defined so that an image which shows a light fog density Is comprised within the first fog density threshold and the second fog density threshold.
  • step S04 may be carried out in which a third plurality of foggy images 103 Is obtained.
  • This third plurality may be obtained using the same method used In step SOI.
  • the fog densities of the images of the third plurality of foggy Image are chosen so as to be contained within the first fog density threshold and the second fog density threshold. Hence, in the third plurality of foggy Images, the Image present light fog.
  • the third plurality of foggy Images 103 and the corresponding predefined semantic segmentation is then used In step S05 in which a semantic segmentation model is trained.
  • This training is performed by Inputting each Image of the third plurality of foggy Images 103 to the model, and comparing the output of the model to the predefined semantic segmentation of the Image so as to adjust the parameters of the model and train the model.
  • the second plurality of foggy images 102 can be inputted to the trained model of step S05 In step S06. For each image, a semantic segmentation 102' is obtained.
  • step S07 a fourth plurality of foggy Images 104 Is obtained.
  • This fourth plurality may be obtained using the same method used in step SOI.
  • the fog densities of the images of the fourth plurality of foggy Image are chosen so as to be contained within a third fog density threshold and a fourth fog density threshold, the third and fourth fog density thresholds being both greater than the first and second fog density thresholds.
  • the third and fourth fog density thresholds may be chosen so as to Illustrate dense fog.
  • semantic segmentations 104' of the fourth plurality of foggy Images have been represented. These semantic segmentations may have been used for the generation of synthetic fog.
  • step S08 the model trained (or pre-trained) in step S05 Is trained (or trained further) using:
  • step S6 the fourth plurality of foggy Images 104 and predefined semantic segmentations 104' of the fourth plurality of foggy Images, - the second plurality of foggy images 102 and the semantic segmentations 102' of the plurality of foggy images obtained In step S06.
  • Step S08 comprises inputting, to the semantic segmentation model of step S05, a mix of Images from the second plurality of foggy images 102 and of Images from the fourth plurality 104 of foggy images so as to obtain a stream of images to be fed to the model, wherein the stream of Images comprises a greater proportion of Images from the fourth plurality of foggy Images than images from the second plurality of foggy images.
  • step h comprises minimizing the following value:
  • l is the number of image in the fourth plurality of foggy images
  • w being a predefined weight (for example equal to 1/3),
  • This system 200 which may be a computer, comprises a processor201 and a non-volatile memory 202.
  • a set of instructions is stored and this set of instructions comprises instructions to perform a method for training a model and more precisely:
  • An Instruction 203D to obtain a third plurality of foggy images each having a fog density of synthetic fog comprised within the first fog density threshold and the second fog density threshold,
  • An instruction 203E to train a semantic segmentation model using the third plurality of foggy images and predefined semantic segmentations of the third plurality of Images,
  • a module C for classifying a second plurality of Images using the classification model of module B according to the fog density In the images, so as to obtain a second plurality of foggy Images each having a fog density comprised within a first fog density threshold and a second fog density threshold
  • a module D for obtaining a third plurality of foggy Images each having a fog density of synthetic fog comprised within the first fog density threshold and the second fog density threshold
  • obtaining foggy Images having synthetic fog may be performed by using the method disclosed In the international patent application filed by the same applicants on the same day as the present application and titled "A method and a system for processing images to obtain foggy Images", which is Incorporated entirely to the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un système et un procédé permettant d'apprendre un modèle destiné à la segmentation sémantique d'images, ledit procédé consistant à : a) obtenir (S01) une première pluralité d'images brumeuses (101); b) apprendre (S02) un modèle de classification permettant d'estimer la densité du brouillard; c) classer (S03) une deuxième pluralité d'images (101) comprenant un léger brouillard; d) obtenir (S04) une troisième pluralité d'images brumeuses (103) comprenant un léger brouillard; e) apprendre (S05) un modèle de segmentation sémantique à l'aide de la troisième pluralité d'images brumeuses; f) appliquer (S06) le modèle de segmentation sémantique à la deuxième pluralité d'images brumeuses (102) afin d'obtenir des segmentations sémantiques (102); g) obtenir (S07) une quatrième pluralité d'images brumeuses (104) comprenant un brouillard dense; et h) effectuer un apprentissage (S08) à l'aide des images brumeuses obtenues précédemment.
PCT/EP2018/070074 2018-07-24 2018-07-24 Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses WO2020020446A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/258,519 US11941815B2 (en) 2018-07-24 2018-07-24 Method and a system training a model to perform semantic segmentation on foggy images
PCT/EP2018/070074 WO2020020446A1 (fr) 2018-07-24 2018-07-24 Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses
CN201880095853.4A CN112513928A (zh) 2018-07-24 2018-07-24 训练模型以对有雾图像执行语义分割的方法和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2018/070074 WO2020020446A1 (fr) 2018-07-24 2018-07-24 Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses

Publications (1)

Publication Number Publication Date
WO2020020446A1 true WO2020020446A1 (fr) 2020-01-30

Family

ID=63013045

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/070074 WO2020020446A1 (fr) 2018-07-24 2018-07-24 Procédé et système d'apprentissage d'un modèle pour effectuer une segmentation sémantique sur des images brumeuses

Country Status (3)

Country Link
US (1) US11941815B2 (fr)
CN (1) CN112513928A (fr)
WO (1) WO2020020446A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11321863B2 (en) * 2019-09-23 2022-05-03 Toyota Research Institute, Inc. Systems and methods for depth estimation using semantic features
US11772656B2 (en) * 2020-07-08 2023-10-03 Ford Global Technologies, Llc Enhanced vehicle operation
CN114691912A (zh) * 2020-12-25 2022-07-01 日本电气株式会社 图像处理的方法、设备和计算机可读存储介质
CN113139922B (zh) * 2021-05-31 2022-08-02 中国科学院长春光学精密机械与物理研究所 图像去雾方法及去雾装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101568971B1 (ko) * 2011-08-03 2015-11-13 인디안 인스티튜트 오브 테크놀로지, 카라그푸르 화상 및 동영상에서 안개를 제거하는 방법 및 시스템
CN103188433B (zh) * 2011-12-30 2016-01-20 株式会社日立制作所 图像除雾装置和图像除雾方法
CN104134194A (zh) * 2014-07-23 2014-11-05 中国科学院深圳先进技术研究院 图像去雾方法和系统
US10019652B2 (en) * 2016-02-23 2018-07-10 Xerox Corporation Generating a virtual world to assess real-world video analysis performance
US10692000B2 (en) * 2017-03-20 2020-06-23 Sap Se Training machine learning models
CN107767353A (zh) * 2017-12-04 2018-03-06 河南工业大学 一种基于清晰度评价的自适应图像去雾方法
US20190377981A1 (en) * 2018-06-11 2019-12-12 Venkata Subbarao Veeravasarapu System and Method for Generating Simulated Scenes from Open Map Data for Machine Learning

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
HE, K.; SUN, J.; TANG, X.: "Single image haze removal using dark channel prior", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 33, no. 12, 2011, pages 2341 - 2353, XP011409121, DOI: doi:10.1109/TPAMI.2010.168
NISHINO, K.; KRATZ, L.; LOMBARDI, S.: "Bayesian defogging", INTERNATIONAL JOURNAL OF COMPUTER VISION, vol. 98, no. 3, 2012, pages 263 - 278, XP035047219, DOI: doi:10.1007/s11263-011-0508-1
PARIS, S.; DURAND, F.: "A fast approximation of the bilateral filter using a signal processing approach", INTERNATIONAL JOURNAL OF COMPUTER VISION, 2009
SAKARIDIS CHRISTOS ET AL: "Semantic Foggy Scene Understanding with Synthetic Data", INTERNATIONAL JOURNAL OF COMPUTER VISION, KLUWER ACADEMIC PUBLISHERS, NORWELL, US, vol. 126, no. 9, 23 March 2018 (2018-03-23), pages 973 - 992, XP036568429, ISSN: 0920-5691, [retrieved on 20180323], DOI: 10.1007/S11263-018-1072-8 *
SAKARIDIS, C.; DAI, D.; VAN GOOL, L., INTERNATIONAL JOURNAL OF COMPUTER VISION, 2018
SAKARIDIS, C.; DAI, D.; VAN GOOL, L.: "Semantic foggy scene understanding with synthetic data", INTERNATIONAL JOURNAL OF COMPUTER VISION, 2018
YOSHUA BENGIO ET AL: "Curriculum learning", PROCEEDINGS OF THE 26TH ANNUAL INTERNATIONAL CONFERENCE ON MACHINE LEARNING, ICML '09, ACM PRESS, NEW YORK, NEW YORK, USA, 14 June 2009 (2009-06-14), pages 41 - 48, XP058344301, ISBN: 978-1-60558-516-1, DOI: 10.1145/1553374.1553380 *
ZHANG YANG ET AL: "Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes", 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), IEEE, 22 October 2017 (2017-10-22), pages 2039 - 2049, XP033283066, DOI: 10.1109/ICCV.2017.223 *

Also Published As

Publication number Publication date
US11941815B2 (en) 2024-03-26
CN112513928A (zh) 2021-03-16
US20210158098A1 (en) 2021-05-27

Similar Documents

Publication Publication Date Title
Negru et al. Exponential contrast restoration in fog conditions for driving assistance
US11941815B2 (en) Method and a system training a model to perform semantic segmentation on foggy images
TWI501194B (zh) 基於費雪線性判別的雙暗通道先驗的影像能見度修復方法及其影像處理裝置
TWI514323B (zh) 基於混合暗通道先驗的影像除霧方法及其系統
TWI524734B (zh) 產生深度圖的方法及裝置
US8340422B2 (en) Generation of depth map for an image
DE102018201054A1 (de) System und Verfahren zur Bilddarstellung durch ein Fahrerassistenzmodul eines Fahrzeugs
DE112018007287T5 (de) Fahrzeugsystem und -verfahren zum erfassen von objekten und einer objektentfernung
CN112862702B (zh) 图像增强方法、装置、设备及存储介质
CN111091091A (zh) 目标对象重识别特征的提取方法、装置、设备及存储介质
WO2020020445A1 (fr) Procédé et système de traitement d'images pour obtenir des images brumeuses
JP2004164624A (ja) 低被写界深度画像セグメンテーションの方法及び機器
CN109558901B (zh) 一种语义分割训练方法及装置、电子设备、存储介质
WO2018168539A1 (fr) Procédé et programme d'apprentissage
KR101582779B1 (ko) 영상의 파장 적응적 안개 제거 장치 및 방법
CA2392280A1 (fr) Technique de compression de carte de profondeur
CN111179189A (zh) 基于生成对抗网络gan的图像处理方法、装置以及电子设备、存储介质
Agrawal et al. A comprehensive review on analysis and implementation of recent image dehazing methods
CN113160062A (zh) 一种红外图像目标检测方法、装置、设备及存储介质
KR101998027B1 (ko) 악천후에서의 도로 객체 검출 기법 학습을 위한 안개 및 비 영상 합성 기법 및 시스템
WO2020239196A1 (fr) Système et procédé d'apprentissage d'un modèle adverse génératif générant des échantillons d'image de différents niveaux de luminosité
CN105608683A (zh) 一种单幅图像去雾方法
CN112906450A (zh) 确定车辆附近的占用地图的条目的车辆、系统和方法
CN116433513A (zh) 一种道路监视视频去雾方法、系统、电子设备和存储介质
Negru et al. Exponential image enhancement in daytime fog conditions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18745595

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18745595

Country of ref document: EP

Kind code of ref document: A1