US11978262B2 - Image classification and associated training for safety-relevant classification tasks - Google Patents
Image classification and associated training for safety-relevant classification tasks Download PDFInfo
- Publication number
- US11978262B2 US11978262B2 US17/357,071 US202117357071A US11978262B2 US 11978262 B2 US11978262 B2 US 11978262B2 US 202117357071 A US202117357071 A US 202117357071A US 11978262 B2 US11978262 B2 US 11978262B2
- Authority
- US
- United States
- Prior art keywords
- image
- classifier
- learning
- information density
- resolved
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000012549 training Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012986 modification Methods 0.000 claims 4
- 230000004048 modification Effects 0.000 claims 4
- 238000001454 recorded image Methods 0.000 claims 1
- 238000012360 testing method Methods 0.000 description 43
- 230000004424 eye movement Effects 0.000 description 8
- 230000006399 behavior Effects 0.000 description 7
- 230000003068 static effect Effects 0.000 description 5
- 230000004044 response Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000001000 micrograph Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 238000004080 punching Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
Definitions
- the present invention relates to the automatic classification of image data with regard to its content, for example for the at least semi-automated driving of vehicles.
- Such neural networks may, for example, be made up of multiple layers connected one after another, in which the dimensionality of the task is significantly reduced by using convolutional cores and by downsampling.
- Such neural networks are also characterized in that the data are solidly processed in parallel.
- Great Britain Patent No. GB 2 454 857 B describes an example of a method in which, using a self-learning neural network, a microscope image is classified according to which objects it contains.
- a method for training a classifier for image data with the aid of learning image data and associated labels.
- image data also includes in particular such films and image sequences.
- the complete film, or the complete image sequence contains an additional piece of information regarding the dynamic change in the image content. This dynamic quality is missing from the individual images. The fact that this dynamic quality is important is consistent with the fact that people prefer to perceive moving or flashing image content over static image content.
- Each label includes an allocation to one or more classes of a predefined classification.
- the classes may represent objects which are visible in the learning image data. In image data of traffic situations, these objects may in particular be pedestrians, other vehicles, road boundaries, traffic signs and other traffic-relevant objects.
- space-resolved relevance maps are provided for some or ideally for all data sets of learning image data. These relevance maps indicate how relevant which spatial areas of the particular learning image data are for the assessment of the situation shown in these learning image data.
- a data set of learning image data may be a static image or a frame from a film or from an image sequence.
- a data set of learning image data may also be a film or an image sequence.
- a dynamic change in a particular spatial area in the course of a film or an image sequence may already provide a reason for this spatial area to be classified as particularly relevant.
- learning samples are ascertained from data sets of learning image data and associated relevance maps.
- spatial areas that have a local relevance below a predefined threshold value may, for example, be blanked out, blurred or otherwise garbled in the learning samples.
- the learning samples are fed to the classifier. Thereafter, the classifier maps the learning samples to allocations to one or more classes. Parameters which characterize the behavior of the classifier are then optimized with the aim that these allocations are consistent with the labels of the learning image data from which the learning samples have originated.
- the classifier contains an artificial neural network, ANN, or is an ANN
- the trainable parameters may include, for example, weights with which inputs of neurons or other processing units are settled in a weighted manner for activation of the particular neuron, or of the particular processing unit.
- the space-resolved relevance map may come from an arbitrary source. By way of example, it may be supplied together with the labels as further additional information to the learning image data. However, the space-resolved relevance map may also be retrieved, for example, from another trained ANN. This means that this further ANN may be specifically trained on the question of which spatial areas of the image data require particular attention in which situation, regardless of what exactly is contained in these spatial areas of the image data.
- the effect is particularly pronounced in the case of classifiers which process image data from traffic situations.
- classifiers which process image data from traffic situations.
- usually large solid angles of the surroundings of a vehicle are detected.
- usually only the information from a small part of this solid angle is actually relevant for managing the driving task. If this were not the case, a person who may only look in one direction at a time would have no chance of mastering the driving task.
- the tendency that a recognition by the classifier “zeroes in” on image areas away from the actual traffic event is suppressed.
- the training is also more effective and faster since the classifier no longer has to learn to distinguish what is important from what is unimportant.
- the classifier may be a division of labor between the ANN, which identifies relevant image areas, and the classifier, which subsequently examines these image areas for the objects they contain. Overall, this is easier to train than a monolithic classifier which performs both tasks.
- the present invention also relates to a method for classifying image data using a trained classifier.
- image data may also include entire image sequences or films with dynamic additional information.
- a space-resolved relevance map which indicates how relevant which spatial areas of the image data are for the assessment of the situation shown in these image data. From the image data and the space-resolved relevance map, a sample is ascertained in which, the higher the local relevance according to the relevance map, the locally more pronounced the information from the image data. This sample is fed to the classifier and is mapped by the classifier to an allocation to one or more classes of a predefined classification.
- areas which have a local relevance below a predefined threshold value may in this case, for example, be blanked out, blurred or otherwise garbled.
- the space-resolved relevance map for at least one data set of image data may be retrieved from a trained ANN.
- the present invention also relates to a method for measuring a space-resolved relevance map for a concrete data set of image data (or learning image data).
- This space-resolved relevance map may be used, for example, for training a classifier, for forming a sample to be processed by the classifier, or for training an ANN which ascertains space-resolved relevance maps for image data.
- the image data are presented to at least one test subject.
- the test subject is given the task of perceiving the content of the image data which from its point of view is relevant and, once it has done so, making an input. While the image data are being presented, it is observed to which spatial areas of the image data the test subject turns its attention.
- test subject may optionally be checked whether the test subject not only believes it has identified the content of the image data, but also has actually identified it correctly. For example, not only may an input be requested to the effect that the test subject has perceived the relevant content of the image data, but it may also be asked what exactly the test subject in its opinion has identified. The answer to this question may be compared with a pre-known label as to which objects are actually contained in the image data.
- the presentation may be ended (for instance the image or film may be blanked out), and multiple object names may be presented, from which the test subject must select those that are correct.
- the test subject's input indicating that the relevant content has been perceived may then be rejected, for example, if, when asked, the correct object or at least the correct object class (for instance “dog” or “animal” as the class above “husky”) is not named.
- the local relevance of those areas to which the test subject has turned its attention in the relevance map is increased. This may in particular take place, for example, also in interaction with many test subjects. From the consideration of the image data by each test subject, the information as to which spatial areas of the image data are on average perceived as relevant may be aggregated for example by a voting mechanism.
- the spatial areas of the image data to which the test subject turns its attention does not necessarily depend solely on the image data themselves, but may also be influenced by a task assigned to this test subject.
- the driver may be busy with the driving task while the passenger is looking for a parking space, a mailbox or a certain business. Therefore, if, for example, a relevance map for the at least semi-automated driving of the vehicle is being measured, a driver rather than a passenger should be used as the test subject.
- the head posture, the eye position and/or the eye movements of the test subject are recorded.
- the areas to which the test subject turns its attention may then be evaluated based on the head posture, the eye position and/or the eye movements. This is an indicator that may hardly be consciously influenced by the test subject.
- a driver of a vehicle controls his/her selection of what he/she considers important from the traffic situation, usually via head posture (for instance a shoulder check), eye position and/or eye movements. Other movements are restricted as a result of being strapped into the driver's seat.
- different sub-areas of the image data may successively become visible to the test subject. Those sub-areas of the image data which are visible at the time of the input made by the test subject may then be deemed to be those areas to which the test subject turns its attention. This does not require any special hardware in order to identify exactly where the test subject is looking.
- one and the same data set of image data may for example be presented to multiple test subjects.
- different sequences of sub-areas of the image data which successively become visible may then be presented to these test subjects. This may include changing the order of the sub-areas which successively become visible, and/or presenting to some test subjects sub-areas which are not presented to other test subjects.
- a representative conclusion is then drawn as to which sub-areas of the image data are relevant, for example, for the assessment of traffic situations.
- the space-resolved relevance map and an ANN which generates such relevance maps may also be used to check whether a vehicle driver or machine operator is currently turning his/her attention to those things that are presently important in terms of safety.
- the present invention also relates to a method for observing and/or controlling the attention of a vehicle driver or machine operator.
- image data of the situation in which the driven vehicle or the operated machine is located are detected by at least one sensor.
- a space-resolved relevance map is retrieved from a trained artificial neural network, ANN. This space-resolved relevance map indicates how relevant which areas of the image data are for the assessment of the situation shown in these image data.
- a piece of information and/or a warning is output to the vehicle driver or machine operator.
- the vehicle driver or machine operator may be informed at all times about which aspects of his/her present situation are presently particular important from a safety-related point of view. If it should be found, when comparing the actual behavior of the vehicle driver or machine operator, that he/she is turning his/her attention to something other than the presently important aspects, he/she may be informed of this by way of a warning.
- advertising in shops or at the curbside may attract a lot of attention.
- the advertising is often designed in such a way that certain “hooks,” such as a favorable price, are placed in the foreground and may be read even from a passing vehicle.
- the price is then marked with an asterisk indicating conditions, and any attempt to read these conditions written in small print may take a lot of attention away from the traffic situation.
- an overlay of the situation may be presented to the vehicle driver or machine operator with an indication of at least one spatial area of the image data, the local relevance of which exceeds a predefined threshold value according to the relevance map.
- the area of the situation that is presently particularly relevant may be highlighted for example in a head-up display on a windshield or in data glasses worn by the vehicle driver or machine operator, through the insertion of a border or a similar indication.
- the head posture, the eye position and/or the eye movements of the vehicle driver or machine operator are recorded.
- the head posture, the eye position and/or the eye movements are used to evaluate which part of the situation the vehicle driver or machine operator is predominantly observing.
- this part of the situation is consistent with at least one spatial area of the image data, the local relevance of which exceeds a predefined threshold according to the relevance map.
- a visual, acoustic and/or haptic warning device perceptible to the vehicle driver or machine operator is activated.
- the methods may in particular be entirely or partially computer implemented. Therefore, the present invention also relates to a computer program containing machine-readable instructions which, when executed on one or multiple computer(s), upgrade the computer(s) to the device described above and/or prompt the computer(s) to carry out one of the methods described above.
- control units for vehicles and embedded systems for technical devices which are also capable of executing machine-readable instructions are also to be regarded as computers.
- a download product is a digital product which is transferrable via a data network, i.e. downloaded by a user of the data network, which may be offered for immediate download in an online shop, for example.
- a computer may be equipped with the computer program, with the machine-readable data medium or with the download product.
- FIG. 1 shows an exemplary embodiment of method 100 for training a classifier 1 , in accordance with the present invention.
- FIG. 2 shows an exemplary embodiment of method 200 for classifying image data 2 , in accordance with the present invention.
- FIG. 3 shows an example of the generation of a sample 23 for classification based on image data 2 , in accordance with the present invention.
- FIG. 4 shows an exemplary embodiment of method 300 for measuring a relevance map 12 , 22 , in accordance with the present invention.
- FIG. 5 shows an exemplary embodiment of method 400 for observing and/or controlling the attention of a vehicle driver or machine operator 40 , in accordance with the present invention.
- FIG. 1 is a schematic flowchart of an exemplary embodiment of method 100 for training a classifier 1 for image data 2 .
- step 110 for learning image data 11 , in each case space-resolved relevance maps 12 are provided, those relevance maps 12 according to block 111 may be retrieved, for example, from an appropriately trained ANN.
- step 120 from learning image data 11 and associated relevance maps 12 , learning samples 13 are ascertained, in which, the higher the local relevance according to relevance map 12 , the locally more pronounced the information from learning image data 11 .
- spatial areas whose local relevance is below a predefined threshold value may be blanked out, blurred or otherwise garbled.
- step 130 learning samples 13 are fed to classifier 1 and are mapped to allocations to one or multiple classes 3 a through 3 c .
- step 140 parameters 15 which characterize the behavior of classifier 1 are optimized with the aim that classes 3 a through 3 c delivered by classifier 1 are consistent with labels 14 of learning image data 11 from which learning samples 13 originated. This optimization may be continued until an arbitrary abort criterion is met.
- the fully trained state of parameters 15 is denoted by reference numeral 15 *.
- FIG. 2 is a schematic flowchart of an exemplary embodiment of method 200 for classifying image data 2 .
- image data 2 may be recorded optionally by at least one sensor carried by a vehicle.
- a space-resolved reference map 22 is provided, which in particular according to block 211 , for example, may be retrieved from an ANN.
- a sample 23 is ascertained, in which, the higher the local relevance according to relevance map 22 , the locally more pronounced the information from image data 2 .
- sample 23 is fed to classifier 1 , and according to block 231 in particular, for example, only the significantly data-reduced sample 23 compared to image data 2 may be transferred by the sensor via a bus system of the vehicle, but not image data 2 themselves.
- sample 23 is mapped by classifier 1 to the sought allocation to classes 3 a through 3 c of the predefined classification.
- FIG. 3 shows an example of how image data 2 may be converted into a sample 23 .
- Image data 2 which here are in the form of a static image, show a traffic situation including a road 25 , an oncoming vehicle 26 and a traffic sign 27 .
- a billboard 28 is visible on the left-hand curbside.
- Relevance map 22 assesses road 25 , vehicle 26 and the right-hand curbside, where traffic signs such as sign 27 are located, as relevant. This area is therefore unchanged in sample 23 for the classification, while the details of billboard 28 are blanked out.
- FIG. 4 is a schematic flowchart of an exemplary embodiment of method 300 for measuring a relevance map 12 , 22 for image data 2 , 11 .
- image data 2 , 11 are presented to at least one test subject 4 . Meanwhile, it is observed to which spatial areas 2 a , 11 a of image data 2 , 11 test subject 4 turns its attention.
- step 320 In response to input 41 made by test subject 4 indicating that it has perceived the relevant content of image data 2 , 11 , spatial areas 2 a , 11 a to which test subject 4 has previously turned its attention are detected in step 320 . In step 330 , the local relevance of these areas 2 a , 11 a in relevance map 12 , 22 is increased.
- Box 310 shows, by way of example, two possible ways in which the turning of attention to areas 2 a , 11 a may be established. These possibilities may be used individually or also in combination.
- head posture 42 a , eye position 42 b and/or eye movements 42 c of test subject 4 may be recorded.
- areas 2 a , 11 a of image data 2 , 11 to which test subject 4 turns its attention may then be evaluated based on head posture 42 a , eye position 42 b and/or eye movements 42 c.
- different sub-areas of image data 2 , 11 may be successively made visible to test subject 4 until test subject 4 recognizes image data 2 , 11 and makes input 41 .
- one and the same data set of image data 2 , 11 may be presented to multiple test subjects 4 .
- Different sequences of sub-areas of image data 2 , 11 which successively become visible may be presented to these test subjects 4 . (Block 313 b ).
- those sub-areas of image data 2 , 11 which are visible at the time of input 41 made by test subject 4 may be deemed to be those areas 2 a , 11 a to which test subject 4 turns its attention.
- step 330 the local relevance of ascertained areas 2 a , 11 a in relevance map 12 , 22 is increased, it being possible in particular to use, for example, a voting mechanism across many test subjects 4 .
- FIG. 5 is a schematic flowchart of an exemplary embodiment of method 400 for observing and/or controlling the attention of a vehicle driver or machine operator 40 .
- image data 2 of the situation, in which the driven vehicle or the operated machine is located are detected by at least one sensor.
- a space-resolved relevance map 22 is retrieved from a trained artificial neural network, ANN.
- a piece of information and/or a warning 6 is output to the vehicle driver or machine operator 40 .
- Box 430 shows two possible ways in which the piece of information and/or warning 6 may be generated. These possibilities may be used individually or also in combination.
- an overlay of the situation may be presented to the vehicle driver or machine operator 40 with an indication of at least one image area, the local relevance of which exceeds a predefined threshold value according to the relevance map.
- head posture 42 a , eye position 42 b and/or eye movements 42 c of vehicle driver or machine operator 40 may be recorded.
- these may be used to evaluate which part 7 of the situation vehicle driver or machine operator 40 is predominantly observing.
- it may be checked to what extent this part 7 of the situation is consistent with at least one spatial area of the image data, the local relevance of which exceeds a predefined threshold value according to relevance map 22 . If part 7 of the situation is not consistent with the aforementioned spatial area of the image data (logical value 0), a visual, acoustic and/or haptic warning device perceptible to vehicle driver or machine operator 40 may be activated.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Human Computer Interaction (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Claims (8)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102020208008.9 | 2020-06-29 | ||
DE102020208008.9A DE102020208008A1 (en) | 2020-06-29 | 2020-06-29 | Image classification and related training for security-related classification tasks |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210406587A1 US20210406587A1 (en) | 2021-12-30 |
US11978262B2 true US11978262B2 (en) | 2024-05-07 |
Family
ID=78827053
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/357,071 Active 2042-05-24 US11978262B2 (en) | 2020-06-29 | 2021-06-24 | Image classification and associated training for safety-relevant classification tasks |
Country Status (3)
Country | Link |
---|---|
US (1) | US11978262B2 (en) |
CN (1) | CN113935362A (en) |
DE (1) | DE102020208008A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102020208008A1 (en) * | 2020-06-29 | 2021-12-30 | Robert Bosch Gesellschaft mit beschränkter Haftung | Image classification and related training for security-related classification tasks |
US11800065B2 (en) | 2021-08-19 | 2023-10-24 | Geotab Inc. | Mobile image surveillance systems and methods |
CN116824198A (en) * | 2022-03-21 | 2023-09-29 | 华为云计算技术有限公司 | Bias evaluation method, bias evaluation device, bias evaluation medium, bias evaluation program product and electronic equipment |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2454857B (en) | 2006-09-18 | 2010-06-09 | Bosch Gmbh Robert | Method for processing a microscope intensity image |
US20170103269A1 (en) * | 2015-10-07 | 2017-04-13 | Honda Motor Co., Ltd. | System and method for providing laser camera fusion for identifying and tracking a traffic participant |
US20180018553A1 (en) * | 2015-03-20 | 2018-01-18 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Relevance score assignment for artificial neural networks |
US20180240221A1 (en) * | 2017-02-17 | 2018-08-23 | Cogisen S.R.L. | Method for image processing and video compression |
US10699192B1 (en) * | 2019-01-31 | 2020-06-30 | StradVision, Inc. | Method for optimizing hyperparameters of auto-labeling device which auto-labels training images for use in deep learning network to analyze images with high precision, and optimizing device using the same |
US20210125104A1 (en) * | 2019-10-25 | 2021-04-29 | Onfido Ltd | Machine learning inference system |
US20210406587A1 (en) * | 2020-06-29 | 2021-12-30 | Robert Bosch Gmbh | Image classification and associated training for safety-relevant classification tasks |
-
2020
- 2020-06-29 DE DE102020208008.9A patent/DE102020208008A1/en active Pending
-
2021
- 2021-06-24 US US17/357,071 patent/US11978262B2/en active Active
- 2021-06-28 CN CN202110716868.1A patent/CN113935362A/en active Pending
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2454857B (en) | 2006-09-18 | 2010-06-09 | Bosch Gmbh Robert | Method for processing a microscope intensity image |
US20180018553A1 (en) * | 2015-03-20 | 2018-01-18 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Relevance score assignment for artificial neural networks |
US20170103269A1 (en) * | 2015-10-07 | 2017-04-13 | Honda Motor Co., Ltd. | System and method for providing laser camera fusion for identifying and tracking a traffic participant |
US20180240221A1 (en) * | 2017-02-17 | 2018-08-23 | Cogisen S.R.L. | Method for image processing and video compression |
US10699192B1 (en) * | 2019-01-31 | 2020-06-30 | StradVision, Inc. | Method for optimizing hyperparameters of auto-labeling device which auto-labels training images for use in deep learning network to analyze images with high precision, and optimizing device using the same |
US20210125104A1 (en) * | 2019-10-25 | 2021-04-29 | Onfido Ltd | Machine learning inference system |
US20210406587A1 (en) * | 2020-06-29 | 2021-12-30 | Robert Bosch Gmbh | Image classification and associated training for safety-relevant classification tasks |
Also Published As
Publication number | Publication date |
---|---|
US20210406587A1 (en) | 2021-12-30 |
CN113935362A (en) | 2022-01-14 |
DE102020208008A1 (en) | 2021-12-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11978262B2 (en) | Image classification and associated training for safety-relevant classification tasks | |
US20190162549A1 (en) | Cognitive state vehicle navigation based on image processing | |
DE112018007287T5 (en) | VEHICLE SYSTEM AND METHOD FOR DETECTING OBJECTS AND OBJECT DISTANCE | |
DE112014007249B4 (en) | Image processing device, vehicle display system, display device, image processing method and image processing program | |
WO2019048011A1 (en) | Gesture control for communication with an autonomous vehicle on the basis of a simple 2d camera | |
DE102017100198A1 (en) | FIXING GENERATION FOR MACHINE LEARNING | |
DE102021114620A1 (en) | PRESENTATION OF DYNAMIC THREAT INFORMATION BASED ON THREAT AND TRAJECTORY PREDICTION | |
DE102014220302A1 (en) | Real-time multi-class driver action detection using a Random Forest | |
WO2014173863A1 (en) | Method and apparatus for detecting non-motorised road users | |
DE102008043743A1 (en) | Sensor signals e.g. video sensor signals, evaluating method for detecting e.g. traffic sign in surrounding of vehicle, involves evaluating information based on evaluation specification, and outputting information based on evaluation result | |
DE102007001099A1 (en) | Driver assistance system for traffic sign recognition | |
WO2013152929A1 (en) | Learning method for automated recognition of traffic signs, method for determining an updated parameter set for the classification of a traffic sign and traffic sign recognition system | |
WO2018215242A2 (en) | Method for determining a driving instruction | |
DE102017215283A1 (en) | Method for anonymizing an image for a camera system of a motor vehicle, image processing device, camera system and motor vehicle | |
DE102018128634A1 (en) | Method for providing visual information about at least part of an environment, computer program product, mobile communication device and communication system | |
DE102021130548A1 (en) | Presentation of objects in an image based on anomalies | |
DE102016120066A1 (en) | A computer implemented method for controlling an object recognition system | |
US11823305B2 (en) | Method and device for masking objects contained in an image | |
EP4287147A1 (en) | Training method, use, software program and system for the detection of unknown objects | |
DE102020205825A1 (en) | System for deception detection, prevention and protection of ADAS functions | |
DE102019108722A1 (en) | Video processing for machine learning | |
DE102020205831A1 (en) | System for deception detection, prevention and protection of ADAS functions | |
EP3553695A1 (en) | Method and system for displaying image data from at least one night vision camera of a vehicle | |
DE102021207258B3 (en) | Method for automatically controlling at least one vehicle function of a vehicle and notification system for a vehicle | |
US20230267653A1 (en) | Generating realistic images from specified semantic maps |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: ROBERT BOSCH GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MAYER, UDO;REEL/FRAME:058694/0725 Effective date: 20210916 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
ZAAB | Notice of allowance mailed |
Free format text: ORIGINAL CODE: MN/=. |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |