US20200193225A1 - System and method for detecting objects in a digital image, and system and method for rescoring object detections - Google Patents

System and method for detecting objects in a digital image, and system and method for rescoring object detections Download PDF

Info

Publication number
US20200193225A1
US20200193225A1 US16/608,399 US201716608399A US2020193225A1 US 20200193225 A1 US20200193225 A1 US 20200193225A1 US 201716608399 A US201716608399 A US 201716608399A US 2020193225 A1 US2020193225 A1 US 2020193225A1
Authority
US
United States
Prior art keywords
candidate
detection
detections
windows
latent representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/608,399
Other languages
English (en)
Inventor
Daniel Olmeda Reino
Bernt Schiele
Jan Hendrik Hosang
Rodrigo Benenson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toyota Motor Europe NV SA
Max Planck Gesellschaft zur Foerderung der Wissenschaften eV
Original Assignee
Toyota Motor Europe NV SA
Max Planck Gesellschaft zur Foerderung der Wissenschaften eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toyota Motor Europe NV SA, Max Planck Gesellschaft zur Foerderung der Wissenschaften eV filed Critical Toyota Motor Europe NV SA
Publication of US20200193225A1 publication Critical patent/US20200193225A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G06K9/6256—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06K9/3241—
    • G06K9/623—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the present disclosure is related to a system and a method for detecting objects in a digital image, and a system and a method for rescoring object detections.
  • NMS non-maximum suppression
  • NMS is one step in the pipeline that, for the most part, has evaded the end-to-end learning paradigm. All of the above detectors train the classifier in a procedure that ignores the fact that the NMS problem exists and then runs a conventional NMS as a disconnected postprocessing.
  • a system for detecting objects in a digital image comprises a neural network being configured to generate candidate windows indicating object locations, and to generate for each candidate window a score representing the confidence of detection.
  • Generating the scores comprises:
  • Said neighboring candidate windows are referring desirably to the neighbors of the candidate window, whose latent representation is updated.
  • the rescoring system is desirably configured to recognize multi detections by processing each object detection (i.e. generating the latent representation of each candidate window) together with its neighboring detections (i.e. by updating each candidate window's representation, thereby considering the latent representations of the neighboring candidate windows of the currently updated candidate window).
  • each object detection i.e. generating the latent representation of each candidate window
  • its neighboring detections i.e. by updating each candidate window's representation, thereby considering the latent representations of the neighboring candidate windows of the currently updated candidate window.
  • a latent representation may be a multi-dimensional feature vector.
  • latent representation may be determined based on the candidate window, and in particular image content in the candidate window.
  • An example would be to take the image content in the window and process it by at least one (or e.g. three) layer of the neural network, in order to obtain the latent representation/feature vector.
  • the at least one layer consists of at least one of a convolution, a pooling, and/or a FC (fully connected) layer.
  • the latent representations may be updated such that the resulting scores of the candidate windows are changed.
  • this change is such that one candidate window of a detected object increases its score, while all other candidate windows on the same object decrease their score.
  • the amount of increase and/or decrease may be e.g. a learned amount that is deduced from the latent representations.
  • Updating the latent representation of a candidate window may be performed by considering pairs of neighboring candidate windows.
  • a pair of neighboring candidate windows may comprise said candidate window and one of its neighboring candidate windows.
  • the neighboring candidate windows may comprise all windows overlapping to a predetermined extent with the candidate window, of which the latent representation is updated.
  • the neural network may comprise a repeating structure for updating the latent representation of each candidate window based on the latent representation of neighboring candidate windows.
  • Updating the latent representation of a candidate window may comprise forming with each of its neighboring candidate windows a pair of detections.
  • the pair of detections may include the latent representations of said candidate window and said neighboring window.
  • the pair of detections may be a pair feature vector.
  • Said pair feature vector may have equal or different dimensions as the feature vector of a latent representation. Generally, after every FC layer the dimensions may be freely selectable.
  • Updating the latent representation of a candidate window may further comprise determining detection pair features based on the two candidate windows, for example, the geometry of the candidate windows.
  • the detection pair features of the pair of candidate windows may be concatenated to the pair feature vector.
  • the pair feature vector, to which the detection pair features are concatenated, may be mapped independently through fully connected layers.
  • the number of pair feature vectors corresponding to the variable number of neighboring candidate windows may be reduced to a fixed-size representation by pooling.
  • the pair feature vectors may be reduced with an elementwise operation to one n-dimensional pooling feature vector.
  • Said pooling feature vector may have equal or different dimensions as the feature vector of a latent representation and/or as the pair feature vector.
  • the multiple (k) pair feature vectors of n dimensions are reduced to one n-dimensional pooling feature vector.
  • the dimensionalities of the latent representations of candidate windows may be reduced before being combined into the pair feature vector.
  • the dimensionalities may be increased to match the size of the latent representations of candidate windows.
  • a candidate window may comprise a rectangular frame and/or a pixelwise mask of a detected object.
  • the neural network may be trained by using at least one digital training image as a ground truth having a plurality of objects and respective object annotations indicating the actual locations of the objects.
  • the training may comprise:
  • a matching (association) between unique couples of object annotations and candidate windows can be determined, so that none of the object annotations or the candidate window is matched (associated) twice.
  • This matching may be obtained e.g. by:
  • the neural network may be trained by using successfully matched detections as positive training examples, and unmatched detections as negative training examples.
  • the invention further relates to a system for rescoring object detections in a digital image.
  • An object detection comprises a candidate window indicating the object location and a score representing the confidence of detection.
  • the system comprises a neural network being configured to:
  • the present disclosure proposes a “pure NMS network” which is able to do the task of non-maximum suppression without image content or access to decisions of another algorithm.
  • Said system for rescoring object detections may be a part (or a sub-system) of the system for detecting objects in a digital image, as described above.
  • the system for detecting objects may comprise the system for rescoring object detections.
  • the system for detecting objects in a digital image may comprise a neural network being configured to generate candidate windows indicating object locations, and for each candidate window generate a score representing the confidence of detection, wherein said neural system may further comprise said system for rescoring object detections, as described above.
  • the system for detecting objects may comprise a first neural network being configured to generate candidate windows indicating object locations, and for each candidate window generate a score representing the confidence of detection.
  • the system for detecting objects may comprise a second neural system constituting the system for rescoring object detections.
  • this change is such that one candidate window of a detected object increases its score, while all other candidate windows on the same object decrease their score.
  • the amount of increase and/or decrease may be e.g. a learned amount that is deduced from the latent representations.
  • Updating the latent representation of a candidate window may be performed by considering pairs of neighboring candidate windows.
  • a pair of neighboring candidate windows may comprise said candidate window and one of its neighboring candidate windows.
  • the neighboring candidate windows may comprise all windows overlapping to a predetermined extent with the candidate window, of which the latent representation is updated.
  • the neural network may comprise a repeating structure for updating the latent representation of each candidate window based on the latent representation of neighboring candidate windows.
  • Updating the latent representation of a candidate window may comprise forming with each of its neighboring candidate windows a pair of detections.
  • the pair of detections may include the latent representations of said candidate window and said neighboring window.
  • the pair of detections may be a pair feature vector.
  • Updating the latent representation of a candidate window may further comprise determining detection pair features based on the two candidate windows, for example, the geometry of the candidate windows.
  • the detection pair features of the pair of candidate windows may be concatenated to the pair feature vector.
  • the pair feature vector, to which the detection pair features are concatenated, may be mapped independently through fully connected layers.
  • the number of pair feature vectors corresponding to the variable number of neighboring candidate windows may be reduced to a fixed-size representation by pooling.
  • the pair feature vectors may be reduced with an elementwise operation to one n-dimensional pooling feature vector.
  • the dimensionalities of the latent representations of candidate windows may be reduced before being combined into the pair feature vector.
  • the dimensionalities may be increased to match the size of the latent representations of candidate windows.
  • a candidate window may comprise a rectangular frame and/or a pixelwise mask of a detected object.
  • the neural network may be trained by using at least one digital training image as a ground truth having a plurality of objects and respective object annotations indicating the actual locations of the objects.
  • the training may comprise:
  • the neural network may be trained by using successfully matched detections as positive training examples, and unmatched detections as negative training examples.
  • the present disclosure further relates to a method of detecting objects in a digital image.
  • a neural network performs the steps of:
  • the step of generating the scores comprises:
  • the method may comprise further method steps which correspond to the functions of the system for detecting objects in a digital image, as described above.
  • the further desirable method steps are described in the following.
  • the present disclosure further relates to a method of rescoring object detections in a digital image.
  • An object detection comprises a candidate window indicating the object location and a score representing the confidence of detection.
  • a neural network performs the steps of:
  • the method may comprise further method steps which correspond to the functions of the system for rescoring object detections in a digital image, as described above.
  • the further desirable method steps are described in the following.
  • the present disclosure further relates to a computer program comprising instructions for executing the steps of the method of detecting objects in a digital image, when the program is executed by a computer.
  • the present disclosure further relates to a computer program comprising instructions for executing the steps of the method of rescoring object detections in a digital, when the program is executed by a computer.
  • FIG. 1 shows a block diagram of a system with a detector and a rescoring system according to embodiments of the present disclosure
  • FIG. 2 shows a schematic diagram of how detection features are combined into a pairwise context according to embodiments of the present disclosure
  • FIG. 3 shows a schematic flow chart illustrating the operation of the rescoring system, in particular a neural network for the rescoring procedure according to embodiments of the present disclosure
  • FIG. 4 a shows a schematic flow chart illustrating a training architecture of the neural network according to embodiments of the present disclosure.
  • FIG. 4 b shows a schematic flow chart illustrating a test architecture of the neural network according to embodiments of the present disclosure.
  • FIG. 1 shows a block diagram of a system 10 with an object detector 1 and a rescoring system 2 (i.e. a system for rescoring object detections) according to embodiments of the present disclosure.
  • the system may have various further functions, e.g. may be a robotic system or a camera system. It may further be integrated in a vehicle.
  • the system 10 may comprise an electronic circuit, a processor (shared, dedicated, or group), a combinational logic circuit, a memory that executes one or more software programs, and/or other suitable components that provide the described functionality.
  • system 10 may be a computer device.
  • the system may be connected to a memory, which may store data, e.g. a computer program which when executed, carries out the method according to the present disclosure.
  • the system or the memory may store software which comprises the neural network according to the present disclosure.
  • the system 10 has an input for receiving a digital image or a stream of digital images.
  • the system 10 in particular the detector 1 , may be connected to an optical sensor 3 , in particular a digital camera.
  • the digital camera 3 is configured such that it can record a scene, and in particular output digital data to the system 10 , in particular the detector 1 .
  • the detector 1 may be implemented as software running on the system 10 or as a hardware element of the system 10 .
  • the detector 1 carries out a computer vision algorithm for detecting the presence and location of objects in a sensed scene. For example, vehicles, persons, and other objects may be detected.
  • the detector outputs candidate windows indicating object locations and generates for each candidate window a score representing the confidence of detection.
  • the rescoring system 2 may be implemented as a software running on the system 10 or as a hardware element of the system 10 .
  • the system may comprise a neural network which includes both the detector and the rescoring system.
  • the rescoring system may be realized as an independent neural network (in particular beside a neural network comprising the detector).
  • the rescoring system 2 receives as an input from the detector the detection results. In particular it receives information regarding one or more object detections. Each object detection comprises a candidate window indicating the object location and a score representing the confidence of detection. The rescoring system rescores these objects detections such that double detections are suppressed. In detail, the rescoring system generates a latent representation for each candidate window. The latent representation of each candidate window is subsequently updated based on the latent representation of neighboring candidate windows. The (thus re-evaluated) score for each candidate window is then generated based on its updated latent representation.
  • the rescoring system is configured to recognize double detections by processing each object detection together with its neighboring detections. Those object detections which are recognized to be multi detections of the same object are suppressed such that only one object detection remains.
  • Present-day detectors do not return all detections that have been scored, but instead use NMS as a post-processing step to remove redundant detections.
  • NMS a post-processing step to remove redundant detections.
  • the present disclosure relates to detectors without any post-processing. To understand why NMS is necessary, it is useful to look at the task of detection and how it is evaluated.
  • the task of object detection is to map an image to a set of boxes (i.e. candidate windows): one box per object of interest in the image, each box tightly enclosing an object. This means detectors ought to return exactly one detection per object. Since uncertainty is an inherent part of the detection process, evaluations allow detections to be associated to a confidence. Confident erroneous detections are penalized more than less confident ones. In particular mistakes that are less confident than the least confident correct detection are not penalized at all.
  • the detection problem can be interpreted as a classification problem that estimates probabilities of object classes being present for every possible detection in an image.
  • This viewpoint gives rise to “hypothesize and score” detectors that build a search space of detections (e.g. sliding window, proposals) and estimate class probabilities independently for each detection.
  • detections e.g. sliding window, proposals
  • class probabilities independently for each detection.
  • two strongly overlapping windows covering the same object will both result in high score since they look at almost identical image content.
  • each object instead of one detection per object, each object triggers several detections of varying confidence, depending on how well the detection windows cover the object.
  • the present disclosure is based on these requirements to an NMS algorithm but proposes a solution where the NMS is a “pure NMS network”, in particular it can be incorporated into a detector.
  • Typical inference of detectors consist of a classifier that discriminates between image content that contains an object and image content that does not.
  • the positive and negative training examples for this detector are usually defined by some measure of overlap between objects and bounding boxes. Since similar boxes will produce similar confidences anyway, small perturbation of object locations can be considered positive examples, too.
  • This technique augments the training data and leads to more robust detectors. Using this type of classifier training does not reward one high scoring detection per object, and instead deliberately encourages multiple high scoring detections per object.
  • the neural network design accommodates both ingredients.
  • the neural network design avoids hard decisions and does not discard detections to produce a smaller set of detections.
  • NMS is reformulated as a rescoring task that seeks to decrease the score of detections that cover objects that already have been detected. After rescoring, simple thresholding may be done to reduce the set of detections. For evaluation the full set of rescored detections may be passed to the evaluation script without any post processing.
  • a detector is supposed to output exactly one high scoring detection per object. The loss for such a detector must inhibit multiple detections of the same object, irrespective of how close these detections are.
  • the detector may be judged by the evaluation criterion of a benchmark, which in turn defines a matching strategy to decide which detections are correct or wrong. This is the matching that should be used at training time.
  • benchmarks sort detections in descending order by their confidence and match detections in this order to objects, preferring most overlapping objects. Since already matched objects cannot be matched again surplus detections are counted as false positives that decrease the precision of the detector.
  • This matching strategy may be used for training.
  • the result of the matching may be used as labels for the classifier: successfully matched detections are positive training examples, while unmatched detections are negative training examples for a standard binary loss.
  • all detections that are used for training of a classifier have a label associated as they are fed into the network.
  • the network has access to detections and object annotations and the matching layer generates labels, that depend on the predictions of the network. This class assignment directly encourages the rescoring behaviour that is desired to be achieved.
  • the weighting wyi is used to counteract the extreme class imbalance of the detection task.
  • the weights may be chosen so the expected class conditional weight of an example equals a parameter
  • detections are associated to both a confidence and a class. Since only detections are rescored, detections are allowed to be “switched off” but not to change their class. As a result, only detections are matched to objects of the same class, but the classification problem remains binary and the above loss still applies.
  • a one-hot encoding may be used: a zero vector that only contains the score at the location in the vector that corresponds to the class. Since mAP computation does not weight classes by their size, the instance weights may be assigned in a way that their expected class conditional weight is uniformly distributed.
  • a neural network is designed with a repeating structure, which is called “blocks” (shown in FIG. 3 ).
  • One block gives each detection access to the representation of its neighbours and subsequently updates its own representation.
  • Stacking multiple blocks means the network alternates between allowing every detection “talk” to its neighbours and updating its own representation. In other words, detections talk to their neighbours to update their representation.
  • the first is a layer, that builds representations for pairs of detections, as shown in FIG. 2 . This leads to the key problem: an irregular number of neighbours for each detection. Since it is desired to avoid a discretisation scheme, this issue is desirably solved with pooling across detections (the second key).
  • the first block takes an all-zero vector as input.
  • the detections' information is fed into the network in the “pairwise computations” section of FIG. 3 as described below. This zero input could potentially be replaced with image features.
  • the first is a layer that builds representations for pairs of detections, as shown in FIG. 2 .
  • FIG. 2 shows a schematic diagram of how detection features are combined into a pairwise context according to embodiments of the present disclosure.
  • Each solid block is the feature vector of the detection of corresponding pattern (e.g. different dashed lines).
  • the hatched blocks are the “detection pair features” that are defined by the two detections corresponding to the two patterns.
  • Each mini-batch consists of all n detections on an image, each represented by a c dimensional feature vector, so the data has size n ⁇ c and accessing to another detection's representations means operating within the batch elements.
  • a detection context layer is used, that, for every detection di, generates all pairs of detections (di; dj) for which dj sufficiently overlaps with di (IoU>0.2).
  • the features are arranged of all pairs of detections along the batch dimension: if detection di has ki neighbouring detection that yields a batch of size K ⁇ l, where
  • the architecture of the present disclosure uses global max-pooling over all detection pairs that belong to the same detection (K ⁇ l ⁇ n ⁇ l), after which normal fully connected layers can be used to update the detection representation (see FIG. 3 ).
  • each detection provides a scores vector instead of a scalar thus increasing the number of pair features. All these raw features are fed into 3 fully connected layers, to learn the g detection pair features that are used in each block.
  • FIG. 3 shows a schematic flow chart illustrating the operation of the rescoring system, in particular a neural network for the updating procedure according to embodiments of the present disclosure.
  • One block of the neural network of the present disclosure is shown here for one detection.
  • the representation of each detection is reduced and then combined into neighbouring detection pairs and concatenated with detection pair features (hatched boxes, corresponding features and detections have the same pattern).
  • Features of detection pairs are mapped independently through fully connected layers.
  • the variable number of pairs is reduced to a fixed-size representation by max-pooling. Pairwise computations are done for each detection independently.
  • the neural network consists of a dimensionality reduction, a pairwise detection context layer, 2 fully connected layers applied to each pair independently, pooling across detections, and two fully connected layers, where the last one increases dimensionality again.
  • the input and output of a block are added as in the Resnet architecture, cf.:
  • the first block receives zero features as inputs, so all information that is used to make the decision is bootstrapped from the detection pair features.
  • the output of the last block is used by three fully connected layers to predict a new score for each detection independently (cf. FIG. 4 a , 4 b ).
  • FIG. 4 a shows a schematic flow chart illustrating a training architecture of the neural network according to embodiments of the present disclosure
  • FIG. 4 b shows a schematic flow chart illustrating a test architecture of the neural network according to embodiments of the present disclosure.
  • FIGS. 4 a and 4 b are understood to be high level diagram of the disclosure.
  • In the training architecture of FIG. 4 a blocks are, as described in FIG. 3 .
  • “FC” denotes fully connected layers. All features in this diagram may have 128 dimensions (input vector and features between the layers/blocks).
  • the output is a scalar.
  • the neural networks may have 16 blocks.
  • the feature dimension for the detection features may be 128 and may be reduced to 32 before building the pairwise detection context.
  • the detection pair features may also have 32 dimensions.
  • the fully connected layers after the last block may output 128 dimensional features. When the feature dimension is changed, the ratio between the number of features in each layer is kept constant, so indicating the detection feature dimension is sufficient.
  • the forward pass over several stacked blocks can be interpreted as message passing. Every detection sends messages to all of its neighbours in order to negotiate which detection is assigned an object and which detections should decrease their scores. Instead of handcrafting the message passing algorithm and its rules, the network is configured to latently learn the messages that are being passed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
US16/608,399 2017-04-28 2017-04-28 System and method for detecting objects in a digital image, and system and method for rescoring object detections Abandoned US20200193225A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2017/060273 WO2018197019A1 (en) 2017-04-28 2017-04-28 System and method for detecting objects in a digital image, and system and method for rescoring object detections.

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2017/060273 A-371-Of-International WO2018197019A1 (en) 2017-04-28 2017-04-28 System and method for detecting objects in a digital image, and system and method for rescoring object detections.

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/562,859 Continuation US11715281B2 (en) 2017-04-28 2021-12-27 System and method for detecting objects in a digital image, and system and method for rescoring object detections

Publications (1)

Publication Number Publication Date
US20200193225A1 true US20200193225A1 (en) 2020-06-18

Family

ID=58671627

Family Applications (2)

Application Number Title Priority Date Filing Date
US16/608,399 Abandoned US20200193225A1 (en) 2017-04-28 2017-04-28 System and method for detecting objects in a digital image, and system and method for rescoring object detections
US17/562,859 Active US11715281B2 (en) 2017-04-28 2021-12-27 System and method for detecting objects in a digital image, and system and method for rescoring object detections

Family Applications After (1)

Application Number Title Priority Date Filing Date
US17/562,859 Active US11715281B2 (en) 2017-04-28 2021-12-27 System and method for detecting objects in a digital image, and system and method for rescoring object detections

Country Status (4)

Country Link
US (2) US20200193225A1 (ja)
JP (1) JP6889279B2 (ja)
DE (1) DE112017007492T5 (ja)
WO (1) WO2018197019A1 (ja)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210192245A1 (en) * 2017-10-30 2021-06-24 Shanghai Cambricon Information Technology Co., Ltd. Information processing method and terminal device
US11080884B2 (en) * 2019-05-15 2021-08-03 Matterport, Inc. Point tracking using a trained network
US11532095B2 (en) * 2017-12-01 2022-12-20 Canon Kabushiki Kaisha Apparatus, method, and medium for merging pattern detection results
US11967137B2 (en) 2021-12-02 2024-04-23 International Business Machines Corporation Object detection considering tendency of object location

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110826572B (zh) * 2018-08-09 2023-04-21 京东方科技集团股份有限公司 多目标检测的非极大值抑制方法、装置及设备
CN109540138B (zh) * 2018-11-12 2021-05-25 中南大学 基于视觉神经网络的室内导航方法、系统及可读存储器
KR20200063329A (ko) * 2018-11-21 2020-06-05 삼성전자주식회사 영상 처리 장치 및 그 제어방법
CN111415461B (zh) 2019-01-08 2021-09-28 虹软科技股份有限公司 物品识别方法及系统、电子设备
EP3832491A1 (en) * 2019-12-06 2021-06-09 Idemia Identity & Security France Methods for processing a plurality of candidate annotations of a given instance of an image, and for learning parameters of a computational model
US10998006B1 (en) * 2020-12-08 2021-05-04 Turku University of Applied Sciences Ltd Method and system for producing binaural immersive audio for audio-visual content

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9019538D0 (en) 1990-09-07 1990-10-24 Philips Electronic Associated Tracking a moving object
US5214744A (en) 1990-12-14 1993-05-25 Westinghouse Electric Corp. Method and apparatus for automatically identifying targets in sonar images
US6298143B1 (en) * 1997-04-03 2001-10-02 Kabushiki Kaisha Toshiba Moving target detecting system and moving target detecting method
AU8586098A (en) 1997-07-25 1999-02-16 Arch Development Corporation Method and system for the segmentation of lung regions in lateral chest radiographs
US6647139B1 (en) 1999-02-18 2003-11-11 Matsushita Electric Industrial Co., Ltd. Method of object recognition, apparatus of the same and recording medium therefor
US6549646B1 (en) 2000-02-15 2003-04-15 Deus Technologies, Llc Divide-and-conquer method and system for the detection of lung nodule in radiological images
JP4228641B2 (ja) * 2002-09-20 2009-02-25 セイコーエプソン株式会社 出力対象画像データ選択
JP4111198B2 (ja) * 2004-06-29 2008-07-02 セイコーエプソン株式会社 画像検索システム、画像検索プログラムおよび記憶媒体、並びに画像検索方法
JP2006010652A (ja) * 2004-06-29 2006-01-12 Toyota Motor Corp 物体検出装置
US8812473B1 (en) 2005-06-16 2014-08-19 Gere Dev. Applications, LLC Analysis and reporting of collected search activity data over multiple search engines
US9305088B1 (en) 2006-11-30 2016-04-05 Google Inc. Personalized search results
US7925072B2 (en) 2007-03-08 2011-04-12 Kla-Tencor Technologies Corp. Methods for identifying array areas in dies formed on a wafer and methods for setting up such methods
US8340435B2 (en) 2009-06-11 2012-12-25 California Institute Of Technology Method and system for object recognition search
JP5685031B2 (ja) * 2010-09-15 2015-03-18 キヤノン株式会社 画像処理装置および画像形成システムおよび画像形成方法
JP6448036B2 (ja) * 2015-09-03 2019-01-09 日本電信電話株式会社 物体領域特定方法、装置、及びプログラム
JP2017059207A (ja) * 2015-09-18 2017-03-23 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 画像認識方法
US9753949B1 (en) 2016-03-14 2017-09-05 Shutterstock, Inc. Region-specific image download probability modeling

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210192245A1 (en) * 2017-10-30 2021-06-24 Shanghai Cambricon Information Technology Co., Ltd. Information processing method and terminal device
US11762631B2 (en) * 2017-10-30 2023-09-19 Shanghai Cambricon Information Technology Co., Ltd. Information processing method and terminal device
US11532095B2 (en) * 2017-12-01 2022-12-20 Canon Kabushiki Kaisha Apparatus, method, and medium for merging pattern detection results
US11080884B2 (en) * 2019-05-15 2021-08-03 Matterport, Inc. Point tracking using a trained network
US11967137B2 (en) 2021-12-02 2024-04-23 International Business Machines Corporation Object detection considering tendency of object location

Also Published As

Publication number Publication date
US11715281B2 (en) 2023-08-01
US20220129701A1 (en) 2022-04-28
JP2020518076A (ja) 2020-06-18
DE112017007492T5 (de) 2020-02-13
JP6889279B2 (ja) 2021-06-18
WO2018197019A1 (en) 2018-11-01

Similar Documents

Publication Publication Date Title
US11715281B2 (en) System and method for detecting objects in a digital image, and system and method for rescoring object detections
Hosang et al. Learning non-maximum suppression
Yang et al. Unsupervised moving object detection via contextual information separation
EP0554083B1 (en) Neural network learning system
CN106846355B (zh) 基于提升直觉模糊树的目标跟踪方法及装置
Fathy et al. Learning with imbalanced data in smart manufacturing: A comparative analysis
Gattal et al. Segmentation and recognition system for unknown-length handwritten digit strings
US11295240B2 (en) Systems and methods for machine classification and learning that is robust to unknown inputs
EP3349152A1 (en) Classifying data
CN111126482A (zh) 一种基于多分类器级联模型的遥感影像自动分类方法
Cardoso et al. A bounded neural network for open set recognition
Tang et al. Classification for overlapping classes using optimized overlapping region detection and soft decision
CN111222589A (zh) 图像文本识别方法、装置、设备及计算机存储介质
Weber et al. Automated focal loss for image based object detection
WO2015146113A1 (ja) 識別辞書学習システム、識別辞書学習方法および記録媒体
CN116912796A (zh) 一种基于新型动态级联YOLOv8的自动驾驶目标识别方法及装置
CN113811894A (zh) 对车辆的驾驶功能的ki模块的监测
Karthik et al. No cost likelihood manipulation at test time for making better mistakes in deep networks
CN116229112A (zh) 一种基于多重注意力的孪生网络目标跟踪方法
CN111626291A (zh) 一种图像视觉关系检测方法、系统及终端
Munir et al. Bridging precision and confidence: A train-time loss for calibrating object detection
Verikas et al. Fusing neural networks through space partitioning and fuzzy integration
CN117372684A (zh) 一种基于改进YOLOv5s网络模型的目标检测方法
CN114724226B (zh) 表情识别模型训练方法、电子设备、存储介质
Kauschke et al. Towards neural network patching: Evaluating engagement-layers and patch-architectures

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE