WO2021155082A1

WO2021155082A1 - Computer-based systems configured for automated feature detection for image analysis and methods thereof

Info

Publication number: WO2021155082A1
Application number: PCT/US2021/015607
Authority: WO
Inventors: Kristian Sandberg; L. Dean RISINGER, Jr.
Original assignee: Reliant Immune Diagnostics, Inc.
Priority date: 2020-01-31
Filing date: 2021-01-29
Publication date: 2021-08-05

Abstract

Systems and methods of the present disclosure include processors (110) to receive an image, detect edges in the image according to object edges and determine edge measurements for each edge to produce line features. The line features are used to determine relative measurements between pairs of the line features and compare the relative measurements to model relative measurements associated with pairs of model line features of a model of the object-of-interest to identify matching pairs. The processor employ the matching pairs and the model to determine an object location for a matching object based on a scaling and rotation of the model according to differences between the relative measurements and the model relative measurements. The processors generate a modified image of the image to identify a boundary of the matching object based on the object location to identify the object and its position within the image.

Description

COMPUTER-BASED SYSTEMS CONFIGURED FOR AUTOMATED FEATURE DETECTION FOR IMAGE ANALYSIS AND METHODS THEREOF

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to United States Provisional Patent Application number 63/039,602 filed on 16 June 2020 and to United States Provisional Patent Application number 62/968,622 filed on 31 January 2020, which are both herein incorporated by reference in their entirety.

BACKGROUND OF TECHNOLOGY

[0002] Feature detection (or object detection) is a rich topic. One of the most straightforward methods for feature detection is template matching, where a model template is convolved with an image to generate a match map. However, such methods are sensitive to partial occlusions and to perspective distortions. Also, unless several templates are used, template matching does not handle rotations and scaling efficiently.

[0003] So-called key point-based methods overcome many of these drawbacks by generating local key points in an image and comparing the relative locations of such key points with a known model. One of the most well-known key point-based algorithms is scale-invariant feature transform (SIFT). SIFT and related methods can even handle situations where a significant portion of an object is occluded, and also handle rotations and scaling in an efficient manner. However, the local key points many of these algorithms rely on are typically only stable around sharp comers in an image. For objects that do not contain many corners, or in cases with noise obscuring comers, key point-based methods are often unreliable.

[0004] Recently, deep learning methods have increasingly been used for feature detection problems. If a large data set is provided (which often has to be labeled), a deep neural network can learn to recognize objects and automatically design optimal filters for detecting objects in a large variety of images. In cases where large training data sets and computational resources are available, these methods have proven very useful for certain tasks, such as computer vision for driverless cars and image-based search engines. However, deep learning methods are often only as good as the data that was used for training, and typically require large amounts of data to be reliable. Deep learning methods are also difficult to diagnose in instances of failures.

[0005] Thus, there is need for a feature detection technique that overcomes the problems described above. For example, a feature detection algorithm that does not rely on comer key points, among other localized descriptors, for better robustness to noise would advance the state of the art. Traditional approaches rely on highly localized descriptors that are highly sensitive to artifacts, such as blur, noise, and shadows.

SUMMARY OF DESCRIBED SUBJECT MATTER

[0006] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of receiving, by at least one processor, an image; detecting, by the at least one processor, at least one line in the image associated with a respective at least one object based on object edge-detection between pixels in the image; determining, by the at least one processor, line measurements for each respective at least one line to produce line features; determining, by the at least one processor, relative measurements between pairs of the line features; comparing, by the at least one processor, the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of-interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determining, by the at least one processor, an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements of the matching pairs and the model relative measurements; and generating, by the at least one processor, a modified image of the image to identify a boundary of the matching object based on the object location.

[0007] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection wherein the relative measurements of a line feature pair comprise a length difference, an angle difference, a midpoint distance and an object center line angle representing an angle of a line connecting a midpoint of each line feature of the line feature pair.

[0008] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of determining, by the at least one processor, an affinity value for each pair of a line feature pair of the pairs of line features and a model line feature pair of the pairs of model line features based on a difference between the respective relative measurements and the respective model relative measurements. [0009] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of identifying, by the at least one processor, the matching pairs of line features for each respective pair of the line feature pairs and the model line feature pairs having a respective affinity value that exceeds an affinity threshold.

[0010] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of determining, by the at least one processor, at least one object center vector for each line feature of each matching pair of line features based on a rotation of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model obj ect center vector; and determining, by the at least one processor, the object location in the image based on a location voting by overlapping locations of the at least one object center vector for each line feature of each matching pair of line features.

[0011] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of determining, by the at least one processor, an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determining, by the at least one processor, a refinement to the object scale in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0012] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of determining, by the at least one processor, a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determining, by the at least one processor, a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features. [0013] In some embodiments, the present disclosure provides an exemplary computer-based method for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following steps of determining, by the at least one processor, an angle consistency map according to an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determining, by the at least one processor, a refinement to the object angle in the image based on an orientation addition of the angle consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0014] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection that includes at least the following components of at least one processor configured to perform steps to: receive an image; detect at least one edge in the image associated with a respective at least one object edge in the image; determine edge measurements for each respective at least one edge to produce line features; determine relative measurements between pairs of the line features; compare the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of-interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determine an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements and the model relative measurements; and generate a modified image of the image to identify a boundary of the matching object based on the object location.

[0015] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the relative measurements comprise a length difference, an angle difference, a midpoint distance and an object center line angle representing an angle of a line connecting a midpoint of each line feature of a respective pair of line features.

[0016] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to determine an affinity value between each line feature pair and each model line feature pair based on a difference between the respective relative measurements and the respective model relative measurements.

[0017] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to identify the matching pairs of line features for each respective pair of the line feature pairs and the model line feature pairs having a respective affinity value that exceeds an affinity threshold.

[0018] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to: determine at least one object center vector for each line feature of each matching pair of line features based on a rotation of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model object center vector; and determine the object location in the image based on a location voting by overlapping locations of the at least one object center vector for each line feature of each matching pair of line features.

[0019] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to: determine an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determine a refinement to the object location in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0020] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to: determine a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determine a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0021] In some embodiments, the present disclosure provides an exemplary computer-based system for detecting an object-of-interest in an image using edge-based feature detection wherein the at least one processor is further configured to perform steps to: determine an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determine a refinement to the object location in the image based on an orientation addition of the object angle for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0022] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of receiving an image; detecting at least one edge in the image associated with a respective at least one edge in the image; determining edge measurements for each respective at least one edge to produce line features; determining relative measurements between pairs of the line features; comparing the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of-interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determining an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements and the model relative measurements; and generating a modified image of the image to identify a boundary of the matching object based on the object location.

[0023] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection wherein the relative measurements comprise a length difference, an angle difference, a midpoint distance and an object center line angle representing an angle of a line connecting a midpoint of each line feature of a respective pair of line features.

[0024] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of determining an affinity value for each pair of a line feature pair of the pairs of line features and a model line feature pair of the pairs of model line features based on a difference between the respective relative measurements and the respective model relative measurements.

[0025] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of identifying the matching pairs of line features for each respective pair of the line feature pair and the model line feature pair having a respective affinity value that exceeds an affinity threshold.

[0026] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of determining at least one object center vector for each line feature of each matching pair of line features based on a rotation of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model object center vector; and determining the object location in the image based on a location voting by overlapping locations of the at least one object center vector for each line feature of each matching pair of line features.

[0027] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of determining an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determining a refinement to the object location in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0028] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of determining a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determining a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

[0029] In some embodiments, the present disclosure provides an exemplary computer-based software product for detecting an object-of-interest in an image using edge-based feature detection that includes the following software instructions for causing a processor to perform steps of determining an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determining a refinement to the object location in the image based on an orientation addition of the object angle for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

DEFINITIONS

[0030] Unless defined otherwise, all technical and scientific terms used herein have the meaning commonly understood by a person skilled in the art to which this invention belongs. As used herein, the following terms have the meanings ascribed to them below, unless specified otherwise.

[0031] As used herein, the term “orientation” refers to an assignment of a weighted line segment to each pixel in an image to produce the orientation for each pixel having an angle and a weight indicating the importance of the orientation, as described in Sandberg, K., Brega, M. : Segmentation of thin structures in electron micrographs using orientation fields. J. Struct. Biol. 157, 403-415 (2007) (hereinafter “Sandberg 2007”) and Sandberg, K.: Curve enhancement using orientation fields. In Bebis, G. (ed.), Advances in Visual Computing, Part I. LNCS, vol. 5875, pp. 564-575, Springer, Heidelberg (2009) (hereinafter “Sandberg 2009”), both of which are herein incorporated by reference in their entireties.

[0032] As used herein, the term “orientation field” refers to An orientation field refers to the assignment of an orientation to each location in the image, as described in Sandberg 2007 and Sandberg 2009 incorporated by reference above.

[0033] By “line feature” is meant a line segment with associated metadata. [0034] By “edge” is meant a side or edge of an object, specifically a change in contrast or color within an image.

[0035] By “model” is meant a representation of a known object that the line-based feature detection algorithm is configured to detect, where the model has a known size, shape, center and orientation.

[0036] By “object-of-interest” is meant an object that the line-based feature detection algorithm is configured to detect within a target image.

[0037] By “target image” is meant an image and the contents thereof that is under analysis by the present system for detecting, based on line features, objects-of-interest.

[0038] By “match map” is meant a possibly multi-dimensional array of intensities over the target image that indicates the “likelihood” of a match with a model object in the target image at a given set of coordinates.

[0039] By “label” is meant a unique numerical value that identifies a line feature or model feature.

[0040] By “midpoint coordinate” is meant a two-dimensional (2D) vector representing a midpoint of a line feature.

[0041] By “length” is meant a measurement of length of a line feature.

[0042] By “angle” is meant a measurement of the angle of a line feature relative to an x-axis of an image; the angle is between 0 and p.

[0043] By “object center” is meant the coordinates of the center of an object.

[0044] By “obj ect center vector” is meant a 2D vector that points from a line feature’ s midpoint to the object center.

[0045] By “match identifier” is meant one or two numbers that refer to labels of other line features that a line feature might be associated with.

[0046] By “line feature space” is meant a set of line features.

[0047] By “edge map” is meant an image where edges and lines are enhanced or amplified.

[0048] By “translation” is meant a geometric transformation that moves an object’s points.

[0049] By “rotation” is meant a geometric transformation that rotates an object's points by a fixed angle about a fixed point. [0050] By “scale” is meant a geometric transformation that uniformly increases or decreases an object’s size by a fixed value.

[0051] By “midpoint distance” is meant a measurement of the relative distance between the midpoints of the two line features.

[0052] By “center line angle” is meant a measurement of the angle of a line connecting the midpoints of two line segments, relative to the angles of the two line segments.

[0053] By “scale factor” is meant the ratio of the size of an obj ect-of-interest in the target image relative to the same obj ect-of-interest in the model.

[0054] By “match affinity value” is meant a value between 0 and 1 that estimates how close a pair of target line features match a pair of model line features.

[0055] Unless specifically stated or obvious from context, as used herein, the term “or” is understood to be inclusive. Unless specifically stated or obvious from context, as used herein, the terms “a”, “an”, and “the” are understood to be singular or plural.

[0056] Unless specifically stated or obvious from context, as used herein, the term “and” may be used interchangeably with the term “or” to refer to a set of items in both the conjunctive and disjunctive in order to encompass the full description of combinations and alternatives of the items. By way of example, a set of items may be listed with the disjunctive “or”, or with the conjunction “and.” In either case, the set is to be interpreted as meaning each of the items singularly as alternatives, as well as any combination of the listed items.

[0057] Unless specifically stated or obvious from context, as used herein, the term “about” is understood as within a range of normal tolerance in the art, for example within 2 standard deviations of the mean. About can be understood as within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the stated value. Unless otherwise clear from context, all numerical values provided herein are modified by the term about.

[0058] The phrases “in one embodiment” and “in some embodiments” as used herein do not necessarily refer to the same embodiment(s), though it may. Furthermore, the phrases “in another embodiment” and “in some other embodiments” as used herein do not necessarily refer to a different embodiment, although it may. Thus, as described below, various embodiments may be readily combined, without departing from the scope or spirit of the present disclosure.

[0059] As used herein, the term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of "a," "an," and "the" include plural references. The meaning of "in" includes "in" and "on."

[0060] As used herein, the term “real-time” is directed to an event/action that can occur instantaneously or almost instantaneously in time when another event/action has occurred. For example, the “real-time processing,” “real-time computation,” and “real-time execution” all pertain to the performance of a computation during the actual time that the related physical process (e.g., a user interacting with an application on a mobile device) occurs, in order that results of the computation can be used in guiding the physical process.

[0061] As used herein, the term “dynamically” and term “automatically,” and their logical and/or linguistic relatives and/or derivatives, mean that certain events and/or actions can be triggered and/or occur without any human intervention. In some embodiments, events and/or actions in accordance with the present disclosure can be in real-time and/or based on a predetermined periodicity of at least one of: nanosecond, several nanoseconds, millisecond, several milliseconds, second, several seconds, minute, several minutes, hourly, several hours, daily, several days, weekly, monthly, etc.

[0062] As used herein, the terms “computer engine” and “engine” identify at least one software component and/or a combination of at least one software component and at least one hardware component which are designed/programmed/configured to manage/control other software and/or hardware components (such as the libraries, software development kits (SDKs), obj ects, etc.).

[0063] Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. In some embodiments, the one or more processors may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processors; x86 instruction set compatible processors, multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, the one or more processors may be dual-core processor(s), dual-core mobile processor(s), and so forth.

[0064] As used herein, the terms “computer-related systems”, “computer systems”, and “systems”, include any combination of hardware and software. Examples of software may include software components, programs, applications, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computer code, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints.

[0065] As used herein, the term “server” should be understood to refer to a service point which provides processing, database, and communication facilities. By way of example, and not limitation, the term “server” can refer to a single, physical processor with associated communications and data storage and database facilities, or it can refer to a networked or clustered complex of processors and associated network and storage devices, as well as operating software and one or more database systems and application software that support the services provided by the server. Cloud servers are examples.

[0066] As used herein, the term “mobile device” or “mobile electronic device,” or the like, may refer to any portable electronic device that may or may not be enabled with location tracking functionality (e.g., MAC address, Internet Protocol (IP) address, or the like). For example, a mobile electronic device can include, but is not limited to, a mobile phone, Personal Digital Assistant (PDA), Blackberry ™, Pager, Smartphone, or any other reasonable mobile electronic device.

[0067] As used herein, the terms “cloud,” “Internet cloud,” “cloud computing,” “cloud architecture,” and similar terms correspond to at least one of the following: (1) a large number of computers connected through a real-time communication network (e.g., Internet); (2) providing the ability to run a program or application on many connected computers (e.g., physical machines, virtual machines (VMs)) at the same time; (3) network-based services, which appear to be provided by real server hardware, and are in fact served up by virtual hardware (e.g., virtual servers), simulated by software running on one or more real machines (e.g., allowing to be moved around and scaled up (or down) on the fly without affecting the end user).

[0068] As used herein, the term “user” shall have a meaning of at least one user. In some embodiments, the terms “user”, “subscriber” “consumer” or “customer” should be understood to refer to a user of an application or applications as described herein and/or a consumer of data supplied by a data provider. By way of example, and not limitation, the terms “user” or “subscriber” can refer to a person who receives data provided by the data or service provider over the Internet in a browser session, or can refer to an automated software application which receives the data and stores or processes the data.

BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Various embodiments of the present disclosure can be further explained with reference to the attached drawings, wherein like structures are referred to by like numerals throughout the several views. The drawings shown are not necessarily to scale, with emphasis instead generally being placed upon illustrating the principles of the present disclosure. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ one or more illustrative embodiments.

[0070] FIG. l is a block diagram of an illustrative computer-based image analysis system and platform configured for image analysis utilizing line-based feature detection in accordance with one or more embodiments of the present disclosure.

[0071] FIG. 2 is a block diagram of an exemplary feature extraction engine of an image analysis system for line-based feature detection in accordance with one or more embodiments of the present disclosure;

[0072] FIG. 3 illustrates comparing edge pairs in a target image to known edge pairs in a model to infer the location in accordance with one or more embodiments of the present disclosure;

[0073] FIG. 4 illustrates a flowchart of an illustrative methodology for object extraction in accordance with one or more embodiments of the present disclosure;

[0074] FIG. 5 A illustrates model line features (labeled 0, 1, 2, and 3) high-lighted in accordance with one or more embodiments of the present disclosure along with the model center, represented by a cross;

[0075] FIG. 5B illustrates a target image in accordance with one or more embodiments of the present disclosure;

[0076] FIG. 6 illustrates two line features labeled 1 and 2 (£₁ and f ₂f along with their midpoints _{1 2}, angles 0_{l 2}, and object center vectors c_{1 2}in accordance with one or more embodiments of the present disclosure. The center of the object is marked with a cross, which we have put in the center of the rectangle. In our approach, the location of the center is in fact arbitrary, as long as all line features of the object share the same center;

[0077] FIG. 7 illustrates a model object with two line features that each have four object center vectors associated with them in accordance with one or more embodiments of the present disclosure. FIG. 7 exemplifies an even more complex situation that can arise, which requires not only two, but four object center vectors. This case happens if the angle between a center vector c is not perpendicular to the line segment it is associated with;

[0078] FIG. 8A illustrates step edges in accordance with one or more embodiments of the present disclosure;

[0079] FIG. 8B illustrates peak edges in accordance with one or more embodiments of the present disclosure;

[0080] FIG. 9A illustrates an edge map generated for short edges in FIG. 5B in accordance with one or more embodiments of the present disclosure;

[0081] FIG. 9B illustrates an edge map generated for long edges in FIG. 5B in accordance with one or more embodiments of the present disclosure;

[0082] FIG. 10 illustrates line features identified for the target image in FIG. 5B in accordance with one or more embodiments of the present disclosure;

[0083] FIG. 11 is a flowchart illustrating the methodology used to generate a match map in accordance with one or more embodiments of the present disclosure;

[0084] FIG. 12A illustrates a target image in accordance with one or more embodiments of the present disclosure;

[0085] FIG. 12B illustrates a match map for the target image of FIG. 12A where pixel intensity indicates the “strength” of the match with the model in FIG. 5 A and the location indicates the predicted center of the model in accordance with one or more embodiments of the present disclosure;

[0086] FIG. 13 illustrates an angle difference which measures the cosine of twice the angle difference between two line segments in accordance with one or more embodiments of the present disclosure. In this example, the angle difference value would be /_{de (}A.G) = cos = -l· The length difference value would be

Note that these

values would not change if the rectangle is translated, rotated, or re-sized; [0087] FIG. 14 illustrates a midpoint distance which measures the relative distance between the midpoints of the two line segments in accordance with one or more embodiments of the present disclosure. In this example, the midpoint difference value would be f_dm (A, ^2) ⁼

~~ ~ 0.56. Note that these values would not change if the rectangle is translated, rotated, or re-sized;

[0088] FIG. 15 illustrates projections of the unit vector of the line connecting the midpoints onto the directions of the two line segments. These projections provide a useful measure to classify pairs of line features in accordance with one or more embodiments of the present disclosure;

[0089] FIG. 16 illustrates a flowchart of the main steps for finding matching line feature pairs in accordance with one or more embodiments of the present disclosure;

[0090] FIG. 17 illustrates a case where the two line features in the pair have different relative lengths, and can be unambiguously matched to line features in the model in accordance with one or more embodiments of the present disclosure;

[0091] FIG. 18 illustrates a case where two line features in the pair have same relative lengths, and cannot be unambiguously matched to line features in the model. In this case, which model label that is assigned the match identifier for the target line feature is arbitrary in accordance with one or more embodiments of the present disclosure;

[0092] FIG. 19 illustrates a case where two line features in the pair have same relative lengths, but the associated object center vectors have different angles with respect to the line feature. In this case, both model labels are assigned as match identifiers to each target line feature in accordance with one or more embodiments of the present disclosure;

[0093] FIG. 20 illustrates a flowchart of the main steps for estimating the object center locations in accordance with one or more embodiments of the present disclosure;

[0094] FIG. 21 illustrates an example of how the relative scale factors Sy and angles 0y are estimated for a given matching line pair in accordance with one or more embodiments of the present disclosure;

[0095] FIG. 22 illustrates an example of how to estimate the object center vectors for the target line features in accordance with one or more embodiments of the present disclosure. Note that for clarity of the illustration, only two of the line feature matches from FIG. 19 are considered. Note that one of the line features has two object center vectors, while the other one has four such vectors;

[0096] FIG. 23 illustrates an example of center vectors for a line feature pair in accordance with one or more embodiments of the present disclosure. Even though there are 2 X 4 = 8 pairs of center vectors associated with the line features, only one such pair,

agree (approximately) on a location. The dotted circle indicates the uncertainty we allow for when determining if two center vectors overlap. All obj ect center vectors

are shown for this target line feature pair. The goal is to determine which center vectors point to roughly the same location; since there will be inaccuracies in line feature extraction and in measurements, center vectors from the two line features are unlikely to point to exactly the same location. Therefore, center vectors may agree on a location within a certain tolerance (indicated by dotted circles in the FIG );

[0097] FIG. 24 illustrates another example of center vectors for a line feature pair in accordance with one or more embodiments of the present disclosure. In this case, both the pairs fl) fl) fl) f2)

(c) , c₂ ) and (c) , c₂ ) overlap. In such cases, either both of these vector pairs may be kept for the next step of the algorithm, or a mean vector may be formed;

[0098] FIG. 25 illustrates the final estimated location for a matching line feature pair with the associated center vectors; the location is computed as the mean of the center vector end points (marked by a cross in the FIG.) in accordance with one or more embodiments of the present disclosure;

[0099] FIG. 26 illustrates an example of one matching target line feature pair when applied to the image in FIG. 5B with the red lines showing a target line feature pair that match model features 2 and 3 in FIG. 5A in accordance with one or more embodiments of the present disclosure. The green and blue lines indicate the center object vectors associated with each line feature. The circles indicate the “radii of uncertainty” associated with each center vector. If these circles overlap, the line feature pair will generate an estimated target location at the mean of the associated circle centers, which is denoted by the cross;

[0100] FIG. 27 illustrates all matching pairs for the example of FIGs. 5A and 5B in accordance with one or more embodiments of the present disclosure. Illustration of all matching line features (in red) and their associated matching object center vectors (in green). The dotted blue lines indicate line feature pairs that were found to match line feature pairs in the model. The green filled squares represent estimated object centers from the matching object center vectors;

[0101] FIG. 28 illustrates a close up of the matching pairs and their associated center vectors for the tilted rectangle of FIG. 27 in accordance with one or more embodiments of the present disclosure. Note that the estimated center locations (green squares) do not match exactly. For this reason each vote may cover a neighborhood of pixels. The overlapping votes then give a type of mean object location;

[0102] FIG. 29 illustrates a match map based on FIG. 27 in accordance with one or more embodiments of the present disclosure. The centers of the blurred square regions provide accurate estimates of the center locations of the rectangles to be located. Here all estimated object center locations have been summed, using a square neighborhood of radius 10 pixels around each location estimate. We note that there is one “false” match in the FIG. 12B, which will be typical in most application. However, such stray matches do not give a large response compared to all the overlapping matches that agree on a location. Therefore, the “stray” match is barely visible in the final match map;

[0103] FIG. 30 illustrates a scale map corresponding to the target image in FIG. 5B in accordance with one or more embodiments of the present disclosure. The brightness indicates the magnitude of the scale factor;

[0104] FIG. 31 illustrates a scale consistency map corresponding to the target image in FIG. 5B in accordance with one or more embodiments of the present disclosure;

[0105] FIG. 32 illustrates an angle consistency map corresponding to the target image in FIG. 5B where the weights of all the added orientations are depicted in accordance with one or more embodiments of the present disclosure;

[0106] FIG. 33 illustrates a first example of a target image for the first example application of line-based feature detection in autocapture for a diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0107] FIG. 34 illustrates a downsampled target image for the first example application of line- based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure; [0108] FIG. 35 illustrates an edge map derived from orientation field processing for the target image for the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0109] FIG. 36 illustrates line features extracted from the edge map of the target image for the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0110] FIG. 37 illustrates a Hough scale map used to determine the scale of the matching object (cassette) in the target image using line-based feature detection in the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0111] FIG. 38 illustrates a Hough scale consistency map used, in conjunction with the Hough location map, to find the object center with the most consistent scale in the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0112] FIG. 39 illustrates a Hough angle map used to determine the object orientation in the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0113] FIG. 40 illustrates a final, processed Hough Map used to determine object location to be used in conjunction with the Hough angle and Hough scale maps to determine the scale and angle of the object in the target image using line-based feature detection in the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0114] FIG. 41 illustrates an object location including object comers and target (center) identified in the target image using line-based feature detection in the first example application of line-based feature detection in autocapture for the diagnostic flu test in accordance with one or more embodiments of the present disclosure;

[0115] FIG. 42 illustrates a target image for feature detection in a second example application of line-based feature detection for the diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0116] FIG. 43 illustrates line features extracted from an edge map produced using orientation field techniques on the target image for line-based feature detection in the second example application of feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0117] FIG. 44 illustrates a graphical representation of the flu cassette model used by the line- based feature detection algorithm with, e.g., object corners and target (e.g., the test strip in the center of the flu cassette) marked with crosses, where a single round of feature detection serves to both locate the cassette and obtain the first estimate of the test strip location using line-based feature detection in the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0118] FIG. 45 illustrates a Hough location match map for object location where the bright spot right of center is the maximum of the match map, and indicates the location of the object target within the image using line-based feature detection in the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0119] FIG. 46 illustrates the object target and comer locations from FIG. 45 superimposed on the original image, where the object comers correspond to the corners of the test strip and the object target corresponds to the center of the test strip with a padded crop of this region as input for the next step including fine-tuning according to the second example application of line- based feature detection in diagnostic flu test image analysis test in accordance with one or more embodiments of the present disclosure;

[0120] FIG. 47 illustrates a cropped image of the test strip and surrounding area from the original image depicted in FIG. 42, that is passed as input to the fine-tuning step in the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0121] FIG. 48 illustrates a Hough location match map resulting from the fine-tuning step, where the fine-tuning step uses a related feature detection algorithm that works with curves instead of lines according to the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0122] FIG. 49 illustrates the boundary of the flu test strip identified by the fine-tuning step according to the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure; [0123] FIG. 50 illustrates a final image of the flu test strip after fine-tuning, where the final image may be passed on to a line classifier and other code to determine a diagnostic result based on the number and location of the lines detected in the test strip according to the second example application of line-based feature detection in diagnostic flu test image analysis in accordance with one or more embodiments of the present disclosure;

[0124] FIG. 51 depicts a block diagram of an exemplary computer-based system and platform 5100 in accordance with one or more embodiments of the present disclosure;

[0125] FIG. 52 depicts a block diagram of another exemplary computer-based system and platform 5200 in accordance with one or more embodiments of the present disclosure;

[0126] FIG. 53 illustrate schematics of exemplary implementations of the cloud computing architecture in which the illustrative computer-based systems of the present disclosure may be specifically configured to operate; and

[0127] FIG. 54 illustrate schematics of exemplary implementations of the cloud computing platform architecture in which the illustrative computer-based systems or platforms of the present disclosure may be specifically configured to operate.

DETAILED DESCRIPTION

[0128] Various detailed embodiments of the present disclosure, taken in conjunction with the accompanying FIGs., are disclosed herein; however, it is to be understood that the disclosed embodiments are merely illustrative. In addition, each of the examples given in connection with the various embodiments of the present disclosure is intended to be illustrative, and not restrictive.

[0129] Systems and methods are described herein for identifying objects in images by employing technically improved feature detection that improves on the technologies of computer vision and image recognition through more robust and accurate feature detection for object localization and identification. Embodiments of the present invention are based on comparing pairs of line features in an image with known relations of line feature pairs that have been pre-determined and stored in a model object. Embodiments of the present invention are robust to variations in color and contrast, and are also translational, rotational, and scale invariant.

[0130] By using orientation fields, object edges and associated line features may be generated robustly even in very noisy images. However, the feature detection method can be used in conjunction with line feature extraction technique such as any edge-finding method, e.g., Sobel edge detection, or Radon transform-based algorithms.

[0131] In some embodiments, the present invention improves upon SIFT-like key point-based methods by employing several differences that enable embodiments of the present invention to work in cases where SIFT-like methods perform poorly.

[0132] For example, by utilizing line features, which can be extended throughout the image, feature descriptors of embodiments of the present invention can be global in nature, as opposed to traditional approaches that rely on highly localized descriptors. Localized descriptors are highly sensitive to artifacts, such as blur, noise, and shadows.

[0133] In some embodiments, the present invention employs orientation fields to produce the line features, which are generated by integration as described in Sandberg 2007 and Sandberg 2009. Existing methods use localized gradient vectors, which are based on differentiation, and therefore highly sensitive to small-scale artifacts.

[0134] In some embodiments, line-based feature detection may be utilized with particularly high accuracy in identifying medical test devices in cell phone images, which makes the method useful for telemedicine applications.

[0135] FIG. l is a block diagram of an illustrative computer-based image analysis system and platform configured for image analysis utilizing line-based feature detection in accordance with one or more embodiments of the present disclosure.

[0136] In some embodiments, analysis of an image 101 captured by an image capture device 102 may be effectuated with an image analysis system 110 that robustly and efficiently detects features of the image 101. Tasks such as object detection, color analysis, object recognition, classification, and other image analysis tasks rely on the identification, location, and extraction of features associated with objects-of-interest. However, where the feature extraction is susceptible to image imperfections, such as, e.g., noise, glare, discoloration, shadow, occlusions, among other imperfections in the image 101, the image analysis tasks decrease in reliability and accuracy. Thus, employing feature extraction that is robust to these imperfections improves the quality, efficiency, and accuracy of image analysis.

[0137] Thus, in some embodiments, an image analysis system 110 may employ a feature extraction engine 116 to robustly detect and extract features associated with objects-of-interest in the image 101. In some embodiments, the feature extraction engine 116 utilizes, e.g., a line- based feature detection algorithm that identifies line features, such as edges in the image 101 corresponding to shapes or objects depicted therein. The edges are then used as line features that provide improved robustness to imperfections compared to key point-based techniques. The line features may extend across the image 101, thus noise, occlusions, shadows, glare, and other inaccuracies with the image 101 may only affect parts of the line features. Moreover, even where the imperfections affect the whole line feature, properties of the line feature may still be deduced that may nevertheless provide identifiable attributes that may be used in identifying features of interest. For example, properties such as, e.g., length, orientation, scale, etc., may still be identifiable, even where noise and other artifacts affect part or all of a given line feature.

[0138] Accordingly, in some embodiments, the image analysis system 110 may receive the image 101 from the image capture device 102 and convert the image 101 into a signal suitable for processing by the feature extraction engine 116 using an image-to-signal engine 114. For example, the image 101 may be a lossy or lossless raster format file such as, e.g., Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), Portable Network Graphics (PNG), Exchangeable image file format (Exit), Graphics Interchange Format (GIF), Windows bitmap (BMP), portable pixmap (PPM) or other formats from the Netpbm format family, WebP, High Efficiency Image File Format (HEIF), BAT, Better Portable Graphics (BPG), or a lossy or lossless vector format such as, Computer Graphics Metafile (CGM), Gerber (RS-274X), Scalable Vector Graphics (SVG), or other formats and combinations thereof. The file format of the image 101 may depend on the image capture device 102, such as the format used by a digital camera or smartphone, which can vary from device to device.

[0139] In some embodiments, the image-to-signal engine 114 ingests the file of the image 101 and generates a signal using a selected color space. In some embodiments, the color space is predefined according to the configuration of the image analysis system 110. However, in some embodiments, the color space can be selected by, e.g., a user or selected automatically based on the format of the file for the image 101. The image-to-signal engine 114 creates the color- space signal by analyzing each pixel of the image 101 and measuring values for the pixel for each channel of the color space. For example, in some embodiments, the color space is the YTJV color space. The YTJV color space refers to a color space having a luminance channel (Y) or luma channel (Y’), a blue projection channel (U) and a red projection channel (V). In some embodiments, the YUV color space takes the form of YCbCr, where Cb is a blue- difference chroma channel and Cr and red-difference chroma channel. The YUV color space may alternatively take the form of YPbPr, where Pb is the difference between the blue projection and the luma and Pr is the difference between the red projection and the luma. Thus, in some embodiments, the image-to-signal engine 114 generates a three-channel color space measurement for each pixel in the image 101. However, other color spaces are also contemplated, such as those described above, that may have more, less or the same number of channels as YUV.

[0140] In some embodiments, the feature extraction engine 116 may utilize the line feature detection techniques to process the image signal generated by the image-to-signal engine 114 and produce the features of the image 101. For example, the feature extraction engine 116 may use the image signal to produce an edge map identifying edge pixels in the image 101. The line features formed by the edge pixels may then used to, e.g., match model objects to shapes formed by the line features, e.g., by comparing relative scales, orientations, midpoints, and other measurements between pairs of line segments. A match map may then be produced that identifies objects-of-interest in the image 101.

[0141] In some embodiments, objects and object locations may then be provided to an image analysis engine 118, e.g., along with the image 101 or image signal, to undergo image analysis. Due to the use of line feature detection by the feature extraction engine 116, the objects and object locations are reliably and accurately identified relative to the image 101. Thus, the image analysis engine 118 may perform image analysis with a high degree of confidence that the objects and locations have been correctly identified. Thus, the image analysis may be performed in a more accurate and efficient fashion that is more robust to noise and other artifacts.

[0142] In some embodiments, the image analysis by the image analysis engine 118 may be, e.g., an autocapture analysis that determines whether an obj ect-of-interest is within the frame of the image 101, or a color-based analysis, such as, e.g., medical diagnostic tests such as urine tests strips or blood test strips. In some embodiments, the image analysis may include, e.g., a machine learning model recognizing or otherwise classifying the results of the analysis of the image 101.

[0143] In some embodiments, the machine learning model of the image analysis engine 118 may utilize one or more exemplary AI or machine learning techniques chosen from, but not limited to, decision trees, boosting, support-vector machines, neural networks, nearest neighbor algorithms, Naive Bayes, bagging, random forests, and the like. In some embodiments and, optionally, in combination of any embodiment described above or below, an exemplary neutral network technique may be one of, without limitation, feedforward neural network, radial basis function network, recurrent neural network, convolutional network (e.g., U-net) or other suitable network. In some embodiments and, optionally, in combination of any embodiment described above or below, an exemplary implementation of Neural Network may be executed as follows: i) define Neural Network architecture/model, ii) transfer the input data to the exemplary neural network model, iii) train the exemplary model incrementally, iv) determine the accuracy for a specific number of timesteps, v) apply the exemplary trained model to process the newly-received input data, and vi) optionally and in parallel, continue to train the exemplary trained model with a predetermined periodicity.

[0144] In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary trained neural network model may specify a neural network by at least a neural network topology, a series of activation functions, and connection weights. For example, the topology of a neural network may include a configuration of nodes of the neural network and connections between such nodes. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary trained neural network model may also be specified to include other parameters, including but not limited to, bias values, functions and aggregation functions. For example, an activation function of a node may be a step function, sine function, continuous or piecewise linear function, sigmoid function, hyperbolic tangent function, or other type of mathematical function that represents a threshold at which the node is activated. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary aggregation function may be a mathematical function that combines (e.g., sum, product, etc.) input signals to the node. In some embodiments and, optionally, in combination of any embodiment described above or below, an output of the exemplary aggregation function may be used as input to the exemplary activation function. In some embodiments and, optionally, in combination of any embodiment described above or below, the bias may be a constant value or function that may be used by the aggregation function and/or the activation function to make the node more or less likely to be activated. [0145] In some embodiments, the image analysis results or the detected features, or both, may be provided to a user via, e.g., a user computing device 103 through 105 in communication with the image analysis system 110 via a suitable graphical user interface (GUI). For example, the results and/or features may be stored in a storage system 103, such as, e.g., a database, a cloud storage service, a server, a hard drive, a solid-state drive, or other storage mechanism, combinations thereof and systems including one or more thereof. Similarly, the results and/or features may be provided to the user at a mobile device 104 or computing device 105 associated with the user.

[0146] In some embodiments, a processing system 112 may be configured to implement the image-to-signal engine 114, the feature extraction engine 116 and the image analysis engine 118. As used herein, the terms “computer engine” and “engine”, and “model” identify at least one software component and/or a combination of at least one software component and at least one hardware component which are designed/programmed/configured to manage/control other software and/or hardware components (such as the libraries, software development kits (SDKs), objects, etc.). Each of the image-to-signal engine 114, the feature extraction engine 116 and the image analysis engine 118 may therefore be formed of software components (e.g., logic, algorithms, machine learning models, or other software components), hardware components (e.g., processors, memory, buffers, storage, cache, etc.), or combinations thereof.

[0147] FIG. 2 is a block diagram of another exemplary feature extraction engine of an image analysis system for line-based feature detection in accordance with one or more embodiments of the present disclosure.

[0148] In some embodiments, the feature extraction engine 116 may implement line-based feature detection algorithm for improved robustness against imperfections, noise, occlusions and artifacts in digital imagery. In some embodiments, by matching pairs of edges of shapes between a known model and shapes in the imagery, object location can be inferred using the model object center in a robust and efficient manner (see also FIG. 3).

[0149] In some embodiments, to detect an object-of-interest and infer the object-of-interest’s location using a matching known model object, a line feature generator 210 may represent an image 101 using line-like features, such as edges (e.g., detected by edge generator 201), to provide a line- or edge-based representation of the image 101.

[0150] In some embodiments, the line feature generator 210 may consider pairs of line features in the image 101, and record translational, rotational, and scale invariant measures of each pair. Such measures may include the relative lengths and angles between line features in a pair, as well as other relative measurements and combinations thereof. By using relative measurements, such as the relative lengths and the relative angles between any two lines, such measures will not change if the image 101 is translated, rotated, or re-sized. Accordingly, object detection is independent of the orientation of a setting within the frame of an image, the zoom length or proximity of the setting in the frame, among other variations in the capture of the image 101.

[0151] In some embodiments, a match map generator 220 may compare measures of line feature pairs in the target image 101 with known line feature pairs in the known model recorded in a model library 202.

[0152] In some embodiments, if a line feature pair in the target image 101 matches a line feature pair in the model, the match map generator 220 may use the known object center, rotation, and size of the object in the model to infer the center, rotation, and size in the target image 101.

[0153] In some embodiments, the match map generator 220 may use all matching line feature pairs to vote on the object center. Multiple line feature pairs matching a model object consistently tend to vote on a common location. This results in a match map (or Hough map), that indicates the likelihood of a match in an image.

[0154] In some embodiments, the feature extraction engine 116 may output the object location 203 resulting from the vote procedure. The feature extraction engine 116 may also identify object characteristics such as object angle or orientation 204 and object scale 205 in the image 101 based on a voting for orientation and voting for scale. However, other object characteristics may be identified, including any characteristics that may be defined relative to the image 101 frame or relative to other shapes in the image frame 101. For example, length, location, translation, and other size and distance measures indicative of object location 203 and object scale 205 may be determined based on measurements in pixels of the image 101 (e.g., 50 pixels long for a line feature length). Alternatively, the object location 203 and the object scale 205 measures may be determined based on measurements of a percentage or ratio of distance across the image 101 frame. Similarly, rotation or angle may be measured relative to an x-axis (width) or a y-axis (height) of the image 101.

[0155] FIG. 4 illustrates a flowchart of an illustrative methodology for object extraction using a line-based feature detection algorithm in accordance with one or more embodiments of the present disclosure of line-based feature detection. FIGs. 5-10 illustrate example images of line feature generation using example objects.

[0156] In some embodiments, the line-based feature detection algorithm, e.g., as implemented by the feature detection engine 116 employs line features in both the target image 101 and the model. A line feature is a line with associated properties that allow it to be identified and used in a matching algorithm. FIG. 5 A illustrates examples of model line features (labeled 0, 1, 2, and 3) high-lighted in accordance with one or more embodiments of the present disclosure, and FIG. 5B illustrates a target image in accordance with one or more embodiments of the present disclosure.

[0157] In some embodiments, the output of the algorithm is one or more coordinates that denotes the location of the object(s) in the target image, along with an angle and scale that denote the orientation and size of each object respectively. The one or more coordinates can relate to a location for a center of an object, a comer or comers of an object, edge midpoints of an object, or other part of the object.

[0158] In some embodiments, the main steps of the algorithm can be summarized as follows:

1) Generate the line features 411 in the target image 101 at block 410,

2) Compute the match map at block 420. This is a map over (sharing the pixel coordinate system of) the target image 101 where the intensity indicates the “likelihood” of a match in the target image 101, and

3) Find the object at block 430. In this step, the match map 421 is analyzed to determine any match in the image 101, and also to determine the orientation and the size of the match.

[0159] In order to describe the algorithm, the following description employs a simple example of detecting and locating rectangles, e.g., of aspect ratio 2 to 1 in an image with various shapes. The model is illustrated in FIG. 5 A by a rectangle along with the four lines describing the edges of the rectangle. The target image consists of two rectangles (of different scales and rotations), along with some other shapes (see FIG. 5B). The goal is to detect the two rectangles of aspect ratio 2 to 1 in the target image.

Line Features

[0160] A model is represented by a collection of (model) line features 402. In the test image 101, a feature extraction engine 116 generates target line features 411, e.g., at block 410, and compares them to model lines features 402 to determine matches between target line features 411 and model line features 402.

[0161] Definition 1 (Line feature) A line feature is an object denoted as f with the following properties (cf. FIG. 4-6):

1) Label - A unique numerical value that identifies a line feature.

2) Midpoint coordinate - The midpoint coordinate of the line represented as a 2D vector denoted as m.

3) Length - The length l of the line feature.

4) Angle - A value between 0 and p representing the angle Q of the line with respect to the x- axis.

[0162] In addition, there are two more properties that are not inherent to the line feature itself, but rather to how it relates to the object it belongs to:

1) Object center vector - A list of two or four 2D vectors c®, i E (1,2, 3, 4} that point from the line feature's midpoint m to the object center.

2) Match identifier - One or two numbers that refer to labels of model line features that a line feature might be associated with. For example, a target line feature labeled 3, might match a line feature in the model labeled 1. In this case, the target’s line feature’s label is 3, while the match identifier value is 1. The match identifier is further described below, e.g., with respect to FIGS. 17-19 and 21.

[0163] FIG. 6 illustrates two line features labeled 1 and 2 (£₁ and f ₂f along with their midpoints m_{l 2}, angles q_{1 2}, and object center vectors c_{1 2.}

[0164] In some embodiments, computing the match map 420 between the line features 411 and the model line features 402 can include two functions for extracting the angles and lengths of a line feature: Q( ) = Q and L(- ) = 1.

[0165] In some embodiments, to facilitate producing the match map 421, the algorithm may employ the notion of a line feature space.

[0166] Definition 2 (Line feature space) Let V denote the set of all line features £. We refer to V as the line feature space. [0167] In FIG. 5 A the center of the object is marked with a cross, which we have put in the center of the rectangle. In our approach, the location of the center is in fact arbitrary, as long as all line features of the obj ect share the same center.

Object Center Vectors

[0168] In some embodiments, as can be seen in FIG. 6, there are two object center vectors c® for each line feature. In order to ensure that matches may be detected in a rotationally invariant way each line feature 411 may have at least two center vectors. For example, line feature f _x can be associated with either the bottom part of the model, or its upper edge. When given a line feature 411 in isolation that matches the long edge of a model object, the line feature 411 itself does not have enough information to identify whether it belongs to the upper or lower edges of the model object. Thus, two opposite pointing vectors are employed to accommodate for both cases.

[0169] In the example in FIG. 6, there is a symmetry that hides an even more complex situation that can arise, which will require not only two, but four object center vectors. This case happens if the angle between a center vector c is not perpendicular to the line segment it is associated with. See, for example, FIG. 7 for an illustration of the more general case where four object center vectors c are employed for each line feature of the line feature pair. In the example illustrated in FIG. 7, if only line feature f _x is given, without knowing where the object center is, it may be difficult or impossible to determine whether the center vector should point up or down from the line feature, or if the center was located to the left or right from the line segment’s midpoint mi. For that reason, storing four possible object center vectors with the line feature allows for taking either situation into account.

[0170] In some embodiments, as described below, the line features 411 and the model line features 402 may be matched to compute a match map 421 at block 420. In some embodiments, first, an algorithm to generate line features at block 410 is employed, and then the match map is computed at block 420.

Algorithm I: Generating Line Features

[0171] The feature detection algorithm leverages the line features described above. In some embodiments, the line features may be generated from line segments or edges detected according to any suitable method. Any technique for generating line segments associated with objects represented in the image 101 is contemplated as long as the line features conform to Definition 1 above.

[0172] In some embodiments, algorithms for generating line features may involve the following main steps:

1) use some edge or line detection method(s) to generate an image where lines and edges are enhanced, which may be referred to as the edge map;

2) threshold the edge map;

3) segment the thresholded edge map by tracking (straight) edge sets in the thresholded edge map; and

4) find and record the midpoint, length, and angle of each segmented edge set and generate a line feature from these numbers.

[0173] In some embodiments, a number of methods can be used to generate the edge map. For example, the edge map may be generated using, e.g., a Sobel edge detector, a Radon transform, key-points or key-point pairs, curve-field transforms, or other edge detection technique. In some embodiments, an orientation field-based approach may used to detect edges and generate line features therefrom, as described in Sandberg 2007 and Sandberg 2009, incorporated by reference above.

[0174] In some embodiments, edges may fall into one or both of two main types of edges that are valuable to segment. The two types may include step edges and peak edges (see FIG. 8). A step edge refers to an edge that has a low intensity on one side, and a higher intensity on the other side. For a step edge, the exact (ideal) location would be right at the transition between the two intensities. A peak edge refers to a line with one intensity level that is either higher or lower than the intensity on both sides of the edge. For a peak edge, the exact (ideal) location of the edge would be the halfway point between the two intensity transitions. Depending on the image, it can be beneficial to use different methods to robustly find both of these edge types using, e.g., a method or algorithm that identifies both types, or a combination of methods or algorithms.

[0175] In some embodiments, generating line features can include marking the “ideal” edge location in the image 101. Edges may have different scales, where some may be the short, while others are longer. Hence, a single edge or line detector may not be suitable to capture all edges. For example, a Sobel edge detector might be suitable for short, step edges, whereas the Radon transform may be more suitable for long peak edges. In some embodiments, using the orientation field-based approach may facilitate the selection of a target scale that will favor either short or long lines. FIGS. 9A and 9B illustrate examples of the edge maps generated for the example in FIG. 5B to detect short and long edges. In particular, FIG. 9A depicts an edge map generated for short edges and FIG. 9B depicts an edge map generated for long edges.

[0176] In some embodiments, some edges (and their resulting line features) may be generated in more than one edge map. In such cases, line consolidation may be used to remove duplicate line features. Although this step is not necessary, it can be beneficial in some cases. For example, the complexity of the algorithm scales (roughly) quadratically with the number of line features in the target image, so line consolidation can be used to decrease the number of line features to be processed. In addition, if some edges are represented by many (nearly identical) line features, such edges will be favored in the final feature detection. See FIG. 10, which illustrates the line features we generated for the target image in FIG. 5B.

Algorithm II: Computing the Match Map

[0177] FIG. 11 illustrates a flowchart of generating the match map in accordance with one or more embodiments of the present disclosure of line-based feature detection. FIGs. 12-15 illustrate example images of match map generation using example objects.

[0178] In some embodiments, the line features 411 and the model line features 402 are used to generate a match map 421 at block 420. The purpose of the match map is to generate a map of positive values over the pixel coordinate system of the target image where the value indicates the likelihood of a match with the object-of-interest. In some embodiments, the match map 421 may be generated as, e.g., a Hough map or heat map, or other suitable mapping to depict the likelihood of a match. See FIG. 12 A and 12B for an example where FIG. 12A illustrates the target image 101 and FIG. 12b illustrates the match map 421. The pixel intensity indicates the “strength” of the match with the model in FIG. 5B and the location indicates the predicted center of the model. For example, the pixels having greater intensity (depicted as a brighter white on a black background) are marked off as “True Matches” while the dim, low intensity pixels are marked off as “False Matches.”

[0179] In some embodiments, the main steps of the algorithm are:

1) find matching pairs of line features 1123 in the target image 101 and in the model at block

1122, 2) for each matching pair 1123, estimate where the pair indicates the center of the object is located at block 1124. For example, in some embodiments, by comparing the scale and orientation of the line feature 411 pair in the target image 101 to the scale and orientation of the matching pair of the model line features 402, the known object center location of the model can be used to estimate the object location 1125 in the target image 101, and

3) use the estimated location of each matching pair 1123 to place a “vote” at the estimated target location, e.g., according to pixel coordinates corresponding to pixels of the image 101 to identify object locations 1125. In some embodiments, the collection of all votes gives the final match map 421.

[0180] In some embodiments, whereas single matching pairs can be found in almost any image, only several pairs agreeing on a fixed location, scale and angle will indicate a consistent match with a given object. As a result, the location having the greatest concentration of votes from the matching line feature pairs 1123 indicates a probable location for an object in the target image 101 that matches an object-of-interest represented by a model. In some embodiments, similar voting techniques may be employed to determined object angles 1126 and object scales 1127.

Finding Matching Line Feature Pairs

[0181] In order to identify an object in the test image, we consider pairs of line features 411 in the target image 101 and compare those to pairs of model line features 402 in the model. In some embodiments, the line-based feature detection algorithm of the present invention compares matching features 11123 in the target 101 and model where the comparison has the following properties:

1) Translation invariant (the location of the object does not matter).

2) Rotation invariant (the orientation of the object does not matter).

3) Scale invariant (the scale or size of the object does not matter).

[0182] For example, the angle between two adjacent edges of a square forms a right angle that

7G is always 90° (- radians), regardless of the location, size, and orientation of the square.

[0183] In some embodiments, in order to characterize and map pairs of line features 411, a feature pair map may be computed at block 1601. In some embodiments, feature pair map values may include mappings for relative measurements between each line feature in any given line feature pair so as to satisfy the above properties. For example, the relative measurements between line features of each line feature pair can include, e.g., angle difference, length difference, midpoint distance, and centerline angle, among other measures indicative of relative characteristics of pairs of line features 411.

[0184] Definition 3 (Feature pair map) Let V denote a line feature space as defined in Definition 2. The feature pair map f is a function f: V x V ® l that is translational, rotational, and scale invariant with respect to the pair of features 411.

[0185] Angle difference. The angle difference measures the angular difference of two line features and £_{2 ·} This is done by Equation 1 below:

(Equation 1)

[0186] where 0(£f)and Q(£₂) ^are the angles (between 0 and p ) of the two line features. See FIG. 13, for example, where the angle difference measures the cosine of twice the angle difference between two line segments. In this example, the angle difference value would be

[0187] Length difference. The length difference measures the relative difference in length / of two line features 411 of each pair of line features 411. This is done by Equation 2 below (see FIG. 13):

(Equation 2)

[0188] In the above noted example of FIG. 13 where the angle difference value

= = -l, the length difference value would be

Note that these

values would not change if the rectangle is translated, rotated, or re-sized.

[0189] Midpoint distance. The midpoint distance measures the relative distance between the mid points of the two line features 411 of a pair of line features 411 (denoted as m₁ and m₂ respectively). This is done by Equation 3 below:

(Equation 3)

[0190] See, for example, FIG. 14, where the midpoint distance measures the relative distance between the midpoints of the two line features 411. In this example, the midpoint distance value would be f_dm ( ^2) ⁼

^¾ 0.56. Note that these values would not change if the rectangle is translated, rotated, or re-sized.

[0191] Center line angle. This feature pair map measures the angle of a line connecting the midpoints of each line feature 411 in a pair of line features 411, relative to the angles of the two line features 411. The center line angle may be formalized using three normalized vectors as defined by Equations 4, 5 and 6 below:

[0192] where q_\ and q₂ are the angles of the two line segments, and m₁ and m₂ are the midpoints of the two line features 411 represented as vectors. See, for example, FIG. 15, where the projections of a unit vector of the line connecting the midpoints onto the directions of the two line features 411 provide a useful measure to classify pairs of line features 411.

[0193] Based on these vectors, the following two line feature pair maps 1602 may be generated according to Equations 7 and 8 below:

(Equation 7) (Equation 8)

[0194] For the example in FIG. 15, using the above feature pair maps from Equations 7 and 8, 1602, it is found that according to Equation 9 below:

(Equation 9)

[0195] Thus, in this particular example, / = / . However, in some embodiments, this

^min ^max may not be the case in general, but rather is dependent on the relationships between the vectors and midpoints. FIG. 16 illustrates the algorithm that uses feature pair maps 1602 to find pairs of target line features 411 that match pairs of model line features 402. The feature pair match affinity value.

[0196] In some embodiments, the feature pair match affinity value (also referred to as the match affinity value from now on) is employed to measure how closely two pairs of line features match. In some embodiments, the feature pair match affinity value may be determined using an affinity function associated with a feature pair map defined in Definition 3:

[0197] Definition 4 (Feature pair match affinity function) Let T denote the space of all feature pair map functions and let p\ 7 X T ® denote a metric function on T.

[0198] Consider a model M, a target T, and a feature pair map / as defined in Definition 3. An associated feature pair match affinity function may be defined as a function according to Equation 10 below:

(Equation 10) where F_t is a monotonically non-decreasing function producing values from 0 to 1. [0199] In other words, the match affinity function takes the feature pair map value 1602 of a pair of line features 411 and a pair of model line features 402, and assigns a value between 0 and 1 that indicates how well the feature pair map values 1602 agree with the model line features 402. In some embodiments, the model line features 402 may include feature pair map values for each pair of the model line features 402 of a given model. Thus, in some embodiments, the match affinity values 1604 may be computed at block 1603 between the known line feature pair map values of a pair of model line features 402 and line feature pair map values 1602 determined at block 1601 for a given pair of line features 411 from the image

101. In some embodiments, the output value reflects the “affinity” of the match, such that 0 indicates no match, and 1 indicates a strong match. Exactly how this function is defined depends on the associated feature pair map but a generic function can be constructed as follows:

Example 1 [Generic Feature Pair Match Affinity Function)

[0200] Consider the length difference feature pair map f_dL (see, FIG. 13). In this case, the example model for FIG. 5 A is a rectangle of aspect ratio 2 to 1, which result in the value for the lines that marked in FIG. 13

⁼ ^}· However, in some embodiments, it is unlikely that exactly these values are determined for a matched pair of line features 411 in a target image 101 due to noise, artifacts, and various imperfection. Thus, a length map threshold may be employed to identify approximate matches that indicate a match. For example, the length map threshold may be about, e.g., 0.1. Thus a length difference map value that falls between 0.4 and 0.6 may be identified as an acceptable match to the model of FIG. 13. Therefore, decision making may be performed according to, e.g., the following length difference feature pair match affinity function of Equation 11 : 0.1 0.1

(Equation 11)

[0201] In some embodiments, similar feature pair match affinity functions may be employed for each of the feature pair maps described above according to the same methodology. For example, for angle difference feature map values, the following angle difference feature pair match affinity function may be employed: 0.1 0.1

(Equation 12)

[0202] In some embodiments, for midpoint distance feature map values, the following midpoint distance feature pair match affinity function may be employed: 0.1 0.1

(Equation 13)

[0203] In some embodiments, for centerline angle feature map values, the following centerline angle feature pair match affinity functions according to Equations 14 and 15 may be employed:

(Equation 14)

(Equation 15)

[0204] Accordingly, match affinity values 1604 may be computed for each line feature pair map 1602 and each model line feature pair map at block 1603 to quantify the affinity between each line feature 411 pair and each model line feature 402 pair.

Assigning a model ID to each matching target feature

[0205] In some embodiments, where a match between a target line feature 411 pair and a model line feature 402 pair has been found, this match can be translated into a location in the target image 101 that reflects where the pair indicates the center of the object-of-interest is located.

[0206] For the model, the object center and the location of the model line features 402 are known (see FIG. 5A). Thus, by comparing the relative size and location of the matching line features 411 in the target image 101 to their matching counterparts in the model, the object center vectors of the model may be scaled and rotated (see Definition 1 and FIG. 6, and, for example, FIG. 17 and FIG. 23) to identify the corresponding object center vectors for the target line features 411.

[0207] Similar to the use of pairs of line features for finding matches between line features 411 in the target image 101 and model line features 402 of the model, assigning the match identifier 1606 to a line feature 411 is also done on a pairwise basis. In particular, when a matching pair has been identified, each line feature 411 in the target match pair is assigned either one or both of the labels of the matching pair of model line features 402.

[0208] In some embodiments, line-based feature detection may account for three cases when matching identifiers are assigned to a matching target line feature pair.

1) Case I: Distinguishable line features. - The two line features in the matching pairs have significantly different lengths. In such cases, a unique match identifier 1606 can be assigned to each of the two line features 411 in the matching target pair of line features 411. See, for example, FIG. 17, where the two line features 411 in the pair have different relative lengths, and can be unambiguously matched to model line features 402 in the model. 2) Case II: Indistinguishable line features and object center vectors. - If both line features in the pair have (approximately) the same length, and same angles of their respective object center vectors with respect to the line features 411, it is impossible to know from just considering one pair of matching line features 411, which line feature 411 in the target corresponds to which model line feature 402 in the model. In some embodiments, it can suffice in this case to assign one of the model labels as match identifier 1606 to each target line feature 411. See, for example, FIG. 18, where the two line features 411 in the pair have same relative lengths, and cannot be unambiguously matched to model line features 402 in the model. In this case, which model label that is assigned the match identifier 1606 for the target line feature 411 is arbitrary.

3) Case III: Indistinguishable line features but different object center vectors. If the line features have (approximately) same length, but the angles of the object center vectors differ for the two line features, both model labels may be assigned as match identifiers 1606 to each target line feature 411 in the matching pair. See, for example, FIG. 19, where the two line features 411 in the pair have same relative lengths, but the associated object center vectors have different angles with respect to the line feature 411. In this case, both model labels are assigned as match identifiers 1606 to each target line feature 411. (For clarity, only one of the object center vectors has been depicted for each line feature.)

[0209] Therefore, to account for the three cases, in some embodiments, in order to assign a model’s label as a match identifier 1606 to a matching target line feature 411 pair, two checks may be performed:

1) Check if the relative lengths of the two line features 411 of a line feature pair are different (within some tolerance), in which case Case I from above may be established. In this case, the label of the longer line feature 402 in the model, is assigned to the longer line feature 411 in the target, and vice versa.

2) If the lengths are similar, the angles of the object center vectors with respect to their associated line features 411 are compared. If these angles are the same (within a given tolerance), then Case II may be established, and a model label from the matching model pair of line features 402 may be arbitrarily assigned to each line feature 411 in the target pair. If the angles are different, then Case III may be established, and both model labels are assigned to each target line feature 411.

[0210] Thus, in some embodiments, the main steps of finding pairs of line features 411 in the target image that match pairs of model line features 402 in the model image can be summarized as depicted in the flowchart of FIG. 16. In particular, for each pair of line features 411, the matching pairs of model line features 402 may be identified by computing the feature pair map values 1602 at block 1601. For each pair of model line features 402, the match affinity value 1604 to each pair of line features 411 is computed at block 1603. As described above, the match affinity value is a value between 0 and 1 that estimates how close a pair of target line features match a pair of model line features, where a large value indicates a good match, and a low value indicates a poor match. In some embodiments, the match affinity values 1604 may be filtered such that if the match affinity value 1604 is greater than an affinity threshold, e.g., 0.5, a model identifier may be assigned to the associated line features 411 as a match identifier at block 1606, however other threshold affinity values may be used based on a balance between sensitivity and false positive rate of identifying matches. In some embodiments, each of the two line features 411 in the target pair is assigned a label of the corresponding model line feature in the matching pair that match (the match identifier in Definition 1). The two matching line features 411 may then be appended to a list of matching line features.

Estimating the Object Location from a Single Matching Line Feature Pair

[0211] FIG. 20 illustrates a flowchart for estimating the object center location in accordance with one or more embodiments of the present disclosure of line-based feature detection. FIG. 21-32 depict examples of steps in object center location estimation.

[0212] In some embodiments, a matching line feature pair is used to estimate the location, size, and angle of the model object in the target image. While the present description illustrates the algorithm as applied to a single given target-model matching pair of line features, the same process may be applied to every matching pair, either serially or in parallel. In some embodiments, such a description includes how all such pairs vote for location, scale, and angle to determine if there is sufficient consistency among all matching line feature pairs to establish a match with the model object(s)-of-interest. Estimating the Scale Factor and Angle of the Target Object

[0213] In some embodiments, scale factor s and angle f 2032 between an obj ect-of-interest associated with a line feature 411 and the model of the matching model line feature 402 may be estimated at block 2031. In some embodiments, to estimate the scale factor s between the target object and the model object, along with the angle f of the object in the target image 101, the relations between the matching line features in the target and the model may be leveraged. (When estimating the angle, the angle of the model object may be assumed to be 0 degrees, however other angles of the model object may be employed, such as 90 degrees, 45 degrees or other).

[0214] The algorithm is illustrated in the example in FIG. 21 of how the relative scale factors Si_j and angles

are estimated for a given matching line pair. For example, in some embodiments, estimations are made of two scale factors s^, i,j = 1,2 that estimate the relative scale between the matching line features, and of two angles q^, i,j = 1,2 that estimate the relative angle between the matching line features. In practice, it is unlikely that the measurements of the locations, angles, and lengths of line features will be precise. Thus, perfect matches are unlikely between line features in the target and model.

[0215] To estimate the scale factor and angle for the pair of matching line features, the average of each of

may be determined. Note that for simplicity of the illustration, an example for Case I above is employed, but the same principle can be applied for Case II and III as well.

Estimating all object center vectors for a target line feature pair

[0216] Given the relative scaling factor s and angle Q described above, object center vectors can be assigned to each line feature 411 in the target line feature pair at block 2033. To do this, the match identifier for each line feature 411 may be employed to determine the matching model line feature 402. All object center vectors for that line feature 402 (there are either two or four of them) may be copied to the matching target line feature 411. In some embodiments, the object center vectors may then be modified by applying the scaling factor s, and rotating by angle Q to compute the center vector and center vector angle of each line feature 411.

[0217] See, for example, FIG. 22 illustrating an example of how to estimate the object center vectors for the target line features. Note that for clarity of the illustration, only two of the line feature matches from FIG. 19 are being considered. Note that one of the line features has two object center vectors, while the other one has four such vectors. Selecting pairs of overlapping center vectors

[0218] When finding matching pairs as described in the previous sections, only the relations between the line segments themselves were considered, without using the associated object center vectors to determine if the pair in the target matches a pair in the model consistently. In determining model matches to line feature 411 pairs, e.g., as described above with respect to FIG. 16, the collection of all associated object center vectors are considered, regardless of whether they are expected to point towards the same location, which would indicate a match.

[0219] However, the object center vectors contain important information. In some embodiments, this information is leveraged to rule out object center vector pairs that do not consistently agree at a location and therefore are unlikely to indicate a match.

[0220] In FIG. 23 all object center vectors

for a target line feature pair are shown. In some embodiments, which center vectors point to roughly the same location are determined by selecting overlapping pairs of center vectors 2034. In some embodiments, since there will be inaccuracies in the line feature extraction and in measurements, center vectors from the two line features are unlikely to point to exactly the same location. Therefore, center vectors only need to agree on a location within a certain tolerance (indicated by dotted circles in the FIG.) such as, e.g., about 10 pixels. In the example in FIG. 23, only center vectors

agree. Indeed, in the example depicted in FIG. 23, even though there are 2 X 4 = 8 pairs of center vectors associated with the line features, only one such pair,

and cp* agree (approximately) on a location. The dotted circle indicates the uncertainty allowed for when determining if two center vectors overlap at block 2034. As a result, in some embodiments, only center vectors and c need to be considered and the remaining center vectors may be discarded for the remainder of the algorithm.

[0221] However, in some embodiments, there may be situations when there may be more than one overlapping pair of center vectors. In FIG. 24 a slightly different example is shown where the object center of the model has been chosen differently. As described above, the object center can be selected arbitrarily for the object, as long as all model line features share the same object center.

[0222] In the FIG. 24 example, there are two pairs, (c ,

that overlap.

In such cases, either both of these vector pairs may be kept for the next step of the algorithm, or a mean vector averaging the two pairs of center vectors may be calculated. Estimating the object center location in the target

[0223] Each matching line feature pair can generate one or more matching object center vector pair(s). In the final step to generate an inferred object location 2036 for the line feature 411 pair, the mean of the center vector end points of the two center vectors that make up a matching center vector pair may be computed (see, for example, FIG. 24, where the final estimated location for a matching line feature pair with the associated center vectors is computed as the mean of the center vector end points (marked by a cross)).

[0224] In FIG. 26 an example of one matching target line feature pair 411 is shown when applied to the image in FIG. 5B. In this image, the red lines show a target line feature 411 pair that match model features 2 and 3 in FIG. 5A. The green and blue lines indicate the center object vectors associated with each line feature 411. The circles indicate the “radii of uncertainty” associated with each center vector. If these circles overlap, the line feature 411 pair will generate an estimated target location at the mean of the associated circle centers, which is denoted by the cross.

Finding the Object from the Match Map

[0225] FIG. 27-32 illustrate example images depicting steps in a method for determining object center locations for line-based feature detection in accordance with one or more embodiments of the present invention.

[0226] In some embodiments, all matching line feature pairs 411 may be used to estimate the location, scale and orientation of one or more objects in the target image 101, as described below. In some embodiments, using all matching pairs to vote on the object’s location, size, and orientation may generate a match map depicting a degree of certainty that a given location is the center of an object-of-interest.

Estimating the Location of the Object

[0227] In some embodiments, line features 411 in the target image 101, e.g., of FIG. 5B, are generated as described above with respect to FIG. 4. In FIG. 10 the line features 411 are depicted. In some embodiments, the lines are not “perfect” in the sense that they may not exactly follow the edges of the object or extend all the way out to the corners of the associated object. However, as described above, some uncertainty is taken into account by the feature detection algorithm when finding matching pairs (as described above with respect to the feature pair match affinity value). Thus, even where generated line features 411 are not perfectly matched to the associated object, many line feature pairs 411 matching to model line feature pairs 402 may be identified to generate a robust match map. In FIG. 27 all matching pairs for the example of FIG. 5 A and 5B are illustrated, where all matching line features (in red) and their associated matching object center vectors (in green) are shown. The dotted blue lines indicate line feature pairs that were found to match line feature pairs in the model. The green filled squares represent estimated object centers from the matching object center vectors.

[0228] In FIG. 28 a close up of the matching pairs and their associated center vectors is shown for the tilted rectangle depicted in FIG. 27. Note that the estimated center locations (green squares) do not match exactly. Accordingly, in some embodiment, the matching line feature pairs may each vote to cover a neighborhood of pixels about the estimated center locations. The overlapping votes then gives a type of mean object location that approximates a true or most probably center location for an obj ect-of-interest.

[0229] To generate the final match map, a zero-filled matrix of the same size as the target image 101 is generated. Each index of the zero-filled matrix corresponds to a pixel coordinate position in the image 101. The pixel coordinate corresponding to the object center location of each matching pair may then be mapped to the matrix by adding 1 to the index in the matrix that represents the pixel coordinate of each object center location in the image 101. However, in order to allow for some error, a one may be added in a small neighborhood of index locations of the matrix representing a corresponding small neighborhood of pixels around the estimated pixel location to represent the area in which an object center location is located. In some embodiments, this small neighborhood can include, e.g., each adjacent pixel to the voted pixel, or to each pixel within a radius of, e.g., 2 pixels, 3 pixels, 5 pixels, 10 pixels, 15 pixels, or other radius around a voted pixel.

[0230] FIG. 29 depicts an example of the resulting match map. Here all estimated object center locations have been summed, using a square neighborhood of radius 10 pixels around each location estimate. Note that there is one “false” match in FIG. 29, which may occur in most scenarios. However, such stray matches do not give a large response compared to all the overlapping matches that agree on a location. Therefore, the “stray” match is barely visible in the final match map. The centers of the blurred square regions provide accurate estimates of the center locations of the objects-of-interest. Estimating the Scale of the Object

[0231] In addition to holding information about the estimated object center location, each matching target line feature pair holds the information about the scale estimation of the object. (Cf. Sy in FIG. 21). When a pair is voting on the object location, the pair may vote on the scale as well. In some embodiments, voting on scale may be implemented by any one of many suitable methods. Two such methods are described below.

[0232] Method 1: Mean Scale. In some embodiments, a scale map may be created as a zero- filled matrix with the same size as the original target image 101 where each index of the matrix corresponds to a pixel in the image 101. Each matching pair adds its estimated scale at the pixel representing the estimated center location, and in a neighborhood of pixels around this location. In some embodiments, a scale normalization matrix may also be initialized as a zero-filled matrix with the same size as the original target image 101. A one may then be added at each pixel location in the scale normalization matrix where a scale vote has been cast. Once all matching pairs have been processed, all non-zero pixels in the scale map are divided with the corresponding element of the scale normalization matrix. The non-zero elements in the final, normalized scale map gives a scale factor that represents the estimated scale at the corresponding locations.

[0233] FIG. 30 illustrates the scale map corresponding to the matching pairs in FIG. 27, where the brightness indicates the magnitude of the scale factor.

[0234] A non-zero value of a pixel in the scale map indicates the average scale voted for at that pixel location. In this case, the value for the tilted rectangle is approximately 0.22, indicating that its size is 22% of the size of the model object. The value for the non-tilted rectangle is approximately 0.33, indicating that its size is 33% of the size of the model object.

[0235] Method 2: Median Scale. Instead of adding scales to the scale map, all scale votes cast at each pixel may be recorded. Once an object location has been determined, all scale votes cast at that pixel can be extracted and the median of the votes determined. This method is more resilient to outliers, but also requires a more complicated data structure to be stored efficiently.

Using the Scale Consistency to Estimate Location

[0236] The methods to estimate scale as described above can be used to find the estimated scale at any locations where at least one vote was cast for a match. However, the scales for the various matches that contribute to a location may also indicate which locations appear to come from matches that are consistent in terms of scale, thus indicating a greater confidence of a match to an object-of-interest. In more complex examples, a location may receive many votes that agree on location, while the votes do not agree on the scale. Such a situation would indicate that votes that agree on location but not on scale should not be trusted as much as votes that agree on both location and scale.

[0237] In some embodiments, to measure scale consistency, the minimum and maximum scales may first be specified, denoted as s_min and s_max , respectively. The interval [s_min, s_max] may then be discretized into n_scaie equally spaced bins. For a target image of size M x N pixels an M x N x n_scale volume data structure may be initialized with zeros, which may be denoted as 5.

[0238] At each pixel i,j a scale vote s is cast as described above, and a scale bin i_scaie is determined according to Equation 16 below:

(Equation 16)

[0239] In some embodiments, a 1 may then be added to Sij,i_scale · The scale consistency map is the defined as the M x N matrix C, where each element is given by Equation 17 below:

^Ci = max S _{j k} (Equation 17) kE[0,l ... 7l_Scale — ]

[0240] In FIG. 31 the scale consistency map for the example in FIG. 5B is illustrated.

[0241] In some embodiments, the location match map can be modified by multiplying it element-wise with the scale consistency map to improve the location accuracy.

Estimating the Angle of the Object

[0242] In some embodiments, the angle of the object (given as an angle in the range 0 to 180 is estimated similarly to how the scale is estimated as described above with respect to FIG. 20 21 However, as the angles cannot be simply added and averaged, an orientation addition method may be employed. In some embodiments, orientation addition is used to consider each angle as described above with respect to FIG. 21, to be part of an orientation, where the weights of all orientations are set to a value of one.

[0243] In some embodiments, adding all matching angles at each pixel using orientation addition, effectively generates a map that provides the estimated angle for each pixel location by looking at the added angle component of the orientation field. The weight component of the orientation field, therefore, may generate a type of angle consistency map, similarly to the scale consistency map described above.

[0244] FIG. 32 illustrates the weights of the all added orientations for the example in FIG. 5B to form the angle consistency map for the example in FIG. 5B.

Determining Valid Matches

[0245] In some embodiments, a match map, e.g., described above with respect to FIGs. 20-32 via the process described above using angle and scale maps, may be generated to provide an accurate map of the relative likelihood of a match, in the sense that the strongest response(s) in the map indicates where a match is most likely. However, in some embodiments, a magnitude of a match may be determined and utilized to establish a match. For example, for any a random image, there are likely to be at least a few false matches. The magnitudes of matches in the match map, however, may indicate a weak match, and thus indicate that the largest response may in no way correspond to the object-of-interest.

[0246] Therefore, in some embodiments, the match map may be thresholded to remove weak matches indicative of false matches and only record a match when the match value exceeds a certain threshold 6. In some embodiments, the threshold 6 may be selected based on the maximum match score e_max expected based on the number of model line features n_(> 402 in a given model

[0247] In some embodiments, selecting the threshold 6 may include analyzing the model and the number of the three cases described above (see, FIG. 17-19) that one can expect for different combinations of line feature 411 pairs. In some embodiments, if all line features in a given model can be unambiguously distinguished (e.g., Case I as described above), a minimum bound for the maximum match score is given by Equation 18 below:

Vnax ³ (?) (Equation 18)

[0248] On the other hand, various symmetries may result in the maximum number of combinations being as large as, e.g. Equation 19 below: e_max < 4 n (n - 1) (Equation 19)

[0249] Therefore, Equation 20, below, follows:

(Equation 20) [0250] However, the following other considerations may be taken into account:

1) the number of accurate line features in the target image is highly dependent on the accuracy of the line feature generation algorithm used and the image quality of the target image;

2) if so-called line consolidation (as described above for generating line features) is not used (which is sometimes preferable due to noise or shadow artifacts in an image), e_max can be larger than 471^(71^ — 1);

3) if blurring is used for the final match map, e_max may be lower than the estimate (Equation 20) suggests.

4) the number of matching pairs is greatly dependent on the thresholds used for establishing the match criteria for match affinity values as described above;

5) if the scale consistency map, as described above, is used to modify the match map by element-wise multiplication, the match value in the map is affected as well. This effect can be partially remedied by first normalizing the scale consistency map between 0 and 1, where the max value of the scale consistency map is mapped to 1.

[0251] For example, the maximum match score e_max may be estimated for the example of FIG. 5A and 5B. In this case, n_{ = 4 (see FIG. 5) because there are 4 sides to the model object, which according to (Equation 20) above results in Equation 21 below:

6 < e_max < 24 (Equation 21)

[0252] The magnitude for the matches in the slightly blurred match map in FIG. 29 is 17 for the tilted rectangle and 18 for the larger rectangle. These values are consistent with the estimated maximum match score e_max of between 6 and 24.

[0253] In some embodiments, the match threshold 6 may be selected empirically, e.g., for each type of image quality. However, as a starting point, it is helpful to consider the estimate of the maximum match score e_max , as an estimate of the order of magnitude to use for the threshold 6. Moreover, in some embodiments, selecting the threshold 6 may also include taking into account the sensitivity of the algorithm. For example, if a false positive is less bad than a false negative for the application at hand, then a lower threshold may be selected, and vice versa. Example 2: Line-Based Feature Detection for Diagnostic Flu Test Image Autocapture

[0254] In some embodiments, a feature extraction engine 116 employing a line-based feature detection algorithm may be used to autocapture an image. For example, an image capture device 102 may receive a series or stream of images 101 of a setting and an image analysis system 110 may process the stream of images 101 to determine when an object-of-interest is present in one of the images 101 based on the detection of line-based features.

[0255] Therefore, in some embodiments, one or more of the images 101 may include the image depicted in FIG. 33 of a diagnostic flu test. In some embodiments, the stream of images 101 may be received by the image-to-signal engine 114 to produce an image signal as described above. In some embodiments, the diagnostic flu test can include a cassette 1-1 housing a test strip 1-2. In some embodiments, the cassette 1-1 may be placed on a card 1-3. In some embodiments, the card 1-3 may have a solid color different from a color of the cassette 1-1 to facilitate analysis of contrast and the detection of edges associated with the cassette 1-1. In addition, the card 1-3 may reduce noise and excess detected edges by blocking backgrounds that may have high edge or line concentrations, such as woodgrain, patterned surfaces, or other backgrounds.

[0256] In some embodiments, in the autocapture application of line-based feature detection, speed and efficiency of feature detection can facilitate real-time autocapture. Thus, to improve the speed of the image analysis system 110, the image-to-signal engine 114 may downsample each image 101 as shown in FIG. 34. Downsampling reduces the resolution of the image such that fewer pixels need to be analyzed, thus reducing needed computational resources and increasing speed. Downsampling may also include a contrast filter to enhance contrast in the image, e.g., by boosting highlights and reducing lowlights, or other contrast enhancement filters. The contrast enhancement may amplify differences in contrast and color between two objects, thus enhancing an edge between the objects, such as between the cassette 1-1 and the card 1-3, and between the test strip 1-2 and the cassette 1.

[0257] In some embodiments, using the downsampled image, the image-to-signal engine 114 and/or the feature detection engine 116 may detect edges 1-4 in the image (See FIG. 35). For example, orientation fields, Sobel edge detectors, or Radon transforms, among other edge detection algorithms and combinations thereof may produce an edge image as depicted in FIG. 35. In some embodiments, the edge detection may be used to identify edge pixels located at, e.g., a peak edge or a step edge as described above. [0258] In some embodiments, the edges 1-4 may be extracted from the edge image of FIG. 35 to map the edges 1-4 to locations in the image as depicted in FIG. 36. Thus, mapped edges 1- 5 may be produced at the edges of objects in the image including the cassette 1-2, the test strip 1-1, and other shapes and objects where edges are detected. In some embodiments, the mapped edges 1-5 can be produced as line segments on the image that trace the edges of objects in the image according to the peak and/or step edges, or other edge types. These line segments may be employed as line features for detecting objects-of-interest, such as the test strip 1-2 and the cassette 1-1.

[0259] In some embodiments, the autocapture process may be configured to identify whether the cassette 1-1 is in the frame of the image. As such, the image analysis system 110 may maintain a library of model objects including a model of the cassette 1-1. In some embodiments, the model may include characteristics of each edge of the model including measurements such as length, orientation relative to an x-axis or a y-axis, object center vector pointing from a midpoint of an edge to a center of the cassette model, among others to form model line features for the cassette model. Based on the measurements, the model of the cassette may also include feature map values of each possible pair of model line features as described above, including, e.g., the length difference, angle difference, midpoint distance and center line angle for each possible pair of model line features of the cassette model.

[0260] In some embodiments, the line features 1-5 identified in the edge map may then be compared against the model line features to determine a degree of match, or affinity. Thus, the line features 1-5 are paired in each possible combination (e.g., line feature 1-1 paired with line feature 1-2, line feature 1-1 paired with line feature 0, line feature 1-2 paired with line feature 2, etc.). Feature maps may then be calculated for each pair of line features 1-5 for each of the angle difference /_d6)(f i, f ₂), length difference/_di(-f₁,-f₂), midpoint distance /_di (f i, f₂), maximum center line angle fa_max(^ 1^2) ^ar|d minimum center line angle fa_min(^ 1^2) ^as described above.

[0261] In some embodiments, the feature map values for each of the cassette model line feature pairs may be compared to each of the image line feature pairs to determine the affinity of each particular line feature pair to each cassette model line feature pair according to described above.

[0262] In some embodiments, for each image line feature pair, all cassette model line feature pairs below a threshold affinity value may be discarded for not matching. In some embodiments, the threshold affinity value may be about 0.5 for the autocapture of a cassette 1- 1, however other threshold affinity values may be used based on a balance between sensitivity and false positive rate of identifying matches. However, other scenarios may use different thresholds. Thus, only affinity values above the threshold would indicate a match between an image line feature pair and a particular cassette model line feature pair. In some embodiments, the cassette model line feature pairs that have an affinity to a particular image line feature pair that exceeds the threshold may be matched to the particular image line feature pair using the match identifier as described above. In particular, each line feature of the particular line feature pair may be appended with an attribute indicating the identifier for the cassette model line feature pair, thus identifying the match to the model.

[0263] Where no affinity values are greater than the threshold for a given image line feature pair, the given image line feature pair is discarded as not matching the model and thus not corresponding to a cassette in the image. Thus, where an image has no line feature pairs that match any cassette model line feature pairs according to affinity values, the image may be deemed to not include the object-of-interest including the cassette 1-1. Thus, where no line feature pairs match the cassette model, the image may be immediately discarded and the process stopped to prevent unnecessary use of computational resources.

[0264] In some embodiments, upon identifying all image line features 1-5 that match the cassette model, the cassette 1-1 location may be determined. For example, the image line features 1-5 that match the model may be used to vote on cassette 1-1 center location according to overlaps in object center vectors for each line feature 1-5 as described above. For example, the object center vectors for each line feature 1-5 may be modified by scaling the model object center vectors of associated model line features and adjusting orientation of the model object center vectors based on the length and orientation of matched line features 1-5 relative to the matching model line features.

[0265] Additionally, as described above, the center location may be refined using scale voting, scale consistency voting, and angle voting as described above to reinforce votes at locations where each of scale consistency and angle consistency agree. For example, FIG. 37 illustrates a scale map produced using the scale voting process described above using, e.g., mean scales or median scales, or a combination thereof. Similarly, FIG. 38 illustrates a scale consistency map produced using the scale consistency process described above and FIG. 39 depicts an angle consistency map produced using the angle process described above using, e.g., orientation addition. [0266] In some embodiments, an object center location map produced according to the object center voting may be modified using the scale map, scale consistency map and angle map by, e.g., multiplying element-wise the values of map together. As a result, a verified location map may be produced as illustrated in FIG. 40. In the location map, a concentration of location votes 1-9 indicates the location of the center of the cassette 1-1. However, in some embodiments, the magnitude of the location votes 1-9 may be compared to a threshold to ensure the signal is of sufficient to strength to indicate a true match to the cassette model. Here, the threshold may be between about 6 and about 24, however other examples may dictate different thresholds.

[0267] In some embodiments, where the match map has a location vote magnitude greater than the threshold, a cassette 1-1 and its associated center location 1-11 is identified as illustrated in FIG. 41. Moreover, by scaling the size of the cassette model based on the relative sizes and orientations of the image line feature, the corners of the cassette 1-1 may also be identified, thus signaling to a user or a program the approximate outline of the cassette 1-1. As a result, the cassette 1-1 is detected and the image depicting the cassette 1-1 may be selected for later analysis. Alternatively, a new image may be automatically captured on the assumption that the same contents or setting will be captured. For example, the image depicting the cassette 101 may be uploaded to a cloud platform or server.

Example 3: Line-Based Feature Detection for Diagnostic Flu Test Image Analysis

[0268] In some embodiments, a feature extraction engine 116 employing a line-based feature detection algorithm may be used for analysis of an image. For example, an image capture device 102 may receive an image 101 of a setting and an image analysis system 110 may process the image 101 to locate an object-of-interest based on the detection of line-based features. For example, an image 101 in which a diagnostic flu cassette has been depicted and uploaded to, e.g., a cloud platform upon the autocapture process of Example 2 above may undergo feature detection and image analysis to locate the test strip and determine the results of the diagnostic flu test.

[0269] Therefore, in some embodiments, the image 101 may include the image depicted in FIG. 42 of a diagnostic flu test. In some embodiments, the image 101 may be received by the image-to-signal engine 114 to produce an image signal as described above. In some embodiments, the diagnostic flu test can include a cassette 2-1 housing a test strip 2-2. In some embodiments, the cassette 2-1 may be placed on a card 2-3. In some embodiments, the card 2- 3 may have a solid color different from a color of the cassette 2-1 to facilitate analysis of contrast and the detection of edges associated with the cassette 2-1. In addition, the card 2-3 may reduce noise and excess detected edges by blocking backgrounds that may have high edge or line concentrations, such as woodgrain, patterned surfaces, or other backgrounds.

[0270] In some embodiments, using the image 101, the image-to-signal engine 114 and/or the feature detection engine 116 may detect edges in the image 101 to form an edge map as illustrated in FIG. 43 using, e.g., orientation fields, Sobel edge detectors, or Radon transforms, among other edge detection algorithms and combinations thereof. In some embodiments, the edge detection may be used to identify edge pixels located at, e.g., a peak edge or a step edge as described above. In some embodiments, the edges may then be used to identify line segments forming line features 2-5. In particular, the line features 2-5 may correspond to edges of objects represented within the image 101, such as, in this case, the cassette 2-1 and the test strip 2-2, among other objects such as the card 2-3 as well as writing and markings. In some embodiments, the line features 2-5 can be produced as line segments on the image that trace the edges of objects in the image according to the peak and/or step edges, or other edge types.

[0271] In some embodiments, the test analysis process may be configured to identify the locations of the cassette 2-1 and/or the test strip 2-2 in the frame of the image. As such, the image analysis system 110 may maintain a library of model objects including a model of the cassette 2-1 and a model of the test strip 2-2. In some embodiments, the model may include characteristics of each edge of the model including measurements such as length, orientation relative to an x-axis or a y-axis, object center vector pointing from a midpoint of an edge to a center of the cassette model among others to form model line features for the model. Based on the measurements, the model of the cassette and the model of the test strip may also include feature map values of each possible pair of model line features as described above, including, e.g., the length difference, angle difference, midpoint distance and center line angle. However, the model of the cassette may include the location relative to the cassette of the test strip. Thus, in some embodiments, only the cassette model is used. The example described here employs this technique where only the cassette model is employed for the first stage of line-based feature detection.

[0272] In some embodiments, the line features 2-5 identified in the edge map may then be compared against the model line features to determine a degree of match, or affinity. Thus, the line features 2-5 are paired in each possible combination (e.g., line feature 1-1 paired with line feature 1-2, line feature 1-1 paired with line feature 0, line feature 1-2 paired with line feature 2, etc.). Feature maps may then be calculated for each pair of line features 2-5 for each of the angle difference 1-^2), length difference/_di(-f₁,-f₂), midpoint distance f_dm(^i_> ^2). maximum center line angle fa_max( i_> 2) ^ar|d minimum center line angle fa_min(^i_>^2) ^as described above.

[0273] In some embodiments, the feature map values for each of the cassette model line feature pairs may be compared to each of the image line feature pairs to determine the affinity of each particular line feature pair to each cassette model line feature pair according to described above.

[0274] In some embodiments, for each image line feature pair, all cassette model line feature pairs below a threshold affinity value may be discarded for not matching. In some embodiments, the threshold affinity value may be about 0.5 for the identification and location of a cassette 2-1, however other threshold affinity values may be used based on a balance between sensitivity and false positive rate of identifying matches. However, other scenarios may use different thresholds. Thus, only affinity values above the threshold would indicate a match between an image line feature pair and a particular cassette model line feature pair. In some embodiments, the cassette model line feature pairs that have an affinity to a particular image line feature pair that exceeds the threshold may be matched to the particular image line feature pair using the match identifier as described above. In particular, each line feature of the particular line feature pair may be appended with an attribute indicating the identifier for the cassette model line feature, thus identifying the match to the model line feature.

[0275] In some embodiments, upon identifying all matching line features 2-6 that match the cassette model, as depicted in FIG. 44, the cassette 2-1 location may be determined, including the location therein of the test strip 2-2 based on the relative size and orientation of the model and the image 101. For example, the matching line features 2-6 that match the cassette model may be used to vote on the cassette 2-1 center location according to overlaps in object center vectors for each matching line feature 2-6 as described above and as illustrated in the match map depicted in FIG. 45. For example, the object center vectors for each matching line feature 2-6 may be modified by scaling the model object center vectors of associated model line features and adjusting the orientation of the model object center vectors based on the length and orientation of matched line features 2-6 relative to the matching model line features.

[0276] Additionally, as described above, the center location may be refined using scale voting, scale consistency voting, and angle voting as described above to reinforce votes at locations where each of scale consistency and angle consistency agree as described above to refine the estimated object locations.

[0277] In some embodiments, an object center location map produced according to the object center voting may be modified using the scale consistency map and angle consistency map by, e.g., multiplying element-wise the values of maps together. As a result, an improved location map may be produced as illustrated in FIG. 45. In the location map, a concentration of location votes 2-10 indicates the location of the center of the cassette 2-1. However, in some embodiments, the magnitude of the location votes 2-10 may be compared to a threshold to ensure the signal is of sufficient to strength to indicate a true match to the cassette model. Here, the threshold may be between about 120 and about 960, however other examples may dictate different thresholds.

[0278] In some embodiments, the size and location of the test strip model relative to the cassette model center location is known. Thus, the relative scale and orientation of the matching line features 2-6 relative to the respective edges of the cassette model may used to estimate the location of the test strip 2-2 within the image 101. For example, the corners of the test strip model can be represented as a distance and direction (e.g., by a vector representation) from the cassette model center location. Thus, the relative scale between the cassette 2-1 and the cassette model may be determined according to the line features 2-6 to determine the scale factor. The scale factor may be used to scale the distance to the corners of the test strip to match the scale of the image 101. Similarly, the relative angle between the cassette 2-1 and the cassette model may be determined according to the line features 2-6 to determine the orientation. The orientation may be used to rotate the direction to the corners of the test strip to match the orientation of the image 101. Thus, the comers of the test strip 2-6 may be estimated in the image 101, as depicted by red dots in FIG. 46.

[0279] In some embodiments, the test strip 2-2 may then be located within the image 101 as depicted by FIG. 46. In some embodiments, the test strip 2-2 center and comer locations from FIG. 45 may be superimposed on the original image, where the object corners correspond to the corners of the test strip and the test strip 2-2 center corresponds to the center of the test strip.

[0280] Using the located test strip 2-2, the image analysis system 110 may form a padded crop of test strip 2-2 region as input for a next step including fine-tuning the location of the test-strip 2-2 and features therein. FIG. 47 illustrates the cropped target image from the original image depicted in FIG. 42 based on the test strip 2-2 locations estimated using line-based feature detection as described above.

[0281] In some embodiments, the padded crop of the test strip 2-2 is passed as input to the fine-tuning step. In some embodiments, the fine-tuning step may use a second stage of line- based feature detection using a test strip model. However, in some embodiments, other related feature detection algorithms may be employed that are more suited to locating the test strip at the second stage.

[0282] In some embodiments, based on the second stage of feature detection for fine tuning the test-strip 2-2 location, a match map may be produced estimating a center location of the test strip 2-2 as depicted in FIG. 48. Based on the estimated center location of the test strip 2- 2 and the relative scale and orientation between the test-strip 2-2 and the test strip model, a boundary of the test strip 2-2 may be identified. For example, locations of sides along the x- axis and locations of sides along the y-axis may be marked as depicted in FIG. 49.

[0283] In some embodiments, the identification of the boundary of the test strip 2-2 may be used for a final cropping of the image to isolate the test strip 2-2 as illustrated in FIG. 50. In some embodiments, the test strip 2-2 may include lines and spaces, the presence, colors and/or locations of which may indicate a particular result of the flu test. Thus, in some embodiments, the isolated test strip 2-2 may be provided to, e.g., a line classifier implemented by the feature analysis engine 118 and other code to determine a diagnostic result based on the number and location of the lines detected in the test strip 2-2.

[0284] FIG. 51 depicts a block diagram of an exemplary computer-based system and platform 5100 in accordance with one or more embodiments of the present disclosure. However, not all of these components may be required to practice one or more embodiments, and variations in the arrangement and type of the components may be made without departing from the spirit or scope of various embodiments of the present disclosure. In some embodiments, the illustrative computing devices and the illustrative computing components of the exemplary computer- based system and platform 5100 may be configured to manage a large number of members and concurrent transactions, as detailed herein. In some embodiments, the exemplary computer- based system and platform 5100 may be based on a scalable computer and network architecture that incorporates varies strategies for assessing the data, caching, searching, and/or database connection pooling. An example of the scalable architecture is an architecture that is capable of operating multiple servers. [0285] In some embodiments, referring to FIG. 51, members 5102-5104 (e.g., clients) of the exemplary computer-based system and platform 5100 may include virtually any computing device capable of receiving and sending a message over a network (e.g., cloud network), such as network 5105, to and from another computing device, such as servers 5106 and 5107, each other, and the like. In some embodiments, the member devices 5102-5104 may be personal computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, and the like. In some embodiments, one or more member devices within member devices 5102-5104 may include computing devices that typically connect using a wireless communications medium such as cell phones, smart phones, pagers, walkie talkies, radio frequency (RF) devices, infrared (IR) devices, CBs, integrated devices combining one or more of the preceding devices, or virtually any mobile computing device, and the like. In some embodiments, one or more member devices within member devices 5102-5104 may be devices that are capable of connecting using a wired or wireless communication medium such as a PDA, POCKET PC, wearable computer, a laptop, tablet, desktop computer, a netbook, a video game device, a pager, a smart phone, an ultra-mobile personal computer (UMPC), and/or any other device that is equipped to communicate over a wired and/or wireless communication medium (e.g., NFC, RFID, NBIOT, 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, ZigBee, etc.). In some embodiments, one or more member devices within member devices 5102-5104 may include may run one or more applications, such as Internet browsers, mobile applications, voice calls, video games, videoconferencing, and email, among others. In some embodiments, one or more member devices within member devices 5102-5104 may be configured to receive and to send web pages, and the like. In some embodiments, an exemplary specifically programmed browser application of the present disclosure may be configured to receive and display graphics, text, multimedia, and the like, employing virtually any web based language, including, but not limited to Standard Generalized Markup Language (SMGL), such as HyperText Markup Language (HTML), a wireless application protocol (WAP), a Handheld Device Markup Language (HDML), such as Wireless Markup Language (WML), WMLScript, XML, JavaScript, and the like. In some embodiments, a member device within member devices 5102-5104 may be specifically programmed by either Java, .Net, QT, C, C++ and/or other suitable programming language. In some embodiments, one or more member devices within member devices 5102-5104 may be specifically programmed include or execute an application to perform a variety of possible tasks, such as, without limitation, messaging functionality, browsing, searching, playing, streaming or displaying various forms of content, including locally stored or uploaded messages, images and/or video, and/or games. [0286] In some embodiments, the exemplary network 5105 may provide network access, data transport and/or other services to any computing device coupled to it. In some embodiments, the exemplary network 5105 may include and implement at least one specialized network architecture that may be based at least in part on one or more standards set by, for example, without limitation, Global System for Mobile communication (GSM) Association, the Internet Engineering Task Force (IETF), and the Worldwide Interoperability for Microwave Access (WiMAX) forum. In some embodiments, the exemplary network 5105 may implement one or more of a GSM architecture, a General Packet Radio Service (GPRS) architecture, a Universal Mobile Telecommunications System (UMTS) architecture, and an evolution of UMTS referred to as Long Term Evolution (LTE). In some embodiments, the exemplary network 5105 may include and implement, as an alternative or in conjunction with one or more of the above, a WiMAX architecture defined by the WiMAX forum. In some embodiments and, optionally, in combination of any embodiment described above or below, the exemplary network 5105 may also include, for instance, at least one of a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an enterprise IP network, or any combination thereof. In some embodiments and, optionally, in combination of any embodiment described above or below, at least one computer network communication over the exemplary network 5105 may be transmitted based at least in part on one of more communication modes such as but not limited to: NFC, RFID, Narrow Band Internet of Things (NBIOT), ZigBee, 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite and any combination thereof. In some embodiments, the exemplary network 5105 may also include mass storage, such as network attached storage (NAS), a storage area network (SAN), a content delivery network (CDN) or other forms of computer or machine readable media.

[0287] In some embodiments, the exemplary server 5106 or the exemplary server 5107 may be a web server (or a series of servers) running a network operating system, examples of which may include but are not limited to Microsoft Windows Server, Novell NetWare, or Linux. In some embodiments, the exemplary server 5106 or the exemplary server 5107 may be used for and/or provide cloud and/or network computing. Although not shown in FIG. 51, in some embodiments, the exemplary server 5106 or the exemplary server 5107 may have connections to external systems like email, SMS messaging, text messaging, ad content providers, etc. Any of the features of the exemplary server 5106 may be also implemented in the exemplary server 5107 and vice versa. [0288] In some embodiments, one or more of the exemplary servers 5106 and 5107 may be specifically programmed to perform, in non-limiting example, as authentication servers, search servers, email servers, social networking services servers, SMS servers, IM servers, MMS servers, exchange servers, photo-sharing services servers, advertisement providing servers, fmancial/banking-related services servers, travel services servers, or any similarly suitable service-base servers for users of the member computing devices 5101-5104.

[0289] In some embodiments and, optionally, in combination of any embodiment described above or below, for example, one or more exemplary computing member devices 5102-5104, the exemplary server 5106, and/or the exemplary server 5107 may include a specifically programmed software module that may be configured to send, process, and receive information using a scripting language, a remote procedure call, an email, a tweet, Short Message Service (SMS), Multimedia Message Service (MMS), instant messaging (IM), internet relay chat (IRC), mIRC, Jabber, an application programming interface, Simple Object Access Protocol (SOAP) methods, Common Object Request Broker Architecture (CORBA), HTTP (Hypertext Transfer Protocol), REST (Representational State Transfer), or any combination thereof.

[0290] FIG. 52 depicts a block diagram of another exemplary computer-based system and platform 5200 in accordance with one or more embodiments of the present disclosure. However, not all of these components may be required to practice one or more embodiments, and variations in the arrangement and type of the components may be made without departing from the spirit or scope of various embodiments of the present disclosure. In some embodiments, the member computing devices 5202a, 5202b thru 5202n shown each at least includes a computer-readable medium, such as a random-access memory (RAM) 5208 coupled to a processor 5210 or FLASH memory. In some embodiments, the processor 5210 may execute computer-executable program instructions stored in memory 5208. In some embodiments, the processor 5210 may include a microprocessor, an ASIC, and/or a state machine. In some embodiments, the processor 5210 may include, or may be in communication with, media, for example computer-readable media, which stores instructions that, when executed by the processor 5210, may cause the processor 5210 to perform one or more steps described herein. In some embodiments, examples of computer-readable media may include, but are not limited to, an electronic, optical, magnetic, or other storage or transmission device capable of providing a processor, such as the processor 5210 of client 5202a, with computer- readable instructions. In some embodiments, other examples of suitable media may include, but are not limited to, a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, an ASIC, a configured processor, all optical media, all magnetic tape or other magnetic media, or any other medium from which a computer processor can read instructions. Also, various other forms of computer-readable media may transmit or carry instructions to a computer, including a router, private or public network, or other transmission device or channel, both wired and wireless. In some embodiments, the instructions may comprise code from any computer-programming language, including, for example, C, C++, Visual Basic, Java, Python, Perl, JavaScript, and etc.

[0291] In some embodiments, member computing devices 5202a through 5202n may also comprise a number of external or internal devices such as a mouse, a CD-ROM, DVD, a physical or virtual keyboard, a display, or other input or output devices. In some embodiments, examples of member computing devices 5202a through 5202n (e.g., clients) may be any type of processor-based platforms that are connected to a network 5206 such as, without limitation, personal computers, digital assistants, personal digital assistants, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices. In some embodiments, member computing devices 5202a through 5202n may be specifically programmed with one or more application programs in accordance with one or more principles/methodologies detailed herein. In some embodiments, member computing devices 5202a through 5202n may operate on any operating system capable of supporting a browser or browser-enabled application, such as Microsoft™, Windows™, and/or Linux. In some embodiments, member computing devices 5202a through 5202n shown may include, for example, personal computers executing a browser application program such as Microsoft Corporation's Internet Explorer™, Apple Computer, Inc.'s Safari™, Mozilla Firefox, and/or Opera. In some embodiments, through the member computing client devices 5202a through 5202n, users, 5212a through 5202n, may communicate over the exemplary network 5206 with each other and/or with other systems and/or devices coupled to the network 5206. As shown in FIG. 52, exemplary server devices 5204 and 5213 may be also coupled to the network 5206. In some embodiments, one or more member computing devices 5202a through 5202n may be mobile clients.

[0292] In some embodiments, at least one database of exemplary databases 5207 and 5215 may be any type of database, including a database managed by a database management system (DBMS). In some embodiments, an exemplary DBMS-managed database may be specifically programmed as an engine that controls organization, storage, management, and/or retrieval of data in the respective database. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to provide the ability to query, backup and replicate, enforce rules, provide security, compute, perform change and access logging, and/or automate optimization. In some embodiments, the exemplary DBMS-managed database may be chosen from Oracle database, IBM DB2, Adaptive Server Enterprise, FileMaker, Microsoft Access, Microsoft SQL Server, MySQL, PostgreSQL, and a NoSQL implementation. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to define each respective schema of each database in the exemplary DBMS, according to a particular database model of the present disclosure which may include a hierarchical model, network model, relational model, object model, or some other suitable organization that may result in one or more applicable data structures that may include fields, records, files, and/or objects. In some embodiments, the exemplary DBMS-managed database may be specifically programmed to include metadata about the data that is stored.

[0293] In some embodiments, the illustrative computer-based systems or platforms of the present disclosure may be specifically configured to operate in a cloud computing/architecture such as, but not limiting to: infrastructure a service (IaaS), platform as a service (PaaS), and/or software as a service (SaaS). FIG.s 53 and 54 illustrate schematics of exemplary implementations of the cloud computing/architecture(s) in which the illustrative computer- based systems or platforms of the present disclosure may be specifically configured to operate.

[0294] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as “IP cores” may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that make the logic or processor. Of note, various embodiments described herein may, of course, be implemented using any appropriate hardware and/or computing software languages (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc ).

[0295] In some embodiments, one or more of illustrative computer-based systems or platforms of the present disclosure may include or be incorporated, partially or entirely into at least one personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, and so forth.

[0296] In some embodiments, the illustrative computer-based systems or platforms of the present disclosure may be configured to securely store and/or transmit data by utilizing one or more of encryption techniques (e.g., private/public key pair, Triple Data Encryption Standard (3DES), block cipher algorithms (e.g., IDEA, RC2, RC5, CAST and Skipjack), cryptographic hash algorithms (e.g., MD5, RIPEMD-160, RTRO, SHA-1, SHA-2, Tiger (TTH), WHIRLPOOL, RNGs).

[0297] In some embodiments, as detailed herein, one or more of the computer-based systems of the present disclosure may obtain, manipulate, transfer, store, transform, generate, and/or output any digital object and/or data unit (e.g., from inside and/or outside of a particular application) that can be in any suitable form such as, without limitation, a file, a contact, a task, an email, a message, a map, an entire application (e.g., a calculator), data points, and other suitable data. In some embodiments, as detailed herein, one or more of the computer-based systems of the present disclosure may be implemented across one or more of various computer platforms such as, but not limited to: (1) Linux, (2) Microsoft Windows, (3) OS X (Mac OS), (4) Solaris, (5) UNIX (6) VMWare, (7) Android, (8) Java Platforms, (9) Open Web Platform, (10) Kubemetes or other suitable computer platforms. In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to utilize hardwired circuitry that may be used in place of or in combination with software instructions to implement features consistent with principles of the disclosure. Thus, implementations consistent with principles of the disclosure are not limited to any specific combination of hardware circuitry and software. For example, various embodiments may be embodied in many different ways as a software component such as, without limitation, a stand-alone software package, a combination of software packages, or it may be a software package incorporated as a “tool” in a larger software product.

[0298] For example, exemplary software specifically programmed in accordance with one or more principles of the present disclosure may be downloadable from a network, for example, a website, as a stand-alone product or as an add-in package for installation in an existing software application. For example, exemplary software specifically programmed in accordance with one or more principles of the present disclosure may also be available as a client-server software application, or as a web-enabled software application. For example, exemplary software specifically programmed in accordance with one or more principles of the present disclosure may also be embodied as a software package installed on a hardware device.

[0299] In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to handle numerous concurrent users that may be, but is not limited to, at least 100 (e.g., but not limited to, 100-999), at least 1,000 (e.g., but not limited to, 1,000-9,999 ), at least 10,000 (e.g., but not limited to, 10,000-99,999 ), at least 100,000 (e.g., but not limited to, 100,000-999,999), at least 1,000,000 (e.g., but not limited to, 1,000,000- 9,999,999), at least 10,000,000 (e.g., but not limited to, 10,000,000-99,999,999), at least 100,000,000 (e.g., but not limited to, 100,000,000-999,999,999), at least 1,000,000,000 (e.g., but not limited to, 1,000,000,000-999,999,999,999), and so on.

[0300] In some embodiments, illustrative computer-based systems or platforms of the present disclosure may be configured to output to distinct, specifically programmed graphical user interface implementations of the present disclosure (e.g., a desktop, a web app., etc.). In various implementations of the present disclosure, a final output may be displayed on a displaying screen which may be, without limitation, a screen of a computer, a screen of a mobile device, or the like. In various implementations, the display may be a holographic display. In various implementations, the display may be a transparent surface that may receive a visual projection. Such projections may convey various forms of information, images, or objects. For example, such projections may be a visual overlay for a mobile augmented reality (MAR) application.

[0301] In some embodiments, exemplary inventive, specially programmed computing systems and platforms with associated devices are configured to operate in the distributed network environment, communicating with one another over one or more suitable data communication networks (e.g., the Internet, satellite, etc.) and utilizing one or more suitable data communication protocols/modes such as, without limitation, IPX/SPX, X.25, AX.25, AppleTalk(TM), TCP/IP (e.g., HTTP), near-field wireless communication (NFC), RFID, Narrow Band Internet of Things (NBIOT), 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, ZigBee, and other suitable communication modes.

[0302] The material disclosed herein may be implemented in software or firmware or a combination of them or as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any medium and/or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), and others.

[0303] Publications cited throughout this document are hereby incorporated by reference in their entirety. While one or more embodiments of the present disclosure have been described, it is understood that these embodiments are illustrative only, and not restrictive, and that many modifications may become apparent to those of ordinary skill in the art, including that various embodiments of the inventive methodologies, the illustrative systems and platforms, and the illustrative devices described herein can be utilized in any combination with each other. Further still, the various steps may be carried out in any desired order (and any desired steps may be added and/or any desired steps may be eliminated).

Claims

1. A method for detecting an object-of-interest in an image using edge-based feature detection, comprising: receiving, by at least one processor, an image; detecting, by the at least one processor, at least one line in the image associated with a respective at least one object based on object edge-detection between pixels in the image; determining, by the at least one processor, line measurements for each respective at least one line to produce line features; determining, by the at least one processor, relative measurements between pairs of the line features; comparing, by the at least one processor, the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of-interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determining, by the at least one processor, an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements of the matching pairs and the model relative measurements; and generating, by the at least one processor, a modified image of the image to identify a boundary of the matching object based on the object location.

2. The method of claim 1, wherein the relative measurements comprise a length difference, an angle difference, a midpoint distance and a minimum and maximum object center line angle representing an angle of a line connecting a midpoint of each line feature of a respective pair of line features.

3. The method of claim 1, further comprising determining, by the at least one processor, an affinity value for each pair of a line feature pair of the pairs of line features and a model line feature pair of the pairs of model line features based on a difference between the respective relative measurements and the respective model relative measurements.

4. The method of claim 3, further comprising identifying, by the at least one processor, the matching pairs of line features for each respective pair of the line feature pairs and the model line feature pairs having a respective affinity value that exceeds an affinity threshold.

5. The method of claim 1, further comprising: determining, by the at least one processor, at least one object center vector for each line feature of each matching pair of line features based on a rotation and scaling of the object center vectors of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model object center vector; and determining, by the at least one processor, the object location in the image based on a location voting by overlapping locations of the at least one object center vector for each line feature of each matching pair of line features.

6. The method of claim 5, further comprising: determining, by the at least one processor, an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determining, by the at least one processor, a refinement to the object location in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

7. The method of claim 6, further comprising: determining, by the at least one processor, a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determining, by the at least one processor, a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

8. The method of claim 5, further comprising: determining, by the at least one processor, an angle consistency map according to an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determining, by the at least one processor, a refinement to the object location in the image based on an orientation addition of the angle consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

9. A system for detecting an object-of-interest in an image using edge-based feature detection, comprising: at least one processor configured to perform steps to: receive an image; detect at least one edge in the image associated with a respective at least one object edge in the image; determine edge measurements for each respective at least one edge to produce line features; determine relative measurements between pairs of the line features; compare the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of- interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determine an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements and the model relative measurements; and generate a modified image of the image to identify a boundary of the matching object based on the object location.

10. The system of claim 9, wherein the relative measurements comprise a length difference, an angle difference, a midpoint distance and a minimum and maximum object center line angle representing an angle of a line connecting a midpoint of each line feature of a respective pair of line features.

11. The system of claim 9, wherein the at least one processor is further configured to perform steps to determine an affinity value for each line feature pair of the pairs of line features and a model line feature pair of the pairs of model line features based on a difference between the respective relative measurements and the respective model relative measurements.

12. The system of claim 11, wherein the at least one processor is further configured to perform steps to identify the matching pairs of line features for each respective pair of the line feature pairs and the model line feature pairs having a respective affinity value that exceeds an affinity threshold.

13. The system of claim 9, wherein the at least one processor is further configured to perform steps to: determine at least one object center vector for each line feature of each matching pair of line features based on a rotation and scaling of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model obj ect center vector; and determine the object location in the image based on a location voting by overlapping locations of the at least one obj ect center vector for each line feature of each matching pair of line features.

14. The system of claim 13, wherein the at least one processor is further configured to perform steps to: determine an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determine a refinement to the object location in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

15. The system of claim 14, wherein the at least one processor is further configured to perform steps to: determine a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determine a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

16. The system of claim 13, wherein the at least one processor is further configured to perform steps to: determine an angle consistency map according to an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determine a refinement to the object location in the image based on an orientation addition of the angle consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features

17. A non -transitory computer readable medium storing thereon software, the software comprising program instructions configured to cause at least one processor to perform steps for detecting an object-of-interest in an image using edge-based feature detection, wherein execution of the program instructions by the at least one processor causes that at least one processor to carry the steps of: receiving an image; detecting at least one edge in the image associated with a respective at least one edge in the image; determining edge measurements for each respective at least one edge to produce line features; determining relative measurements between pairs of the line features; comparing the relative measurements to model relative measurements associated with pairs of model line features associated with a model of the object-of-interest to identify matching pairs of the line features that match one or more of the pairs of the model line features; determining an object location for a matching object associated with the matching pairs of the line features based at least in part on a scaling and rotation of the model according to differences between the relative measurements and the model relative measurements; and generating a modified image of the image to identify a boundary of the matching object based on the object location.

18. The method of claim 17, wherein the relative measurements comprise a length difference, an angle difference, a midpoint distance and a minimum and maximum object center line angle representing an angle of a line connecting a midpoint of each line feature of a respective pair of line features.

19. The method of claim 17, further comprising determining an affinity value for each line feature pair of the pairs of line features and a model line feature pair of the pairs of model line features based on a difference between the respective relative measurements and the respective model relative measurements.

20. The method of claim 19, further comprising identifying the matching pairs of line features for each respective line feature pair and the model line feature pairs having a respective affinity value that exceeds an affinity threshold.

21. The method of claim 17, further comprising: determining at least one object center vector for each line feature of each matching pair of line features based on a rotation and scaling of each matching pair of line features relative to a respective associated model pair of line features and a respective at least one model obj ect center vector; and determining the object location in the image based on a location voting by overlapping locations of the at least one object center vector for each line feature of each matching pair of line features.

22. The method of claim 21, further comprising: determining an object scale factor for each matching pair of line features based on a difference in a relative scale of each matching pair of line features and a relative model scale of a respective associated model pair of line features; and determining a refinement to the object location in the image based on an addition of the object scale factor of each matching pair of line features to a respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

23. The method of claim 22, further comprising: determining a scale consistency map for each matching pair of line features representing whether the object scale factor of each matching pair of line features is consistent with each other object scale factor of each other matching pair of line features; and determining a refinement to the object location in the image based on a multiplication of the scale consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features.

24. The method of claim 21, further comprising: determining an angle consistency map according to an object angle for each matching pair of line features based on a difference in a relative angle of each matching pair of line features and a relative model angle of a respective associated model pair of line features; and determining a refinement to the object location in the image based on an orientation addition of the angle consistency map for each matching pair of line features to the respective overlapping location of the at least one object center vector for each respective line feature of each respective matching pair of line features