US20220156618A1 - Ensemble classification algorithms having subclass resolution - Google Patents
Ensemble classification algorithms having subclass resolution Download PDFInfo
- Publication number
- US20220156618A1 US20220156618A1 US17/589,691 US202217589691A US2022156618A1 US 20220156618 A1 US20220156618 A1 US 20220156618A1 US 202217589691 A US202217589691 A US 202217589691A US 2022156618 A1 US2022156618 A1 US 2022156618A1
- Authority
- US
- United States
- Prior art keywords
- distribution
- samples
- probabilities
- known samples
- instructions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G06N7/005—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0202—Market predictions or forecasting for commercial activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N7/00—Computing arrangements based on specific mathematical models
- G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
Definitions
- This disclosure relates generally to classification algorithms, and, more particularly, to ensemble classification algorithms having subclass resolution.
- classifying subjects/samples e.g., persons
- demographics and/or sub-classifications e.g., integer ages
- binary decision trees with tree nodes that end in terminal nodes.
- these tree nodes are used to sub-classify groups of people into demographic categories and/or sub-groups.
- these tree nodes can inaccurately separate and/or isolate characteristics by use of the decision trees, thereby leading to inaccuracies.
- binary decision trees can require significant computational resources and/or time to generate.
- FIG. 1 is a block diagram illustrating an example environment, in which an example data collection may operate to develop class probabilities as disclosed herein.
- FIG. 2 is a block diagram of an example implementation of an example population data analyzer of FIG. 1 in accordance with the teachings of this disclosure.
- FIG. 3 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer of FIGS. 1 and 2 .
- FIG. 4 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer of FIGS. 1 and 2 .
- FIG. 5 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer of FIGS. 1 and 2 .
- FIG. 6 is a set of tables illustrating an example operation of the example population data analyzer of FIGS. 1 and 2 .
- FIG. 7 is a block diagram of an example processor platform capable of executing machine readable instructions to implement the example machine readable instructions of FIGS. 3, 4 and/or 5 .
- Ensemble classification algorithms having subclass resolution are disclosed.
- a group of subjects e.g., persons
- sub-categories e.g., integer age, interests, affiliations, etc.
- binary decision trees are used to determine the sub-categories of primary class categories (e.g., an age range, an age group, etc.).
- the group of subjects represented as input data are recursively split until a terminal node is reached to provide subclass determinations.
- These binary decision trees are susceptible to inaccuracies resulting from divisions at their branching nodes.
- primary class e.g., an age group
- accurate subclass (e.g., an integer age) determination and/or subclass probabilities have been difficult to achieve.
- utilization of branching decision trees/nodes may require significant computational capabilities as well as significant computing time.
- Methods and apparatus disclosed herein leverage ensemble methods while enabling very accurate and computationally efficient determinations and/or probabilistic determinations of category subclasses of a population (e.g., an unknown population).
- the examples disclosed herein utilize fingerprint generation, distribution creation and distribution application to the population to predict probabilities of attributes and/or subclass categories of the population.
- the fingerprint generation may be performed by calculating primary subclass probabilities of primary classes (e.g., an age group) of a group of samples.
- a distribution and/or a probabilistic distribution of sub-classes is created based on the fingerprint generated.
- distribution creation occurs by arranging each of the samples and removing samples in which their respective predicted primary class (e.g., age range) does not match their actual primary class.
- the distribution is compared to calculated fingerprints of individuals of a population to predict subclass probabilities and/or subclass probability distributions for at least a portion (e.g., all) of the population.
- FIG. 1 is a block diagram illustrating an example environment 100 , in which an example data collection may operate to develop class probabilities such as subclass probabilities, as disclosed herein.
- the environment 100 of the illustrated example includes a data collector 102 , and a population data analyzer 104 , which includes a fingerprint generator 106 , a distribution creator 108 and a distribution applicator 110 .
- the population data analyzer 104 and the data collector 102 are communicatively coupled to a network 112 which, in turn, is communicatively coupled to devices 114 as well as a collection facility 116 .
- the data collector 102 of the illustrated example receives data from the devices 114 and/or the collection facility 116 via the network 112 and the data collector 102 .
- the data collector 102 may extract and/or be provided with data pertaining to users (e.g., age information, interest information, affiliation and/or other demographic information) of the respective devices 114 .
- this data which may include known samples and/or control data samples, is provided to the fingerprint generator 106 , which generates a primary class fingerprint for each sample of the data received via the network 112 .
- the primary class fingerprint may be a number (e.g., an integer) representing a characteristic and/or category of each sample.
- the primary class fingerprint may represent an interest or affiliation.
- the example distribution creator 108 generates a probability distribution of sub-classes based on the data and, in turn, the example distribution applicator 110 compares the generated probability distribution to the group/population to determine probability distributions of each of the group/population.
- the distribution is generated or created by removing samples that have different actual primary classes from predicted primary classes to create a more accurate fingerprint and/or collection of fingerprints and/or resultant probability distributions.
- the samples are arranged and/or sorted in an array to generate the distribution.
- the array is pivoted to define a sub-class probability distribution based on attributes and/or classifications of the samples.
- FIG. 2 is a block diagram of an example implementation of the population data analyzer 104 of FIG. 1 in accordance with the teachings of this disclosure.
- the example population data analyzer 104 includes the fingerprint generator 106 , the distribution creator 108 , and the distribution applicator 110 described above in connection with FIG. 1 .
- the population data analyzer 104 also includes a probability calculator 202 , a distribution organizer/pivoter 204 and storage 212 , which stores a calculated distribution 214 .
- the fingerprint generator 106 of the illustrated example calculates primary class probabilities based on received samples (e.g., a fingerprint trained ensemble) received at the fingerprint generator 106 from the data collector 102 .
- a distribution is created based on each sample of the representative sample data by the distribution creator 108 .
- the example distribution creator 108 and/or the probability calculator 202 generates the calculated distribution represented by a table and/or array including each of the samples to related fingerprints to subclass probabilities.
- the aforementioned calculated distribution 214 is used by the example distribution applicator 110 to generate a probability distribution for each of the unknown samples.
- the distribution organizer/pivoter 204 pivots the table to generate the probability distribution.
- the distribution calculator 110 forwards the calculated probability distribution to the network 112 and/or the collection facility 116 shown in FIG. 1 .
- FIGS. 1 and 2 While an example manner of implementing the example population data analyzer 104 is illustrated in FIGS. 1 and 2 , one or more of the elements, processes and/or devices illustrated in FIGS. 1 and 2 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example data collector 102 , the example population data analyzer 104 , the fingerprint generator 106 , the example distribution creator 108 , the example distribution applicator 110 , the example probability calculator 202 , the example distribution organizer/pivoter 204 and/or, more generally, the example population data analyzer 104 of FIGS. 1 and 2 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware.
- any of the example data collector 102 , the example population data analyzer 104 , the fingerprint generator 106 , the example distribution creator 108 , the example distribution applicator 110 , the example probability calculator 202 , the example distribution organizer/pivoter 204 and/or, more generally, the example population data analyzer 104 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)).
- ASIC application specific integrated circuit
- PLD programmable logic device
- FPLD field programmable logic device
- At least one of the example data collector 102 , the example population data analyzer 104 , the fingerprint generator 106 , the example distribution creator 108 , the example distribution applicator 110 , the example probability calculator 202 , and/or the example distribution organizer/pivoter 204 is/are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the software and/or firmware.
- the example population data analyzer 104 of FIGS. 1 and 2 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIGS. 1 and 2 , and/or may include more than one of any or all of the illustrated elements, processes and devices.
- the machine readable instructions comprise a program for execution by a processor such as the processor 712 shown in the example processor platform 700 discussed below in connection with FIG. 7 .
- the program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with the processor 712 , but the entire program and/or parts thereof could alternatively be executed by a device other than the processor 712 and/or embodied in firmware or dedicated hardware.
- example program is described with reference to the flowcharts illustrated in FIGS. 3, 4 and 5 , many other methods of implementing the example population data analyzer 104 may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.
- FIGS. 3, 4 and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
- a tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.
- tangible computer readable storage medium and “tangible machine readable storage medium” are used interchangeably. Additionally or alternatively, the example processes of FIGS. 3, 4 and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
- coded instructions e.g., computer and/or machine readable instructions
- a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk
- non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.
- phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended.
- the example method 300 of FIG. 3 begins as known samples (e.g., trained ensemble samples, samples from sources that are deemed reliable or trustworthy) are being used to generate a fingerprint and/or fingerprint distribution to be used in characterizing an unknown group of samples.
- the known samples are received via the network 112 at the data collector 102 .
- the fingerprint generator 106 retrieves data representing known samples and/or trained ensemble samples from the data collector 102 (block 302 ).
- the fingerprint generator 106 receives the samples as tables representing survey data and/or known data from a third-party source (e.g., purchased demographic data).
- a third-party source e.g., purchased demographic data
- a fingerprint of primary class probabilities is generated (block 304 ).
- the fingerprint generator 106 of the illustrated example utilizes the known samples to generate primary class probabilities which, in turn, are used to generate a fingerprint of a corresponding to each sample of the known samples.
- the primary class e.g., age range groups
- probabilities are calculated by determining primary class probabilities, p.
- a fingerprint, f, corresponding to each sample is calculated by Equation 1 below:
- n is a total number of classes and c is a configurable parameter to allow for a bias variance trade-off.
- the configurable parameter, c can be used to reduce and/or eliminate errors related to statistical bias and/or variance.
- the probabilities, p, for each class number, r are sorted (e.g., r 1 is a class number of the class with the highest probability while r n is a class number with the lowest probability).
- a fingerprint and/or a fingerprint value is calculated for each of the samples.
- a fingerprint is calculated for a portion of the samples (e.g., based on sample reliability, relevance, significance and/or lack of correlation of the samples).
- a distribution of samples based on the generated fingerprint is created (block 306 ).
- the distribution creator 108 generates/creates probabilistic determinations of sub-classes based on the fingerprint(s).
- the distribution creator 108 and/or the probability calculator 202 organizes a table/array of the known samples and pivots this table/array to reflect probability distributions.
- a population is then retrieved (block 308 ).
- an unknown population of samples is retrieved and/or received by the distribution applicator 110 and/or the data collector 102 so that the unknown population can be characterized (e.g., characterized in regards to sub-class probabilities).
- the created distribution is applied to predict subclass probabilities of each of the population (block 310 ).
- the example distribution applicator 110 applies the distribution to each sample of the unknown population of samples.
- step 312 it is determined if additional unknown sample groups and/or individual samples are to be applied with the predicted sub-class probabilities (block 312 ). This determination may be made based on the appropriateness of the predicted sub-class probabilities related to potential new unknown sample(s). For example, in the context of the potential new unknown sample(s), the known samples may be inappropriate, irrelevant and/or include different fingerprints and/or fingerprint values from the current unknown samples. If additional populations and/or unknown sample(s) are to be characterized (block 312 ), control of the process returns to block 308 . Otherwise, control of the process proceeds to block 314 .
- a new fingerprint is to be generated (block 314 ). This determination may occur based on a current relevance of the generated fingerprint and/or its corresponding sub-class probability distribution. Additionally or alternatively, this determination may be based on receiving a new unknown sample group with attributes that do not pertain to the current fingerprint and/or sub-class probability distribution. If a new fingerprint is to be generated, control of the process returns to block 302 . Otherwise, the process ends.
- FIG. 4 the example sub-routine 306 of FIG. 3 is shown.
- a distribution of sub-class probabilities based on the known sample group described above in connection with FIG. 3 is being created.
- the sample data is parsed (block 402 ). For example, data related to the known samples is organized and/or revised, as appropriate, by the example distribution creator 108 so that this data can be processed and/or organized. In some examples, the sample data is re-organized and/or reformatted to be placed in an array.
- the distribution creator 108 organizes, reformats and/or arranges individual samples of the known samples in an array and/or table (block 404 ).
- an array and/or table block 404 .
- each sample of the representative sample data is tabulated and/or organized by the distribution creator 108 .
- the example distribution creator 108 generates the Table 600 organized based on each of the samples.
- the example Table 600 includes a column 604 , which represents true primary classes, a column 606 , which represents true subclasses, a column 608 , which represents predicted primary classes, and a column 610 , which represents finger print values.
- the Table 600 also includes rows 612 representing each of the samples, which are each listed as a user identification (UID).
- UID user identification
- the example table 600 is shown related to age group primary classes (e.g., young, middle, senior, etc.) and integer (e.g., number) subclasses (e.g., age 5, 10, 25 years old, etc.).
- the fingerprint values shown in the Table 600 represent characteristics that are represented as values in this example. While the fingerprints of the illustrated example are shown as values, they may be any appropriate characteristic such as, but not limited to, interests, affiliations, consumer identities, shopper categories, viewership identities and/or identifications (e.g., ethnicity, status), etc.
- the distribution creator 108 compares a true (e.g., known) primary class with a predicted primary class for each sample of the known samples (e.g., samples of a known and/or verified group) (block 406 ).
- a weighting factor is assigned to the samples of the array (block 407 ). For examples, this weighting can be used to provide greater significance to some of the individual samples (e.g., higher reliability samples, etc.) relative to the others to be used in probability distribution generation.
- the rows are removed from the array (block 410 ) and the process proceeds to block 412 .
- individual samples in which a corresponding true primary class does not match a predicted primary class are removed from the array or table. Briefly referring to FIG. 6 , this elimination is illustrated above as a strike through 614 of UID number 2 in the Table 600 .
- a distribution array is generated (block 412 ) and the process ends/returns.
- the distribution array is generated by the example distribution organizer/pivoter 204 and/or the probability calculator 202 pivoting the table and/or array.
- the Table 600 is pivoted to produce a distribution Table 620 of FIG. 6 .
- the individual samples of Table 600 have been calculated and/or compiled into probabilities related to the subclasses (e.g., integer ages) by the probability calculator 202 , for example.
- the probability calculator 202 determines and/or calculates the probabilities of the fingerprints based on multiple samples.
- this calculated distribution is stored in the example storage 212 as the calculated distribution 214 .
- at least a portion of the individual samples of the known samples are weighted (e.g., some samples have a higher weight in predicting the subclass probabilities, weighted samples) to generate the distribution array.
- the samples with higher reliabilities e.g., samples from more reliable sources
- the example subroutine 310 of FIG. 3 determines and/or characterizes a distribution of the unknown population (e.g., an unknown sample of individuals, an unknown population group, a population, etc.).
- the aforementioned calculated distribution 214 is used to characterize the unknown sample by the example distribution applicator 110 .
- the distribution applicator 110 calculates primary class probabilities of the unknown population (block 502 ).
- the distribution applicator 110 uses the calculated primary class probabilities to create fingerprint values of each of the unknown population (block 504 ).
- the distribution applicator 110 and/or the fingerprint generator 106 calculates fingerprint values of each of the unknown samples.
- these fingerprint values may be calculated using Equation 1 described above in connection with FIG. 1 .
- the distribution applicator 110 compares the calculated fingerprints to the generated distribution array (block 506 ) and the process ends/returns.
- the fingerprints are compared to the calculated distribution 214 to determine sub-class probabilities of each of the unknown samples.
- sub-classes and/or sub-class designations are assigned to each of the unknown samples based on the predicted sub-class probabilities.
- a subclass can be randomly assigned to individual sample(s) of the unknown sample based on the calculated distribution 214 .
- FIG. 7 is a block diagram of an example processor platform 700 capable of executing the instructions of FIGS. 3, 4 and 5 to implement the population data analyzer 104 of FIGS. 1 and 2 .
- the processor platform 700 can be, for example, a server, a personal computer, a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a personal video recorder, a set top box, or any other type of computing device.
- a mobile device e.g., a cell phone, a smart phone, a tablet such as an iPad
- PDA personal digital assistant
- an Internet appliance e.g., a DVD player, a CD player, a digital video recorder, a Blu-ray player, a personal video recorder, a set top box, or any other type of computing device.
- the processor platform 700 of the illustrated example includes a processor 712 .
- the processor 712 of the illustrated example is hardware.
- the processor 712 can be implemented by one or more integrated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer.
- the processor 712 of the illustrated example includes a local memory 713 (e.g., a cache).
- the example processor 712 also includes the example fingerprint generator 106 , the example distribution creator 108 , the example distribution applicator 110 , the example probability calculator 202 and the example distribution organizer/pivoter 204 .
- the processor 712 of the illustrated example is in communication with a main memory including a volatile memory 714 and a non-volatile memory 716 via a bus 718 .
- the volatile memory 714 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any other type of random access memory device.
- the non-volatile memory 716 may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory 714 , 716 is controlled by a memory controller.
- the processor platform 700 of the illustrated example also includes an interface circuit 720 .
- the interface circuit 720 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface.
- one or more input devices 722 are connected to the interface circuit 720 .
- the input device(s) 722 permit(s) a user to enter data and commands into the processor 712 .
- the input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system.
- One or more output devices 724 are also connected to the interface circuit 720 of the illustrated example.
- the output devices 724 can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touchscreen, a tactile output device, a printer and/or speakers).
- the interface circuit 720 of the illustrated example thus, typically includes a graphics driver card, a graphics driver chip or a graphics driver processor.
- the interface circuit 720 of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 726 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.).
- a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 726 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.).
- DSL digital subscriber line
- the processor platform 700 of the illustrated example also includes one or more mass storage devices 728 for storing software and/or data.
- mass storage devices 728 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives.
- the coded instructions 732 of FIGS. 3 and 4 may be stored in the mass storage device 728 , in the volatile memory 714 , in the non-volatile memory 716 , and/or on a removable tangible computer readable storage medium such as a CD or DVD.
- the above disclosed methods and apparatus provide an effective and computationally efficient manner of calculating subclass probabilities.
- the examples disclosed herein may utilize probability generation instead of binary trees, which require numerous levels of division before a terminal node is reached, thereby improving computational efficiency.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Entrepreneurship & Innovation (AREA)
- Medical Informatics (AREA)
- Game Theory and Decision Science (AREA)
- Economics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Probability & Statistics with Applications (AREA)
- Algebra (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Collating Specific Patterns (AREA)
- Image Analysis (AREA)
- Computational Linguistics (AREA)
Abstract
Description
- This patent arises from a continuation of U.S. patent application Ser. No. 15/387,302, (now U.S. Pat. No. 11,250,339) which was filed on Dec. 21, 2016, which claims the benefit of U.S. Provisional Application 62/353,341, which was filed Jun. 22, 2016. U.S. patent application Ser. No. 15/387,302 and U.S. Provisional Patent Application No. 62/353,341 are hereby incorporated herein by reference in their entireties. Priority to U.S. patent application Ser. No. 15/387,302 and U.S. Provisional Patent Application No. 62/353,341 is hereby claimed.
- This disclosure relates generally to classification algorithms, and, more particularly, to ensemble classification algorithms having subclass resolution.
- In recent years, classifying subjects/samples (e.g., persons) to determine their demographics and/or sub-classifications (e.g., integer ages) has been typically accomplished using binary decision trees with tree nodes that end in terminal nodes. In particular, these tree nodes are used to sub-classify groups of people into demographic categories and/or sub-groups. However, these tree nodes can inaccurately separate and/or isolate characteristics by use of the decision trees, thereby leading to inaccuracies. Further, binary decision trees can require significant computational resources and/or time to generate.
-
FIG. 1 is a block diagram illustrating an example environment, in which an example data collection may operate to develop class probabilities as disclosed herein. -
FIG. 2 is a block diagram of an example implementation of an example population data analyzer ofFIG. 1 in accordance with the teachings of this disclosure. -
FIG. 3 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer ofFIGS. 1 and 2 . -
FIG. 4 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer ofFIGS. 1 and 2 . -
FIG. 5 is a flowchart representative of example machine readable instructions for implementing the example population data analyzer ofFIGS. 1 and 2 . -
FIG. 6 is a set of tables illustrating an example operation of the example population data analyzer ofFIGS. 1 and 2 . -
FIG. 7 is a block diagram of an example processor platform capable of executing machine readable instructions to implement the example machine readable instructions ofFIGS. 3, 4 and/or 5 . - The figures are not to scale. Instead, to clarify multiple layers and regions, the thickness of the layers may be enlarged in the drawings. Wherever possible, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts.
- Ensemble classification algorithms having subclass resolution are disclosed. Typically, in known examples, to classify a group of subjects (e.g., persons) by sub-categories (e.g., integer age, interests, affiliations, etc.), binary decision trees are used to determine the sub-categories of primary class categories (e.g., an age range, an age group, etc.). In such known examples, the group of subjects represented as input data are recursively split until a terminal node is reached to provide subclass determinations. These binary decision trees are susceptible to inaccuracies resulting from divisions at their branching nodes. In particular, for known binary decision trees, primary class (e.g., an age group) prediction has shown to be effective. However, accurate subclass (e.g., an integer age) determination and/or subclass probabilities have been difficult to achieve. Additionally, utilization of branching decision trees/nodes may require significant computational capabilities as well as significant computing time.
- Methods and apparatus disclosed herein leverage ensemble methods while enabling very accurate and computationally efficient determinations and/or probabilistic determinations of category subclasses of a population (e.g., an unknown population). In particular, the examples disclosed herein utilize fingerprint generation, distribution creation and distribution application to the population to predict probabilities of attributes and/or subclass categories of the population. According to examples disclosed herein, the fingerprint generation may be performed by calculating primary subclass probabilities of primary classes (e.g., an age group) of a group of samples. According to the examples disclosed herein, a distribution and/or a probabilistic distribution of sub-classes is created based on the fingerprint generated. In some examples, distribution creation occurs by arranging each of the samples and removing samples in which their respective predicted primary class (e.g., age range) does not match their actual primary class. In some examples, once a distribution has been created, the distribution is compared to calculated fingerprints of individuals of a population to predict subclass probabilities and/or subclass probability distributions for at least a portion (e.g., all) of the population.
-
FIG. 1 is a block diagram illustrating anexample environment 100, in which an example data collection may operate to develop class probabilities such as subclass probabilities, as disclosed herein. Theenvironment 100 of the illustrated example includes adata collector 102, and apopulation data analyzer 104, which includes afingerprint generator 106, adistribution creator 108 and adistribution applicator 110. In this example, thepopulation data analyzer 104 and thedata collector 102 are communicatively coupled to anetwork 112 which, in turn, is communicatively coupled todevices 114 as well as acollection facility 116. - To determine sub-class probability distributions (e.g., integer age distributions, interest probabilities, affiliation probabilities, etc.) of a group/population of entities (e.g., persons, groups of people, an unknown group of entities, etc.), the
data collector 102 of the illustrated example receives data from thedevices 114 and/or thecollection facility 116 via thenetwork 112 and thedata collector 102. For example, thedata collector 102 may extract and/or be provided with data pertaining to users (e.g., age information, interest information, affiliation and/or other demographic information) of therespective devices 114. In this example, this data, which may include known samples and/or control data samples, is provided to thefingerprint generator 106, which generates a primary class fingerprint for each sample of the data received via thenetwork 112. The primary class fingerprint may be a number (e.g., an integer) representing a characteristic and/or category of each sample. In some examples, the primary class fingerprint may represent an interest or affiliation. - In turn, the
example distribution creator 108 generates a probability distribution of sub-classes based on the data and, in turn, theexample distribution applicator 110 compares the generated probability distribution to the group/population to determine probability distributions of each of the group/population. - In some examples, the distribution is generated or created by removing samples that have different actual primary classes from predicted primary classes to create a more accurate fingerprint and/or collection of fingerprints and/or resultant probability distributions. In some examples, the samples are arranged and/or sorted in an array to generate the distribution. In some examples, the array is pivoted to define a sub-class probability distribution based on attributes and/or classifications of the samples.
-
FIG. 2 is a block diagram of an example implementation of thepopulation data analyzer 104 ofFIG. 1 in accordance with the teachings of this disclosure. As can be seen in the illustrated example ofFIG. 2 , the examplepopulation data analyzer 104 includes thefingerprint generator 106, thedistribution creator 108, and thedistribution applicator 110 described above in connection withFIG. 1 . Further, thepopulation data analyzer 104 also includes aprobability calculator 202, a distribution organizer/pivoter 204 andstorage 212, which stores a calculateddistribution 214. - To generate a fingerprint for each sample of a collection of samples that are based on representative sample data (e.g., a known collection/population of samples, a verified population of samples, a sample group, etc.), the
fingerprint generator 106 of the illustrated example calculates primary class probabilities based on received samples (e.g., a fingerprint trained ensemble) received at thefingerprint generator 106 from thedata collector 102. - According to the illustrated example, a distribution is created based on each sample of the representative sample data by the
distribution creator 108. In particular, theexample distribution creator 108 and/or theprobability calculator 202 generates the calculated distribution represented by a table and/or array including each of the samples to related fingerprints to subclass probabilities. To determine and/or characterize the calculateddistribution 214 of an unknown sample (e.g., an unknown sample of individuals, an unknown population group, etc.), the aforementioned calculateddistribution 214 is used by theexample distribution applicator 110 to generate a probability distribution for each of the unknown samples. In some examples, the distribution organizer/pivoter 204 pivots the table to generate the probability distribution. In this example, once thedistribution applicator 110 calculates a probability distribution for each of the unknown sample, thedistribution calculator 110 forwards the calculated probability distribution to thenetwork 112 and/or thecollection facility 116 shown inFIG. 1 . - While an example manner of implementing the example
population data analyzer 104 is illustrated inFIGS. 1 and 2 , one or more of the elements, processes and/or devices illustrated inFIGS. 1 and 2 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, theexample data collector 102, the examplepopulation data analyzer 104, thefingerprint generator 106, theexample distribution creator 108, theexample distribution applicator 110, theexample probability calculator 202, the example distribution organizer/pivoter 204 and/or, more generally, the examplepopulation data analyzer 104 ofFIGS. 1 and 2 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of theexample data collector 102, the examplepopulation data analyzer 104, thefingerprint generator 106, theexample distribution creator 108, theexample distribution applicator 110, theexample probability calculator 202, the example distribution organizer/pivoter 204 and/or, more generally, the examplepopulation data analyzer 104 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of theexample data collector 102, the examplepopulation data analyzer 104, thefingerprint generator 106, theexample distribution creator 108, theexample distribution applicator 110, theexample probability calculator 202, and/or the example distribution organizer/pivoter 204 is/are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the software and/or firmware. Further still, the example population data analyzer 104 ofFIGS. 1 and 2 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated inFIGS. 1 and 2 , and/or may include more than one of any or all of the illustrated elements, processes and devices. - Flowcharts representative of example machine readable instructions for implementing the
population data analyzer 104 ofFIGS. 1 and 2 is shown inFIGS. 3, 4 and 5 . In this example, the machine readable instructions comprise a program for execution by a processor such as theprocessor 712 shown in theexample processor platform 700 discussed below in connection withFIG. 7 . The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with theprocessor 712, but the entire program and/or parts thereof could alternatively be executed by a device other than theprocessor 712 and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowcharts illustrated inFIGS. 3, 4 and 5 , many other methods of implementing the example population data analyzer 104 may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. - As mentioned above, the example processes of
FIGS. 3, 4 and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, “tangible computer readable storage medium” and “tangible machine readable storage medium” are used interchangeably. Additionally or alternatively, the example processes ofFIGS. 3, 4 and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, when the phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended. - The
example method 300 ofFIG. 3 begins as known samples (e.g., trained ensemble samples, samples from sources that are deemed reliable or trustworthy) are being used to generate a fingerprint and/or fingerprint distribution to be used in characterizing an unknown group of samples. In this example, the known samples are received via thenetwork 112 at thedata collector 102. - According to the illustrated example of
FIG. 3 , thefingerprint generator 106 retrieves data representing known samples and/or trained ensemble samples from the data collector 102 (block 302). For example, thefingerprint generator 106 receives the samples as tables representing survey data and/or known data from a third-party source (e.g., purchased demographic data). - In this example, a fingerprint of primary class probabilities is generated (block 304). The
fingerprint generator 106 of the illustrated example utilizes the known samples to generate primary class probabilities which, in turn, are used to generate a fingerprint of a corresponding to each sample of the known samples. In particular, the primary class (e.g., age range groups) probabilities are calculated by determining primary class probabilities, p. As a result, a fingerprint, f, corresponding to each sample is calculated byEquation 1 below: -
f=Σ i=1 c r i *n (c−i) (1) - , where r is the class number (e.g., numbered based on a ranked order of the respective probabilities, p), n is a total number of classes and c is a configurable parameter to allow for a bias variance trade-off. In particular, the configurable parameter, c, can be used to reduce and/or eliminate errors related to statistical bias and/or variance. In some examples, the probabilities, p, for each class number, r, are sorted (e.g., r1 is a class number of the class with the highest probability while rn is a class number with the lowest probability). In this example, a fingerprint and/or a fingerprint value is calculated for each of the samples. However, in some examples, a fingerprint is calculated for a portion of the samples (e.g., based on sample reliability, relevance, significance and/or lack of correlation of the samples).
- Next, a distribution of samples based on the generated fingerprint is created (block 306). As mentioned above in connection with
FIG. 2 , thedistribution creator 108 generates/creates probabilistic determinations of sub-classes based on the fingerprint(s). As discussed in greater detail below in connection withFIG. 4 , thedistribution creator 108 and/or theprobability calculator 202 organizes a table/array of the known samples and pivots this table/array to reflect probability distributions. - A population is then retrieved (block 308). In particular, an unknown population of samples is retrieved and/or received by the
distribution applicator 110 and/or thedata collector 102 so that the unknown population can be characterized (e.g., characterized in regards to sub-class probabilities). - Next, the created distribution is applied to predict subclass probabilities of each of the population (block 310). As discussed in greater detail below in connection with
FIG. 5 , theexample distribution applicator 110 applies the distribution to each sample of the unknown population of samples. - Next, it is determined if additional unknown sample groups and/or individual samples are to be applied with the predicted sub-class probabilities (block 312). This determination may be made based on the appropriateness of the predicted sub-class probabilities related to potential new unknown sample(s). For example, in the context of the potential new unknown sample(s), the known samples may be inappropriate, irrelevant and/or include different fingerprints and/or fingerprint values from the current unknown samples. If additional populations and/or unknown sample(s) are to be characterized (block 312), control of the process returns to block 308. Otherwise, control of the process proceeds to block 314.
- It is then determined whether a new fingerprint is to be generated (block 314). This determination may occur based on a current relevance of the generated fingerprint and/or its corresponding sub-class probability distribution. Additionally or alternatively, this determination may be based on receiving a new unknown sample group with attributes that do not pertain to the current fingerprint and/or sub-class probability distribution. If a new fingerprint is to be generated, control of the process returns to block 302. Otherwise, the process ends.
- Turning to
FIG. 4 , theexample sub-routine 306 ofFIG. 3 is shown. In this example, a distribution of sub-class probabilities based on the known sample group described above in connection withFIG. 3 is being created. - First, the sample data is parsed (block 402). For example, data related to the known samples is organized and/or revised, as appropriate, by the
example distribution creator 108 so that this data can be processed and/or organized. In some examples, the sample data is re-organized and/or reformatted to be placed in an array. - Next, according to the illustrated example, the
distribution creator 108, theprobability calculator 202 and/or the distribution organizer/pivoter 204 organizes, reformats and/or arranges individual samples of the known samples in an array and/or table (block 404). To illustrate generating this table, referring briefly toFIG. 6 , to create a table/array 600, each sample of the representative sample data is tabulated and/or organized by thedistribution creator 108. In particular, theexample distribution creator 108 generates the Table 600 organized based on each of the samples. The example Table 600 includes acolumn 604, which represents true primary classes, acolumn 606, which represents true subclasses, acolumn 608, which represents predicted primary classes, and acolumn 610, which represents finger print values. The Table 600 also includesrows 612 representing each of the samples, which are each listed as a user identification (UID). For illustrative purposes, the example table 600 is shown related to age group primary classes (e.g., young, middle, senior, etc.) and integer (e.g., number) subclasses (e.g.,age - According to the illustrated example, the
distribution creator 108 compares a true (e.g., known) primary class with a predicted primary class for each sample of the known samples (e.g., samples of a known and/or verified group) (block 406). - In some examples, a weighting factor is assigned to the samples of the array (block 407). For examples, this weighting can be used to provide greater significance to some of the individual samples (e.g., higher reliability samples, etc.) relative to the others to be used in probability distribution generation.
- In this example, it is then determined if there are rows of the array in which the true primary class does not match the corresponding predicted primary class (block 408). If there are rows with such a mismatch (block 408), control of the process proceeds to block 410. Otherwise, the process proceeds to block 412.
- In some examples, if there are rows with mismatches, the rows are removed from the array (block 410) and the process proceeds to block 412. In particular, individual samples in which a corresponding true primary class does not match a predicted primary class, are removed from the array or table. Briefly referring to
FIG. 6 , this elimination is illustrated above as a strike through 614 of UID number 2 in the Table 600. - Next, according to the illustrated example, a distribution array is generated (block 412) and the process ends/returns. In this example, the distribution array is generated by the example distribution organizer/
pivoter 204 and/or theprobability calculator 202 pivoting the table and/or array. For example, as shown inFIG. 6 , after the samples with non-matching primary classes are removed, the Table 600 is pivoted to produce a distribution Table 620 ofFIG. 6 . As can be seen in the example Table 620, the individual samples of Table 600 have been calculated and/or compiled into probabilities related to the subclasses (e.g., integer ages) by theprobability calculator 202, for example. In particular, theprobability calculator 202 determines and/or calculates the probabilities of the fingerprints based on multiple samples. - In some examples, this calculated distribution is stored in the
example storage 212 as thecalculated distribution 214. In some examples, at least a portion of the individual samples of the known samples are weighted (e.g., some samples have a higher weight in predicting the subclass probabilities, weighted samples) to generate the distribution array. In particular, the samples with higher reliabilities (e.g., samples from more reliable sources) may carry a higher weight in the probability calculations. - Turning to
FIG. 5 , theexample subroutine 310 ofFIG. 3 is shown. Theexample subroutine 310 of the illustrated example determines and/or characterizes a distribution of the unknown population (e.g., an unknown sample of individuals, an unknown population group, a population, etc.). In particular, the aforementionedcalculated distribution 214 is used to characterize the unknown sample by theexample distribution applicator 110. - In this example, the
distribution applicator 110 calculates primary class probabilities of the unknown population (block 502). - Next, the
distribution applicator 110 uses the calculated primary class probabilities to create fingerprint values of each of the unknown population (block 504). In this example thedistribution applicator 110 and/or thefingerprint generator 106 calculates fingerprint values of each of the unknown samples. In particular, these fingerprint values may be calculated usingEquation 1 described above in connection withFIG. 1 . - Next, the
distribution applicator 110 compares the calculated fingerprints to the generated distribution array (block 506) and the process ends/returns. In particular, the fingerprints are compared to the calculateddistribution 214 to determine sub-class probabilities of each of the unknown samples. In some examples, sub-classes and/or sub-class designations are assigned to each of the unknown samples based on the predicted sub-class probabilities. Additionally or alternatively, a subclass can be randomly assigned to individual sample(s) of the unknown sample based on the calculateddistribution 214. -
FIG. 7 is a block diagram of anexample processor platform 700 capable of executing the instructions ofFIGS. 3, 4 and 5 to implement thepopulation data analyzer 104 ofFIGS. 1 and 2 . Theprocessor platform 700 can be, for example, a server, a personal computer, a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a personal video recorder, a set top box, or any other type of computing device. - The
processor platform 700 of the illustrated example includes aprocessor 712. Theprocessor 712 of the illustrated example is hardware. For example, theprocessor 712 can be implemented by one or more integrated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer. - The
processor 712 of the illustrated example includes a local memory 713 (e.g., a cache). Theexample processor 712 also includes theexample fingerprint generator 106, theexample distribution creator 108, theexample distribution applicator 110, theexample probability calculator 202 and the example distribution organizer/pivoter 204. Theprocessor 712 of the illustrated example is in communication with a main memory including avolatile memory 714 and anon-volatile memory 716 via abus 718. Thevolatile memory 714 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any other type of random access memory device. Thenon-volatile memory 716 may be implemented by flash memory and/or any other desired type of memory device. Access to themain memory - The
processor platform 700 of the illustrated example also includes aninterface circuit 720. Theinterface circuit 720 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface. - In the illustrated example, one or
more input devices 722 are connected to theinterface circuit 720. The input device(s) 722 permit(s) a user to enter data and commands into theprocessor 712. The input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system. - One or
more output devices 724 are also connected to theinterface circuit 720 of the illustrated example. Theoutput devices 724 can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touchscreen, a tactile output device, a printer and/or speakers). Theinterface circuit 720 of the illustrated example, thus, typically includes a graphics driver card, a graphics driver chip or a graphics driver processor. - The
interface circuit 720 of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 726 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.). - The
processor platform 700 of the illustrated example also includes one or moremass storage devices 728 for storing software and/or data. Examples of suchmass storage devices 728 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives. - The coded
instructions 732 ofFIGS. 3 and 4 may be stored in themass storage device 728, in thevolatile memory 714, in thenon-volatile memory 716, and/or on a removable tangible computer readable storage medium such as a CD or DVD. - From the foregoing, it will be appreciated that the above disclosed methods and apparatus provide an effective and computationally efficient manner of calculating subclass probabilities. For example, the examples disclosed herein may utilize probability generation instead of binary trees, which require numerous levels of division before a terminal node is reached, thereby improving computational efficiency.
- This application claims the benefit under 35 U.S.C. § 119(e) to U.S. Provisional Application 62/353,341 titled “Subclass Resolution for Ensemble Classification Algorithms,” filed Jun. 22, 2016, which is incorporated herein by this reference in its entirety.
- Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.
Claims (23)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/589,691 US20220156618A1 (en) | 2016-06-22 | 2022-01-31 | Ensemble classification algorithms having subclass resolution |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662353341P | 2016-06-22 | 2016-06-22 | |
US15/387,302 US11250339B2 (en) | 2016-06-22 | 2016-12-21 | Ensemble classification algorithms having subclass resolution |
US17/589,691 US20220156618A1 (en) | 2016-06-22 | 2022-01-31 | Ensemble classification algorithms having subclass resolution |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/387,302 Continuation US11250339B2 (en) | 2016-06-22 | 2016-12-21 | Ensemble classification algorithms having subclass resolution |
Publications (1)
Publication Number | Publication Date |
---|---|
US20220156618A1 true US20220156618A1 (en) | 2022-05-19 |
Family
ID=60677710
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/387,302 Active 2039-05-18 US11250339B2 (en) | 2016-06-22 | 2016-12-21 | Ensemble classification algorithms having subclass resolution |
US17/589,691 Pending US20220156618A1 (en) | 2016-06-22 | 2022-01-31 | Ensemble classification algorithms having subclass resolution |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/387,302 Active 2039-05-18 US11250339B2 (en) | 2016-06-22 | 2016-12-21 | Ensemble classification algorithms having subclass resolution |
Country Status (1)
Country | Link |
---|---|
US (2) | US11250339B2 (en) |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB0304639D0 (en) * | 2003-02-28 | 2003-04-02 | Kiq Ltd | Classification using re-sampling of probability estimates |
US7490071B2 (en) * | 2003-08-29 | 2009-02-10 | Oracle Corporation | Support vector machines processing system |
JP4764273B2 (en) * | 2006-06-30 | 2011-08-31 | キヤノン株式会社 | Image processing apparatus, image processing method, program, and storage medium |
US20100070339A1 (en) | 2008-09-15 | 2010-03-18 | Google Inc. | Associating an Entity with a Category |
CA2737486A1 (en) | 2008-09-16 | 2010-03-25 | Bayer Healthcare Llc | Interferon response in clinical samples (iris) |
JP2016533497A (en) | 2013-07-09 | 2016-10-27 | エスアールアイ インターナショナルSRI International | Biomarker panel and microplasma filter for dose assessment of radiation injury |
US9466029B1 (en) * | 2013-10-15 | 2016-10-11 | Google Inc. | Demographic inference calibration |
US20160189182A1 (en) | 2014-12-31 | 2016-06-30 | The Nielsen Company (Us), Llc | Methods and apparatus to correct age misattribution in media impressions |
-
2016
- 2016-12-21 US US15/387,302 patent/US11250339B2/en active Active
-
2022
- 2022-01-31 US US17/589,691 patent/US20220156618A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US20170372213A1 (en) | 2017-12-28 |
US11250339B2 (en) | 2022-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110347835B (en) | Text clustering method, electronic device and storage medium | |
US7756535B1 (en) | Lightweight content filtering system for mobile phones | |
US11023778B2 (en) | Techniques to embed a data object into a multidimensional frame | |
US10637826B1 (en) | Policy compliance verification using semantic distance and nearest neighbor search of labeled content | |
CN112528025A (en) | Text clustering method, device and equipment based on density and storage medium | |
CN110636445B (en) | WIFI-based indoor positioning method, device, equipment and medium | |
US20160019671A1 (en) | Identifying multimedia objects based on multimedia fingerprint | |
CN110019790B (en) | Text recognition, text monitoring, data object recognition and data processing method | |
CN113011889B (en) | Account anomaly identification method, system, device, equipment and medium | |
CN110674636B (en) | Power consumption behavior analysis method | |
CN111428028A (en) | Information classification method based on deep learning and related equipment | |
WO2019191266A1 (en) | Object classification method, apparatus, server, and storage medium | |
US11030532B2 (en) | Information processing apparatus, information processing method, and non-transitory computer readable storage medium | |
CN114547257B (en) | Class matching method and device, computer equipment and storage medium | |
CN111241106A (en) | Approximate data processing method, device, medium and electronic equipment | |
US11625630B2 (en) | Identifying intent in dialog data through variant assessment | |
CN115905648B (en) | Gaussian mixture model-based user group and financial user group analysis method and device | |
US20220156618A1 (en) | Ensemble classification algorithms having subclass resolution | |
CN114266255B (en) | Corpus classification method, apparatus, device and storage medium based on clustering model | |
CN112148902B (en) | Data processing method, device, server and storage medium | |
CN114513578A (en) | Outbound method, device, computer equipment and storage medium | |
US10861436B1 (en) | Audio call classification and survey system | |
CN116823069B (en) | Intelligent customer service quality inspection method based on text analysis and related equipment | |
CN117392402A (en) | Self-supervision learning method and device, computer equipment and storage medium | |
CN118656380A (en) | Model training method, contract term retrieval method and related equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: THE NIELSEN COMPANY (US), LLC, NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SULLIVAN, JONATHAN;BRYDON, EVAN;SIGNING DATES FROM 20161227 TO 20170221;REEL/FRAME:060498/0822 |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063560/0547 Effective date: 20230123 |
|
AS | Assignment |
Owner name: CITIBANK, N.A., NEW YORK Free format text: SECURITY INTEREST;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063561/0381 Effective date: 20230427 |
|
AS | Assignment |
Owner name: ARES CAPITAL CORPORATION, NEW YORK Free format text: SECURITY INTEREST;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063574/0632 Effective date: 20230508 |