US20220277244A1 - Methods, systems and apparatus for calibrating data using relaxed benchmark constraints - Google Patents
Methods, systems and apparatus for calibrating data using relaxed benchmark constraints Download PDFInfo
- Publication number
- US20220277244A1 US20220277244A1 US17/745,608 US202217745608A US2022277244A1 US 20220277244 A1 US20220277244 A1 US 20220277244A1 US 202217745608 A US202217745608 A US 202217745608A US 2022277244 A1 US2022277244 A1 US 2022277244A1
- Authority
- US
- United States
- Prior art keywords
- criteria
- calibration model
- variables
- data
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06313—Resource planning in a project environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
Definitions
- This disclosure relates generally to methods, systems, and apparatus for calibrating data and, more specifically, to methods, systems, and apparatus for calibrating data using relaxed benchmark constraints.
- Traditional models and/or techniques for calibrating data adjust and/or modify projection weights for the purpose of improving upon the accuracy of estimates that may be yielded from applying the projection weights to panelist and/or reporting data (e.g., survey data).
- the goal of such traditional calibration models and/or techniques is to adjust and/or modify the projection weights such that the weighted totals and/or results of the panelist and/or reporting data closely match known totals and/or results of reference and/or benchmark data (e.g., census data).
- FIG. 1 is a block diagram of an example calibration apparatus constructed in accordance with the teachings of this disclosure.
- FIG. 2 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus of FIG. 1 to calibrate data using relaxed benchmark constraints incorporated into a soft calibration model.
- FIG. 3 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus of FIG. 1 to determine criteria-based input data for the target loss function of the soft calibration model to be executed by the calibration apparatus.
- FIG. 4 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus of FIG. 1 to determine unit-based input data for the weight loss function of the soft calibration model to be executed by the calibration apparatus.
- FIG. 5 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus of FIG. 1 to determine model parameter data for the soft calibration model to be executed by the calibration apparatus.
- FIG. 6 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus of FIG. 1 to validate the soft calibration model executed by the calibration apparatus.
- FIG. 7 is an example processor platform capable of executing the instructions of FIGS. 2-6 to implement the example calibration apparatus of FIG. 1 .
- Panelist and/or reporting data e.g., data obtained from tightly controlled participants having a particular demographic representation
- reference and/or benchmark data e.g., raw data acquired from the market, such as actual sales dollars from retail stores that may be obtained from scanner data at checkout.
- the data measurement company's clients may question the accuracy and/or validity of the data measurement company's ability to accurately project sales and/or usage numbers using the panelist and/or reporting data.
- the methods, systems and apparatus for calibrating data disclosed herein use relaxed benchmark constraints.
- the term “soft calibration” refers to a convex optimized calibration model and/or technique providing a unique solution for calibrating a set of weights (e.g., projection weights) relative to a set of relaxed benchmark constraints.
- the benchmark constraints that may be implemented in the disclosed soft calibration models and/or techniques are relaxed (e.g., loosened, broadened, and/or made less rigid) relative to the benchmark constraints that might otherwise be implemented in traditional hard calibration models and/or techniques.
- the disclosed soft calibration models and/or techniques relax the exact matching requirement of the traditional hard calibration process.
- the disclosed soft calibration models and/or techniques advantageously generate adjusted and/or modified projection weights such that the distance and/or deviation between the estimated totals and the reference and/or benchmark totals is smaller than some particular value (e.g., a threshold and/or budget).
- some particular value e.g., a threshold and/or budget.
- the disclosed soft calibration models and/or techniques advantageously reduce (e.g., eliminate) the above-described errors that may arise in connection with the use of hard calibration models and/or techniques when the calibration weights take on a relatively large magnitude.
- a data measurement company implementing the disclosed soft calibration models and/or techniques in lieu of traditional hard calibration models and/or techniques may advantageously provide and/or generate calibrated panelist and/or reporting data that more closely matches and/or resembles reference and/or benchmark data.
- the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on criteria-based input data to be incorporated into a soft calibration model.
- the criteria-based input data provides inputs for a target loss function of the soft calibration model.
- the criteria-based input data may include criteria associated with panelist and/or reporting data, criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables.
- criteria and/or “criteria” refer to one or more panelist and/or reporting data point(s) to be taken into account when executing and/or performing a calibration model and/or technique.
- the criteria may be identified by a unique set of names and/or characters. In other examples, the criteria may be identified by a unique set of numbers.
- An example indexed set of criteria (M) may be expressed and/or defined as follows:
- k represents a dummy variable
- >0 represents the set of strictly positive natural numbers (e.g., excluding 0)
- m represents the number of criteria.
- a set of criteria variables represents panelist and/or reporting values for each criterion.
- An example set of criteria variables (y) may be expressed and/or defined in scalar and vector notation as follows:
- the criteria variables are not identified and/or made to be equal to any specific prescribed value. Instead, the criteria variables are identified and/or made to be as close as possible to a set of ideal values (e.g., reference and/or benchmark data values).
- a set of ideal values e.g., reference and/or benchmark data values.
- target and/or “criteria target” refer to an ideal value associated with a criteria variable. Targets for the criteria variables are to be provided as input parameters to the soft calibration model.
- An example set of targets (t) may be expressed and/or defined as follows:
- an upper bound and a lower bound may additionally be associated with each criteria variable.
- the upper bound may be a non-negative number that is greater than or equal to the corresponding target.
- the lower bound may be a non-negative number that is smaller than or equal to the corresponding target. Setting both the upper and lower bounds equal to the corresponding target causes the soft calibration model and/or technique to mimic a traditional hard calibration model and/or technique. Conversely, setting the lower bound to zero (0) and the upper bound to a large multiple of the target may result in the lower and upper bounds being practically inactive.
- the upper and lower bounds associated with respective ones of the criteria variables and/or targets are identified independently. Thus, some criteria variables may be required to match their corresponding targets exactly, while others criteria variables may have very loose bounds relative to their corresponding targets.
- Upper and lower bounds for a set of criteria variables are to be supplied as input parameters to the soft calibration model.
- An example set of constraints corresponding to upper bounds (u) and lower bounds (l) to be associated with a set of criteria variables (y) and/or targets (t) may be expressed and/or defined as follows:
- respective ones of the targets to be associated with corresponding ones of the criteria variables may have different measurement scales and/or units. For example, demographic targets measured in terms of a number of people in a group may be mixed together with purchased amount targets measured in terms of a local currency. To enable computation of a reasonable target loss, these different scales and/or units of measurement may be converted to measureless comparable quantities. In some examples, such a conversion may be based on relative distances (e.g., ((y i ⁇ t i )/t i )) between respective ones of the targets and corresponding ones of the criteria variables. In other examples, a scaling parameter (c) may be applied to respective ones of the criteria variables.
- the scaling parameters have positive values.
- Example scaling parameters (c) per criteria variable for computing target loss may be expressed and/or defined as follows:
- an importance parameter (v) may be applied to respective ones of the scaled criteria variables.
- the importance parameters may capture and/or represent preferences associated with respective ones of the scaled criteria variables.
- each importance parameter (v) may be based on a monotonically increasing concave function of the corresponding target (t)
- Example importance parameters (v) per criteria variable for computing target loss may be expressed and/or defined as follows:
- the criteria-based input data may be provided by, accessed from, and/or stored in a tab-separated values (TSV) file having multiple columns.
- TSV tab-separated values
- a first column of a TSV file for the criteria-based input data may include identifiers (e.g., a set of unique names and/or characters) for each criterion.
- a second column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each target.
- a third column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each lower bound.
- a fourth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each upper bound.
- a fifth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each scaling parameter.
- a sixth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each importance parameter.
- the respective values of the second column of the TSV file are greater than or equal to corresponding ones of the respective values of the third column of the TSV file, and less than or equal to corresponding ones of the respective values of the fourth column of the TSV file.
- the respective values of the fifth column and the sixth column of the TSV file are positive, and cannot be zero (0).
- the criteria-based input data may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file.
- the criteria-based input data may be provided by, accessed from, and/or stored in a comma-separated values (CSV) file, or some other type of data file.
- CSV comma-separated values
- target loss refers to a measured distance between (a) a panelist and/or reporting value captured by a criteria variable, and (b) a corresponding target. In some examples, target loss may be expressed and/or defined in terms of a target loss function (F(y)).
- the target loss function (F(y)) will express a weighted average of criteria variable distances while at the same time being robust with respect to outliers.
- the target loss function (F(y)) is both convex and robust.
- the target loss function (F(y)) may be based on a known Huber loss function (H ⁇ ( ⁇ )).
- the target loss function (F(y)) may be expressed and/or defined as follows:
- H ⁇ ( ⁇ ) is the Huber loss function expressed and/or defined as follows:
- H ⁇ ( z ) ⁇ 1 2 ⁇ z 2 for ⁇ ⁇ " ⁇ [LeftBracketingBar]” z ⁇ " ⁇ [RightBracketingBar]” ⁇ ⁇ , ⁇ ⁇ ( ⁇ " ⁇ [LeftBracketingBar]” z ⁇ " ⁇ [RightBracketingBar]” - 1 2 ⁇ ⁇ ) , otherwise .
- the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on unit-based input data to be incorporated into the soft calibration model.
- the unit-based input data provides inputs for a weight loss function of the soft calibration model.
- the unit-based input data may include units associated with panelist and/or reporting data, unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables.
- the term “unit” refers to one or more projection weight(s) to be used and/or calibrated to attain one or more corresponding panelist and/or reporting data point(s) relative to one or more corresponding target(s) when executing and/or performing a calibration model and/or technique.
- the units may be identified by a unique set of names and/or characters. In other examples, the units may be identified by a unique set of numbers.
- An example indexed set of units (N) may be expressed and/or defined as follows:
- k represents a dummy variable
- >0 represents the set of strictly positive natural numbers (e.g., excluding 0)
- n represents the number of units.
- a set of unit variables represents the calibrated weight for each unit.
- An example set of unit variables (x) may be expressed and/or defined in scalar and vector notation as follows:
- an upper bound and a lower bound may additionally be associated with each unit variable.
- the upper bound may be a non-negative number that is greater than or equal to the corresponding initial weight.
- the lower bound may be a non-negative number that is smaller than or equal to the corresponding initial weight.
- the lower bound may be set to zero (0) and the upper bound may be set to a large multiple of the initial weight.
- the lower bound and the upper bound may respectively be determined by dividing (for the lower bound) and/or multiplying (for the upper bound) the initial weight by a constant.
- the upper and lower bounds associated with respective ones of the unit variables and/or initial weights are identified independently.
- Upper and lower bounds for the unit variables are to be supplied as input parameters to the soft calibration model.
- An example set of constraints corresponding to upper bounds (u) and lower bounds (l) to be associated with a set of unit variables (x) and/or initial weights (x o ) may be expressed and/or defined as follows:
- respective ones of the initial weights associated with corresponding respective ones of the unit variables do not have different measurement scales and/or units. Accordingly, there is no need to convert differing scales and/or units of measurement to measureless comparable quantities in connection with computing weight loss.
- a size parameter (s) may be applied to respective ones of the unit variables.
- the size parameters may capture and/or represent preferences associated with respective ones of the unit variables.
- each size parameter (s) may be based on an average of non-zero contributions of a unit to the criteria.
- the size parameters have positive values.
- Example size parameters (s) per unit variable for computing weight loss may be expressed and/or defined as follows:
- the unit-based input data may be provided by, accessed from, and/or stored in a TSV file having multiple columns.
- a first column of a TSV file for the unit-based input data may include identifiers (e.g., a set of unique names and/or characters) for each unit.
- a second column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each initial weight.
- a third column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each lower bound.
- a fourth column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each upper bound.
- a fifth column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each size parameter.
- the respective values of the second column of the TSV file are greater than or equal to corresponding ones of the respective values of the third column of the TSV file, and less than or equal to corresponding ones of the respective values of the fourth column of the TSV file.
- the respective values of the fifth column of the TSV file are positive, and cannot be zero (0).
- the unit-based input data may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file.
- the unit-based input data may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- weight loss refers to a measured distance between (a) an evaluated weight captured by a unit variable, and (b) a corresponding initial weight.
- weight loss may be expressed and/or defined in terms of a weight loss function (G(x)).
- G(x) weight loss function
- the weight loss function (G(x)) may be expressed and/or defined as follows:
- the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on matrix data to be incorporated into the soft calibration model.
- a design matrix including the matrix data may be generated and/or derived from the criteria-based input data and the unit-based input data.
- the design matrix may capture and/or generate relationships between the above-described units and the above-described criteria. For example, the non-expanded contribution of each unit to each criterion may be captured by a design matrix having non-negative entries.
- An example design matrix may be expressed and/or defined as follows:
- the design matrix may be provided by, accessed from, and/or stored in a TSV file having multiple columns.
- a first column of a TSV file for the design matrix may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data.
- a second column of the TSV file for the design matrix may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the criteria-based input data.
- a third column of the TSV file for the design matrix may include values (e.g., double precision numerical values) for the contribution of respective ones of the units to respective ones of the criteria without weighting.
- the respective values of the third column of the TSV file for the design matrix are positive and not equal to zero (0).
- the data is assumed to be zero (0).
- the design matrix may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file.
- the design matrix may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on model parameter data to be incorporated into the soft calibration model.
- the model parameter data includes a Huber function parameter associated with target loss and a budget parameter associated with weight loss.
- a Huber loss function H ⁇ ( ⁇ ) may be implemented to measure the distance between (a) a panelist and/or reporting value captured by a criteria variable, and (b) a corresponding target.
- the Huber loss function is parametric in ⁇ . Identification of a Huber function parameter ( ⁇ ) applicable across all criteria enables a fixed target loss to be computed.
- Target loss and the weight loss are conflicting objectives of the soft calibration process. For example, the smaller that the value of the target loss becomes, the larger that the value of the weight loss becomes by necessity, and vice-versa.
- the target loss may be minimized and a constraint may be placed on the value that the weight loss is allowed to obtain.
- the term “budget” refers to a constraint placed on the value that the weight loss is allowed to obtain.
- a budget parameter ( ⁇ ) is an upper bound (e.g., ⁇ >0 ) associated with the weight loss.
- An example constraint corresponding to the budget parameter ( ⁇ ) may be expressed and/or defined as follows:
- an example calibration model to calibrate data using relaxed benchmark constraints may be expressed and/or defined as follows:
- the soft calibration model of Equation 17 may be implemented and/or executed to provide a unique solution whereby respective calibrated panelist and/or reporting values from among a set of calibrated panelist and/or reporting values are close in value to, but not necessarily equal to, respective target and/or benchmark values from among a corresponding set of target and/or benchmark values.
- Implementation and/or execution of the soft calibration model of Equation 17 also results in a determination and/or identification of the calibrated weights (e.g., optimized weights). As the initial weights (x o ) are known and fixed, the calibrated weights may be identified and/or determined based on the weight adjustments per unit
- calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a TSV file having multiple columns.
- a first column of a TSV file for the calibrated weights may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data.
- a second column of the TSV file for the calibrated weights may include values (e.g., double precision numerical values) for the weight adjustment (x adj ) of each unit or, alternatively, the calibrated weight of each unit.
- calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file.
- calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- Table 1 below provides an example of panelist and/or reporting data to be calibrated:
- the panelist and/or reporting data of Table 1 includes two demographic dimensions: (1) an age group dimension having three levels; and (2) a sex dimension having two levels. A population estimate is provided for each intersection of age and group dimensions.
- the panelist and/or reporting data of Table 1 also includes the average spending per person for two retailers (e.g., RetailerA and RetailerB).
- Criteria-based input data may be associated with the panelist and/or reporting data of Table 1 above.
- Table 2 below provides an example of criteria-based input data associated with the panelist and/or reporting data of Table 1 in a scenario where the scaling parameters (c) described above in connection with Equation 5 have been set to equal the corresponding targets (t), and where the importance parameters (v) described above in connection with Equation 6 have been set to a value of 1.00 for the demographic targets:
- Unit-based input data may also be associated with the panelist and/or reporting data of Table 1 above.
- Table 3 below provides an example of unit-based input data associated with the panelist and/or reporting data of Table 1 in a scenario where the initial weights (x 0 ) are set to equal the population estimates of the panelist and/or reporting data at intersection level:
- Table 4 provides an example of a design matrix including matrix data associated with the criteria-based input data of Table 2 above and the unit-based input data of Table 3 above:
- Table 5 below provides an example of calibrated weights data resulting from executing the soft calibration model of Equation 17 based in part on the criteria-based input data of Table 2, the unit-based input data of Table 3, and the matrix data of Table 4, in a scenario where the budget parameter ( ⁇ ) has been set to a value of 0.35 and the Huber function parameter ( ⁇ ) has been set to a value of 0.50:
- the target loss function (F(y)) has a flat region (D) (e.g., a flat region (D) expressed and/or defined as ⁇ D ⁇ ⁇ 0 m : y 1 ,y 2 ⁇ D ⁇
- D flat region
- ⁇ the budget parameter
- the soft calibration model of Equation 17 does not specify which one of the countless possibilities is to be preferred.
- Different solvers might choose to implement heuristics picking up one of the edges of the optimal set or, alternatively, a central point of this set.
- the soft calibration model of Equation 17 does not capture a preference for a solution that minimizes the weight loss.
- D nearly flat region
- implementation heuristics become unreliable, and may result in a progressively increasing weight loss function (G(x)) that provides no meaningful improvement in relation to the target attainment.
- execution of the soft calibration model may result in calibrated weights that do not provide a unique solution for the executed soft calibration model.
- a stability parameter ( ⁇ ) may be incorporated into (e.g., appended to) the soft calibration model of Equation 17 to remedy and/or prevent unexpected performance of the soft calibration model resulting from the above-described circumstances and/or conditions.
- the stability parameter ( ⁇ ) reduces the influence of the budget parameter ( ⁇ ) on the soft calibration model.
- incorporation of the stability parameter ( ⁇ ) may prevent the minimization procedure of the soft calibration model of Equation 17 from reaching the flat region (D) of the target loss function (F(y)).
- the stability parameter ( ⁇ ) is included as part of the model parameter data described above.
- the stability parameter ( ⁇ ) may be incorporated into the soft calibration model of Equation 17 by adding the product of the stability parameter ( ⁇ ) and the weight loss function (G(x)) to the target loss function (F(y)).
- the revised soft calibration model may be expressed and/or defined as follows:
- the stability parameter ( ⁇ ) may be determined and/or provided at run-time.
- the stability parameter ( ⁇ ) may preferably have a value between 0.00001 and 0.01 (e.g., comparable to a precision sensitivity associated with the soft calibration model).
- the value of the stability parameter ( ⁇ ) may be increased until the calibrated weights that result from executing the soft calibration model provide a unique solution for the soft calibration model.
- implementation and/or incorporation of the stability parameter ( ⁇ ) may reduce the occurrence of errors (e.g., the generation of calibrated weights providing non-unique solutions) attributable to and/or arising from an influence of the value of the budget parameter ( ⁇ ) on the soft calibration model.
- the soft calibration model of Equation 18 may be implemented and/or executed to provide a unique solution whereby respective calibrated panelist and/or reporting values from among a set of calibrated panelist and/or reporting values are close in value to, but not necessarily equal to, respective target and/or benchmark values from among a corresponding set of target and/or benchmark values.
- Implementation and/or execution of the soft calibration model of Equation 18 also results in a determination and/or identification of the calibrated weights (e.g., optimized weights). As the initial weights (x 0 ) are known and fixed, the calibrated weights may be identified and/or determined based on the weight adjustments per unit
- calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a TSV file having multiple columns.
- a first column of a TSV file for the calibrated weights may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data.
- a second column of the TSV file for the calibrated weights may include values (e.g., double precision numerical values) for the weight adjustment of each unit or, alternatively, the calibrated weight of each unit.
- calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file.
- calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- FIG. 1 is a block diagram of an example calibration apparatus 100 constructed in accordance with the teachings of this disclosure.
- the calibration apparatus 100 of FIG. 1 may be implemented to calibrate panelist and/or reporting data using relaxed benchmark constraints incorporated into a soft calibration model (e.g., the example soft calibration model of Equation 17 or the example soft calibration model of Equation 18 described above).
- the calibration apparatus 100 includes an example criteria-based input determiner 102 , an example unit-based input determiner 104 , an example matrix determiner 106 , an example model parameter determiner 108 , an example calibration engine 110 , an example user interface 112 , and an example memory 114 .
- the example criteria-based input determiner 102 of FIG. 1 determines and/or identifies example criteria-based input data 118 to be incorporated into an example target loss function 120 of a soft calibration model to be executed by the example calibration engine 110 of the calibration apparatus 100 of FIG. 1 .
- the criteria-based input data 118 determined and/or identified by the criteria-based input determiner 102 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the criteria-based input determiner 102 of FIG. 1 determines and/or identifies the criteria-based input data 118 by accessing and/or obtaining the criteria-based input data 118 from the memory 114 of FIG. 1 .
- the criteria-based input determiner 102 of FIG. 1 determines and/or identifies criteria-based input data 118 received from the example user interface 112 of FIG. 1 described below.
- the criteria-based input determiner 102 of FIG. 1 may cause the memory 114 of FIG. 1 to store the received criteria-based input data 118 .
- the criteria-based input data 118 to be determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may include criteria associated with example panelist and/or reporting data (e.g., survey data) 116 , criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables.
- the criteria of the criteria-based input data 118 may be expressed in a manner consistent with Equation 1 described above.
- the criteria variables of the criteria-based input data 118 may be expressed in a manner consistent with Equation 2 described above.
- the targets of the criteria-based input data 118 may be expressed in a manner consistent with Equation 3 described above.
- the lower and upper bounds of the criteria-based input data 118 may be expressed in a manner consistent with Equation 4 described above.
- the scaling parameters of the criteria-based input data 118 may be expressed in a manner consistent with Equation 5 described above.
- the importance parameters of the criteria-based input data 118 may be expressed in a manner consistent with Equation 6 described above.
- the target loss function 120 into which the criteria-based input data 118 determined and/or identified by the criteria-based input determiner 102 of FIG. 1 is to be incorporated may be expressed in a manner consistent with Equation 7 described above.
- the example unit-based input determiner 104 of FIG. 1 determines and/or identifies example unit-based input data 122 to be incorporated into an example weight loss function 124 of the soft calibration model to be executed by the example calibration engine 110 of the calibration apparatus 100 of FIG. 1 .
- the unit-based input data 122 determined and/or identified by the unit-based input determiner 104 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the unit-based input determiner 104 of FIG. 1 determines and/or identifies the unit-based input data 122 by accessing and/or obtaining the unit-based input data 122 from the memory 114 of FIG. 1 .
- the unit-based input determiner 104 of FIG. 1 determines and/or identifies unit-based input data 122 received from the example user interface 112 of FIG. 1 described below.
- the unit-based input determiner 104 of FIG. 1 may cause the memory 114 of FIG. 1 to store the received unit-based input data 122 .
- the unit-based input data 122 to be determined and/or identified by the unit-based input determiner 104 of FIG. 1 may include units associated with the panelist and/or reporting data 116 , unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables.
- the units of the unit-based input data 122 may be expressed in a manner consistent with Equation 8 described above.
- the unit variables of the unit-based input data 122 may be expressed in a manner consistent with Equation 9 described above.
- the initial weights of the unit-based input data 122 may be expressed in a manner consistent with Equation 10 described above.
- the lower and upper bounds of the unit-based input data 122 may be expressed in a manner consistent with Equation 11 described above.
- the size parameters of the unit-based input data 122 may be expressed in a manner consistent with Equation 12 described above.
- the weight loss function 124 into which the unit-based input data 122 determined and/or identified by the unit-based input determiner 104 of FIG. 1 is to be incorporated may be expressed in a manner consistent with Equation 13 described above.
- the example matrix determiner 106 of FIG. 1 determines, generates, derives and/or identifies example matrix data 126 to be incorporated into the soft calibration model to be executed by the example calibration engine 110 of the calibration apparatus 100 of FIG. 1 .
- the matrix data 126 determined, generated, derived and/or identified by the matrix determiner 106 of FIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the matrix determiner 106 of FIG. 1 determines and/or identifies the matrix data 126 by accessing and/or obtaining the matrix data 126 from the memory 114 of FIG.
- the matrix determiner 106 of FIG. 1 determines and/or identifies matrix data 126 received from the example user interface 112 of FIG. 1 described below. In such other examples, the matrix determiner 106 of FIG. 1 may cause the memory 114 of FIG. 1 to store the received matrix data 126 .
- the matrix determiner 106 of FIG. 1 determines, generates, derives and/or identifies the matrix data 126 based on the example criteria-based input data 118 of FIG. 1 and the example unit-based input data 122 of FIG. 1 .
- the matrix data 126 determined, generated derived and/or identified by the matrix determiner 106 of FIG. 1 may be expressed in a manner consistent with Equation 14 and/or Equation 15 described above.
- the matrix data 126 determined, generated, derived and/or identified by the matrix determiner 106 may capture relationships between the criteria of the criteria-based input data 118 and the units of the unit-based input data 122 . For example, the non-expanded contribution of each unit to each criterion may be captured by the matrix data 126 .
- the example model parameter determiner 108 of FIG. 1 determines and/or identifies example model parameter data 128 to be incorporated into the soft calibration model to be executed by the example calibration engine 110 of the calibration apparatus 100 of FIG. 1 .
- the model parameter data 128 determined and/or identified by the model parameter determiner 108 of FIG. 1 may be of any type, form and/or format, and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the model parameter determiner 108 of FIG. 1 determines and/or identifies the model parameter data 128 by accessing and/or obtaining the model parameter data 128 from the memory 114 of FIG. 1 .
- model parameter determiner 108 of FIG. 1 may cause the memory 114 of FIG. 1 to store the received model parameter data 128 .
- the model parameter data 128 to be determined and/or identified by the model parameter determiner 108 of FIG. 1 may include a Huber function parameter, a budget parameter, and/or a stability parameter.
- the budget parameter of the model parameter data 128 determined and/or identified by the model parameter determiner 108 of FIG. 1 may be expressed in a manner consistent with Equation 16 described above.
- the example calibration engine 110 of FIG. 1 builds, constructs and/or compiles the soft calibration model to be executed by the calibration apparatus 100 .
- the calibration engine 110 of FIG. 1 builds, constructs and/or compiles the soft calibration model based on the criteria-based input data 118 , the target loss function 120 , the unit-based input data 122 , the weight loss function 124 , the matrix data 126 , and the model parameter data 128 of FIG. 1 .
- the soft calibration model built, constructed and/or compiled by the calibration engine 110 of FIG. 1 may be expressed in a manner consistent with Equation 17 described above.
- the soft calibration model built, constructed and/or compiled by the calibration engine 110 of FIG. 1 may be expressed in a manner consistent with Equation 18 described above.
- the calibration engine 110 of FIG. 1 executes and/or controls the execution of the built, constructed and/or compiled soft calibration model.
- the calibration engine 110 includes an example calibrated weights determiner 130 .
- the calibrated weights determiner 130 of FIG. 1 determines and/or calculates example calibrated weights data 132 in response to execution of the soft calibration model by the calibration engine 110 .
- the calibrated weights determiner 130 of FIG. 1 determines the calibrated weights data 132 based on the criteria-based input data 118 , the target loss function 120 , the unit-based input data 122 , the weight loss function 124 , the matrix data 126 , and the model parameter data 128 incorporated into the executed soft calibration model.
- the calibrated weights data 132 determined and/or calculated by the calibrated weights determiner 130 of FIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the calibration engine 110 also includes an example calibrated reporting data determiner 134 .
- the example calibrated reporting data determiner 134 of FIG. 1 determines and/or calculates example calibrated panelist and/or reporting data 136 .
- the calibrated panelist and/or reporting data 136 determined and/or calculated by the calibrated reporting data determiner 134 of FIG. 1 is based on initial panelist and/or reporting data (e.g., the panelist and/or reporting data 116 of FIG. 1 ) and the calibrated weights data 132 determined and/or calculated by the calibrated weights determiner 130 of FIG. 1 .
- the calibrated panelist and/or reporting data 136 may determine and/or calculate the calibrated panelist and/or reporting data 136 by applying the calibrated weights data 132 to the panelist and/or reporting data 116 of FIG. 1 .
- the calibrated reporting data determiner 134 of FIG. 1 may cause the calibrated panelist and/or reporting data 136 to be presented (e.g., displayed) via the example user interface 112 of FIG. 1 .
- the calibrated panelist and/or reporting data 136 determined and/or calculated by the calibrated reporting data determiner 134 of FIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the calibration engine 110 also includes an example calibration model validator 138 .
- the example calibrated reporting data determiner 134 of FIG. 1 validates the soft calibration model executed by the example calibration engine 110 of FIG. 1 .
- the calibration model validator 138 of FIG. 1 may validate the soft calibration model executed by the calibration engine 110 of FIG. 1 by determining whether the calibrated weights resulting from execution of the soft calibration model (e.g., the calibrated weights data 132 determined by the calibrated weights determiner 130 of FIG. 1 ) provide a unique solution for the executed soft calibration model.
- the calibration model validator 138 may determine whether the executed soft calibration model included a stability parameter. For example, the calibration model validator 138 of FIG. 1 may determine that a soft calibration model expressed in a manner consistent with Equation 17 described above does not include a stability parameter. If the calibration model validator 138 of FIG. 1 determines that the executed soft calibration model does not include a stability parameter, the calibration model validator 138 may incorporate a stability parameter into the soft calibration model, and may further cause the calibration engine 110 of FIG.
- the calibration model validator 138 of FIG. 1 may incorporate a stability parameter expressed in a manner consistent with Equation 16 described above into the soft calibration model expressed in a manner consistent with Equation 17 described above to provide the soft calibration model expressed in a manner consistent with Equation 18 described above (e.g., a soft calibration model including a stability parameter).
- the example user interface 112 of FIG. 1 facilitates interactions and/or communications between an end user and the criteria-based input determiner 102 , the unit-based input determiner 104 , the matrix determiner 106 , the model parameter determiner 108 , the calibration engine 110 , the memory 114 , the calibrated weights determiner 130 , the calibrated reporting data determiner 134 , the calibration model validator 138 , and/or, more generally, the calibration apparatus 100 of FIG. 1 .
- the calibration apparatus 100 includes one or more input device(s) 140 via which the user may input information and/or data to the criteria-based input determiner 102 , the unit-based input determiner 104 , the matrix determiner 106 , the model parameter determiner 108 , the calibration engine 110 , the memory 114 , the calibrated weights determiner 130 , the calibrated reporting data determiner 134 , the calibration model validator 138 , and/or, more generally, the calibration apparatus 100 .
- the input device(s) 140 via which the user may input information and/or data to the criteria-based input determiner 102 , the unit-based input determiner 104 , the matrix determiner 106 , the model parameter determiner 108 , the calibration engine 110 , the memory 114 , the calibrated weights determiner 130 , the calibrated reporting data determiner 134 , the calibration model validator 138 , and/or, more generally, the calibration apparatus 100 .
- the one or more input device(s) 140 of the user interface 112 may include a button, a switch, a keyboard, a mouse, a microphone, and/or a touchscreen that enable(s) the user to convey data and/or commands to the criteria-based input determiner 102 , the unit-based input determiner 104 , the matrix determiner 106 , the model parameter determiner 108 , the calibration engine 110 , the memory 114 , the calibrated weights determiner 130 , the calibrated reporting data determiner 134 , the calibration model validator 138 , and/or, more generally, the calibration apparatus 100 of FIG. 1 .
- the user interface 112 of FIG. 1 also includes one or more output device(s) 142 via which the user interface 112 presents information and/or data in visual and/or audible form to the user.
- the one or more output device(s) 142 of the user interface 112 may include a light emitting diode, a touchscreen, and/or a liquid crystal display for presenting visual information, and/or a speaker for presenting audible information.
- the one or more output device(s) 142 of the user interface 112 may present and/or display the example calibrated weights data 132 and/or the example calibrated panelist and/or reporting data 136 resulting from execution of the soft calibration model by the example calibration engine 110 of FIG. 1 .
- the one or more output device(s) 142 of the user interface 112 may be directed to present and/or display the example calibrated weights data 132 and/or the example calibrated panelist and/or reporting data 136 by one or more of the calibration engine 110 , the calibrated weights determiner 130 , the calibrated reporting data determiner 134 , the calibration model validator 138 , and/or, more generally, the calibration apparatus 100 of FIG. 1 .
- Data and/or information that is presented and/or received via the user interface 112 of FIG. 1 may be of any type, form and/or format, and may be stored in a computer-readable storage medium such as the example memory 114 of FIG. 1 described below.
- the example memory 114 of FIG. 1 may be implemented by any type(s) and/or any number(s) of storage device(s) such as a storage drive, a flash memory, a read-only memory (ROM), a random-access memory (RAM), a cache and/or any other physical storage medium in which information is stored for any duration (e.g., for extended time periods, permanently, brief instances, for temporarily buffering, and/or for caching of the information).
- the data and/or information stored in the memory 114 may be stored in any file and/or data structure format, organization scheme, and/or arrangement.
- the memory 114 is accessible to one or more of the example criteria-based input determiner 102 , the example unit-based input determiner 104 , the example matrix determiner 106 , the example model parameter determiner 108 , the example calibration engine 110 , the example user interface 112 , the example calibrated weights determiner 130 , the example calibrated reporting data determiner 134 , and/or the example calibration model validator 138 of FIG. 1 , and/or, more generally, to the calibration apparatus of FIG. 1 .
- the memory 114 of FIG. 1 stores data and/or information received via the one or more input device(s) 140 of the user interface 112 of FIG. 1 . In some examples, the memory 114 stores data and/or information to be presented via the one or more output device(s) 142 of the user interface 112 of FIG. 1 . In some examples, the memory 114 stores panelist and/or reporting data 116 . In some examples, the memory 114 stores criteria-based input data 118 including criteria associated with the panelist and/or reporting data 116 , criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables.
- the memory 114 stores a target loss function 120 to be incorporated into the soft calibration model to be executed by the calibration engine 110 of FIG. 1 .
- the memory 114 stores unit-based input data 122 including units associated with the panelist and/or reporting data 116 , unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables.
- the memory 114 stores a weight loss function 124 to be incorporated into the soft calibration model to be executed by the calibration engine 110 of FIG. 1 .
- the memory 114 stores matrix data 126 .
- the memory 114 stores model parameter data 128 including a Huber function parameter, a budget parameter, and/or a stability parameter.
- the memory 114 stores calibrated weights data 132 determined and/or calculated by the calibrated weights determiner 130 and/or, more generally, by the calibration engine 110 of FIG. 1 .
- the memory 114 stores calibrated panelist and/or reporting data 136 determined and/or calculated by the calibrated reporting data determiner 134 and/or, more generally, by the calibration engine 110 of FIG. 1 .
- While an example manner of implementing a calibration apparatus 100 is illustrated in FIG. 1 , one or more of the elements, processes and/or devices illustrated in FIG. 1 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way.
- the example criteria-based input determiner 102 , the example unit-based input determiner 104 , the example matrix determiner 106 , the example model parameter determiner 108 , the example calibration engine 110 , the example user interface 112 , the example memory 114 , the example calibrated weights determiner 130 , the example calibrated reporting data determiner 134 , and/or the example calibration model validator 138 of FIG. 1 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware.
- any of the example criteria-based input determiner 102 , the example unit-based input determiner 104 , the example matrix determiner 106 , the example model parameter determiner 108 , the example calibration engine 110 , the example user interface 112 , the example memory 114 , the example calibrated weights determiner 130 , the example calibrated reporting data determiner 134 , and/or the example calibration model validator 138 of FIG. 1 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)).
- ASIC application specific integrated circuit
- PLD programmable logic device
- FPLD field programmable logic device
- At least one of the example criteria-based input determiner 102 , the example unit-based input determiner 104 , the example matrix determiner 106 , the example model parameter determiner 108 , the example calibration engine 110 , the example user interface 112 , the example memory 114 , the example calibrated weights determiner 130 , the example calibrated reporting data determiner 134 , and/or the example calibration model validator 138 of FIG. 1 is/are hereby expressly defined to include a tangible computer-readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc.
- DVD digital versatile disk
- CD compact disk
- Blu-ray disk etc.
- example calibration apparatus 100 of FIG. 1 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIG. 1 , and/or may include more than one of any or all of the illustrated elements, processes and devices.
- FIGS. 2-6 Flowcharts representative of example machine readable instructions which may be executed to calibrate data using relaxed benchmark constraints are shown in FIGS. 2-6 .
- the machine-readable instructions may implement one or more program(s) for execution by a processor such as the example processor 702 shown in the example processor platform 700 discussed below in connection with FIG. 7 .
- the one or more program(s) may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with the processor 702 of FIG. 7 , but the entire program(s) and/or parts thereof could alternatively be executed by a device other than the processor 702 of FIG.
- example program(s) is/are described with reference to the flowcharts illustrated in FIGS. 2-6 , many other methods for calibrating data using relaxed benchmark constrains man alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.
- the example instructions of FIGS. 2-6 may be stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
- a tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.
- tangible computer readable storage medium and “tangible machine readable storage medium” are used interchangeably. Additionally or alternatively, the example instructions of FIGS. 2-6 may be stored on a non-transitory computer and/or machine-readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
- a non-transitory computer and/or machine-readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
- non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.
- phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended.
- FIG. 2 is a flowchart representative of example machine readable instructions 200 that may be executed at the example calibration apparatus 100 of FIG. 1 to calibrate data using relaxed benchmark constraints.
- the example program 200 of FIG. 2 may be implemented to calibrate panelist and/or reporting data using relaxed benchmark constraints incorporated into a soft calibration model (e.g., the example soft calibration model of Equation 17 or the example soft calibration model of Equation 18 described above).
- the example program 200 of FIG. 2 begins when the example criteria-based input determiner 102 of FIG. 1 determines criteria-based input data for a target loss function of the soft calibration model (block 202 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify the example criteria-based input data 118 of FIG. 1 by accessing and/or obtaining the criteria-based input data 118 from the example memory 114 of FIG. 1 , and/or by receiving the criteria-based input data 118 from the example user interface 112 of FIG. 1 .
- the criteria-based input data 118 determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may include criteria associated with the example panelist and/or reporting data 116 of FIG.
- the example unit-based input determiner 104 of FIG. 1 determines unit-based input data for a weight loss function of the soft calibration model (block 204 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify the example unit-based input data 122 of FIG. 1 by accessing and/or obtaining the unit-based input data 122 from the example memory 114 of FIG. 1 , and/or by receiving the unit-based input data 122 from the example user interface 112 of FIG. 1 .
- the unit-based input data 122 determined and/or identified by the unit-based input determiner 104 of FIG. 1 may include units associated with the example panelist and/or reporting data 116 of FIG.
- unit variables for the units initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables.
- An example process that may be used to implement block 204 is further described below in connection with FIG. 4 . Following block 204 , control of the example program 200 of FIG. 2 proceeds to block 206 .
- the example matrix determiner 106 of FIG. 1 determines matrix data for the soft calibration model (block 206 ).
- the matrix determiner 106 of FIG. 1 may determine and/or identify the example matrix data 126 of FIG. 1 by accessing and/or obtaining the matrix data 126 from the example memory 114 of FIG. 1 , and/or by receiving the matrix data 126 from the example user interface 112 of FIG. 1 .
- the matrix determiner 106 of FIG. 1 determines and/or identifies the matrix data 126 based on the example criteria-based input data 118 of FIG. 1 and the example unit-based input data 122 of FIG. 1 .
- the matrix data 126 determined and/or identified by the matrix determiner 106 of FIG. 1 may be expressed in a manner consistent with Equation 14 and/or Equation 15 described above.
- control of the example program 200 of FIG. 2 proceeds to block 208 .
- the example model parameter determiner 108 of FIG. 1 determines model parameter data for the soft calibration model (block 208 ).
- the model parameter determiner 108 of FIG. 1 may determine and/or identify the example model parameter data 128 of FIG. 1 by accessing and/or obtaining the model parameter data 128 from the example memory 114 of FIG. 1 , and/or by receiving the model parameter data 128 from the example user interface 112 of FIG. 1 .
- the model parameter data 128 to be determined and/or identified by the model parameter determiner 108 of FIG. 1 may include a Huber function parameter, a budget parameter, and/or a stability parameter.
- An example process that may be used to implement block 208 is further described below in connection with FIG. 5 .
- control of the example program 200 of FIG. 2 proceeds to block 210 .
- the example calibration engine 110 of FIG. 1 executes the soft calibration model to determine calibrated weights based on the criteria-based input data, the target loss function, the unit-based input data, the weight loss function, the design matrix, and the model parameters (block 210 ).
- the calibration engine 110 of FIG. 1 may execute the soft calibration model based on the example criteria-based input data 118 , the example target loss function 120 , the example unit-based input data 122 , the example weight loss function 124 , the example matrix data 126 , and the example model parameter data 128 of FIG. 1 .
- the calibrated weights determiner 130 of FIG. 1 may determine and/or calculate the calibrated weights data 132 based on adjusted calibration weights output by the soft calibration model.
- the model parameter data 128 incorporated into the soft calibration model to be executed at block 210 includes a stability parameter. In other examples, the model parameter data 128 incorporated into the soft calibration model to be executed at block 210 may lack a stability parameter.
- An example process that may be used to implement block 210 when the model parameter data 128 incorporated into the soft calibration model may lack a stability parameter is further described below in connection with FIG. 6 . Following block 210 , control of the example program 200 of FIG. 2 proceeds to block 212 .
- the example calibrated reporting data determiner 134 of FIG. 1 determines calibrated panelist and/or reporting data based on initial panelist and/or reporting data and the calibrated weights (block 212 ).
- the calibrated reporting data determiner 134 of FIG. 1 may determine and/or calculate the example calibrated panelist and/or reporting data 136 of FIG. 1 by applying the example calibrated weights data 132 of FIG. 1 determined and/or calculated by the calibrated weights determiner 130 of FIG. 1 to the example panelist and/or reporting data 116 of FIG. 1 .
- control of the example program 200 of FIG. 2 proceeds to block 214 .
- the example user interface 112 of FIG. 1 presents the calibrated weights, and/or presents the calibrated panelist and/or reporting data, for display (block 214 ).
- the user interface 112 of FIG. 1 may present (e.g., display) the example calibrated weights data 132 of FIG. 1 determined and/or calculated by the calibrated weights determiner 130 of FIG. 1 .
- the user interface 112 of FIG. 1 may additionally and/or alternatively present (e.g., display) the example calibrated panelist and/or reporting data 136 of FIG. 1 determined and/or calculated by the calibrated reporting data determiner 134 of FIG. 1 .
- the example program 200 of FIG. 2 ends.
- FIG. 3 is a flowchart representative of example machine readable instructions 202 that may be executed at the example calibration apparatus 100 of FIG. 1 to determine criteria-based input data for the target loss function of the soft calibration model to be executed by the calibration apparatus 100 .
- Example operations of blocks 302 , 304 , 306 , 308 , 310 , 312 and 314 of FIG. 3 may be used to implement block 202 of FIG. 2 .
- the example program 202 of FIG. 3 begins when the example criteria-based input determiner 102 of FIG. 1 determines criteria associated with panelist and/or reporting data (block 302 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify criteria associated the example panelist and/or reporting data 116 of FIG. 1 by accessing and/or obtaining the criteria from the example memory 114 of FIG. 1 , and/or by receiving the criteria from the example user interface 112 of FIG. 1 .
- the criteria determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 1 described above.
- control of the example program 202 of FIG. 3 proceeds to block 304 .
- the example criteria-based input determiner 102 of FIG. 1 determines criteria variables for the criteria (block 304 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify criteria variables for the criteria by accessing and/or obtaining the criteria variables from the example memory 114 of FIG. 1 , and/or by receiving the criteria variables from the example user interface 112 of FIG. 1 .
- the criteria variables determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 2 described above.
- control of the example program 202 of FIG. 3 proceeds to block 306 .
- the example criteria-based input determiner 102 of FIG. 1 determines targets for the criteria variables (block 306 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify targets for the criteria variables by accessing and/or obtaining the targets from the example memory 114 of FIG. 1 , and/or by receiving the targets from the example user interface 112 of FIG. 1 .
- the targets determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 3 described above.
- control of the example program 202 of FIG. 3 proceeds to block 308 .
- the example criteria-based input determiner 102 of FIG. 1 determines lower bounds for the criteria variables (block 308 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify lower bounds for the criteria variables by accessing and/or obtaining the lower bounds from the example memory 114 of FIG. 1 , and/or by receiving the lower bounds from the example user interface 112 of FIG. 1 .
- the lower bounds determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 4 described above.
- control of the example program 202 of FIG. 3 proceeds to block 310 .
- the example criteria-based input determiner 102 of FIG. 1 determines upper bounds for the criteria variables (block 310 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify upper bounds for the criteria variables by accessing and/or obtaining the upper bounds from the example memory 114 of FIG. 1 , and/or by receiving the upper bounds from the example user interface 112 of FIG. 1 .
- the upper bounds determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 4 described above.
- control of the example program 202 of FIG. 3 proceeds to block 312 .
- the example criteria-based input determiner 102 of FIG. 1 determines scaling parameters for the criteria variables (block 312 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify scaling parameters for the criteria variables by accessing and/or obtaining the scaling parameters from the example memory 114 of FIG. 1 , and/or by receiving the scaling parameters from the example user interface 112 of FIG. 1 .
- the scaling parameters determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 5 described above.
- control of the example program 202 of FIG. 3 proceeds to block 314 .
- the example criteria-based input determiner 102 of FIG. 1 determines importance parameters for the criteria variables (block 314 ).
- the criteria-based input determiner 102 of FIG. 1 may determine and/or identify importance parameters for the criteria variables by accessing and/or obtaining the importance parameters from the example memory 114 of FIG. 1 , and/or by receiving the importance parameters from the example user interface 112 of FIG. 1 .
- the importance parameters determined and/or identified by the criteria-based input determiner 102 of FIG. 1 may be expressed in a manner consistent with Equation 6 described above.
- the example program 202 of FIG. 3 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- FIG. 4 is a flowchart representative of example machine readable instructions 204 that may be executed at the example calibration apparatus 100 of FIG. 1 to determine unit-based input data for the weight loss function of the soft calibration model to be executed by the calibration apparatus 100 .
- Example operations of blocks 402 , 404 , 406 , 408 , 410 and 412 of FIG. 4 may be used to implement block 204 of FIG. 2 .
- the example program 204 of FIG. 4 begins when the example unit-based input determiner 104 of FIG. 1 determines units associated with panelist and/or reporting data (block 402 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify units associated the example panelist and/or reporting data 116 of FIG. 1 by accessing and/or obtaining the units from the example memory 114 of FIG. 1 , and/or by receiving the units from the example user interface 112 of FIG. 1 .
- the units determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 8 described above.
- control of the example program 204 of FIG. 4 proceeds to block 404 .
- the example unit-based input determiner 104 of FIG. 1 determines unit variables for the units (block 404 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify unit variables for the units by accessing and/or obtaining the unit variables from the example memory 114 of FIG. 1 , and/or by receiving the unit variables from the example user interface 112 of FIG. 1 .
- the unit variables determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 9 described above.
- control of the example program 204 of FIG. 4 proceeds to block 406 .
- the example unit-based input determiner 104 of FIG. 1 determines initial weights for the unit variables (block 406 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify initial weights for the unit variables by accessing and/or obtaining the initial weights from the example memory 114 of FIG. 1 , and/or by receiving the initial weights from the example user interface 112 of FIG. 1 .
- the initial weights determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 10 described above.
- control of the example program 204 of FIG. 4 proceeds to block 408 .
- the example unit-based input determiner 104 of FIG. 1 determines lower bounds for the unit variables (block 408 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify lower bounds for the unit variables by accessing and/or obtaining the lower bounds from the example memory 114 of FIG. 1 , and/or by receiving the lower bounds from the example user interface 112 of FIG. 1 .
- the lower bounds determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 11 described above.
- control of the example program 204 of FIG. 4 proceeds to block 410 .
- the example unit-based input determiner 104 of FIG. 1 determines upper bounds for the unit variables (block 410 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify upper bounds for the unit variables by accessing and/or obtaining the upper bounds from the example memory 114 of FIG. 1 , and/or by receiving the upper bounds from the example user interface 112 of FIG. 1 .
- the upper bounds determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 11 described above.
- control of the example program 204 of FIG. 4 proceeds to block 412 .
- the example unit-based input determiner 104 of FIG. 1 determines size parameters for the unit variables (block 412 ).
- the unit-based input determiner 104 of FIG. 1 may determine and/or identify size parameters for the unit variables by accessing and/or obtaining the size parameters from the example memory 114 of FIG. 1 , and/or by receiving the size parameters from the example user interface 112 of FIG. 1 .
- the size parameters determined and/or identified by the unit-based input determiner 104 of FIG. 1 may be expressed in a manner consistent with Equation 12 described above.
- the example program 204 of FIG. 4 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- FIG. 5 is a flowchart representative of example machine readable instructions 208 that may be executed at the example calibration apparatus 100 of FIG. 1 to determine model parameters for the soft calibration model to be executed by the calibration apparatus 100 .
- Example operations of blocks 502 , 504 and 506 of FIG. 5 may be used to implement block 208 of FIG. 2 .
- the example program 208 of FIG. 5 begins when the example model parameter determiner 108 of FIG. 1 determines a Huber function parameter for the soft calibration model (block 502 ).
- the model parameter determiner 108 of FIG. 1 may determine and/or identify a Huber function parameter for the soft calibration model by accessing and/or obtaining the Huber function parameter from the example memory 114 of FIG. 1 , and/or by receiving the Huber function parameter from the example user interface 112 of FIG. 1 .
- control of the example program 208 of FIG. 5 proceeds to block 504 .
- the example model parameter determiner 108 of FIG. 1 determines a budget parameter for the soft calibration model (block 504 ).
- the model parameter determiner 108 of FIG. 1 may determine and/or identify a budget parameter for the soft calibration model by accessing and/or obtaining the budget parameter from the example memory 114 of FIG. 1 , and/or by receiving the budget parameter from the example user interface 112 of FIG. 1 .
- control of the example program 208 of FIG. 5 proceeds to block 506 .
- the example model parameter determiner 108 of FIG. 1 determines a stability parameter for the soft calibration model (block 506 ).
- the model parameter determiner 108 of FIG. 1 may determine and/or identify a stability parameter for the soft calibration model by accessing and/or obtaining the stability parameter from the example memory 114 of FIG. 1 , and/or by receiving the stability parameter from the example user interface 112 of FIG. 1 .
- the example program 208 of FIG. 5 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- FIG. 6 is a flowchart representative of example machine readable instructions 210 that may be executed at the example calibration apparatus 100 of FIG. 1 to validate the soft calibration model executed by the calibration apparatus.
- Example operations of blocks 602 , 604 , 606 , 608 , 610 , 612 , 614 and 616 of FIG. 6 may be used to implement block 210 of FIG. 2 in instances when the model parameter data 128 incorporated into the soft calibration model may lack a stability parameter.
- the example program 210 of FIG. 6 begins when the example calibration model validator 138 of FIG. 1 determines whether the soft calibration model executed by the example calibration engine 110 of FIG. 1 includes a stability parameter (block 602 ). For example, the calibration model validator 138 of FIG. 1 may determine that a soft calibration model expressed in a manner consistent with Equation 17 described above does not include a stability parameter. If the calibration model validator 138 determines at block 602 that the soft calibration model executed by the calibration engine 110 of FIG. 1 does not include a stability parameter, control of the example program 210 of FIG. 6 proceeds to block 604 . If the calibration model validator 138 instead determines at block 602 that the soft calibration model executed by the calibration engine 110 of FIG. 1 does include a stability parameter, the example program 210 of FIG. 6 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- a stability parameter block 602 .
- the example calibration model validator 138 of FIG. 1 determines whether the calibrated weights resulting from the soft calibration model being executed by the example calibration engine 110 of FIG. 1 provide a unique solution for the executed soft calibration model (block 604 ). If the calibration model validator 138 determines at block 604 that the calibrated weights resulting from the soft calibration model being executed by the example calibration engine 110 of FIG. 1 do not provide a unique solution for the executed soft calibration model, control of the example program 210 of FIG. 6 proceeds to block 606 . If the calibration model validator 138 instead determines at block 604 that the calibrated weights resulting from the soft calibration model being executed by the example calibration engine 110 of FIG. 1 do provide a unique solution for the executed soft calibration model, the example program 210 of FIG. 6 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- the example calibration model validator 138 of FIG. 1 incorporates a stability parameter into the soft calibration model (block 606 ).
- the calibration model validator 138 of FIG. 1 may incorporate a stability parameter expressed in a manner consistent with Equation 16 described above into the soft calibration model expressed in a manner consistent with Equation 17 described above to provide the soft calibration model expressed in a manner consistent with Equation 18 described above (e.g., a soft calibration model including a stability parameter).
- control of the example program 210 of FIG. 6 proceeds to block 608 .
- the example calibration engine 110 of FIG. 1 re-executes the soft calibration model to re-determine the calibrated weights (block 608 ).
- the calibration engine 110 of FIG. 1 may re-execute the soft calibration model based on the example criteria-based input data 118 , the example target loss function 120 , the example unit-based input data 122 , the example weight loss function 124 , the example matrix data 126 , and the example model parameter data 128 of FIG. 1 , including the stability parameter.
- the calibrated weights determiner 130 of FIG. 1 determines the example calibrated weights data 132 of FIG.
- the calibrated weights determined (e.g., re-determined) at block 608 provide a unique solution for the re-executed soft calibration model.
- the example program 210 of FIG. 6 ends and control returns to a calling function or process such as the example program 200 of FIG. 2 .
- FIG. 7 is an example processor platform 700 capable of executing the instructions of FIGS. 2-6 to implement the example calibration apparatus 100 of FIG. 1 .
- the processor platform 700 of the illustrated example includes a processor 702 .
- the processor 702 of the illustrated example is hardware.
- the processor 702 can be implemented by one or more integrated circuit(s), logic circuit(s), controller(s), microcontroller(s) and/or microprocessor(s) from any desired family or manufacturer.
- the processor 702 of the illustrated example includes a local memory 704 (e.g., a cache).
- the processor 702 of the illustrated example also includes the example criteria-based input determiner 102 , the example unit-based input determiner 104 , the example matrix determiner 106 , the example model parameter determiner 108 , the example calibration engine 110 , the example calibrated weights determiner 130 , the example calibrated reporting data determiner 134 , and the example calibration model validator 138 of FIG. 1 .
- the processor 702 of the illustrated example is also in communication with a main memory including a volatile memory 706 and a non-volatile memory 708 via a bus 710 .
- the volatile memory 706 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any other type of random access memory device.
- the non-volatile memory 708 may be implemented by flash memory and/or any other desired type of memory device. Access to the volatile memory 706 and the non-volatile memory 708 is controlled by a memory controller.
- the processor 702 of the illustrated example is also in communication with one or more mass storage device(s) 712 for storing software and/or data.
- mass storage devices 712 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives.
- the mass storage device 712 includes the example memory 114 of FIG. 1 .
- the processor platform 700 of the illustrated example also includes a user interface circuit 714 .
- the user interface circuit 714 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface.
- one or more input device(s) 140 are connected to the user interface circuit 714 .
- the input device(s) 140 permit(s) a user to enter data and commands into the processor 702 .
- the input device(s) 140 can be implemented by, for example, an audio sensor, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint, a voice recognition system, a microphone, and/or a liquid crystal display.
- One or more output device(s) 142 are also connected to the user interface circuit 714 of the illustrated example.
- the output device(s) 142 can be implemented, for example, by a light emitting diode, an organic light emitting diode, a liquid crystal display, a touchscreen and/or a speaker.
- the user interface circuit 714 of the illustrated example may, thus, include a graphics driver such as a graphics driver chip and/or processor.
- the input device(s) 140 , the output device(s) 142 and the user interface circuit 714 collectively form the example user interface 112 of FIG. 1 .
- the processor platform 700 of the illustrated example also includes a network interface circuit 716 .
- the network interface circuit 716 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface.
- the network interface circuit 716 facilitates the exchange of data and/or signals with external machines (e.g., a remote server) via a network 718 (e.g., a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), the Internet, a cellular network, etc.).
- LAN local area network
- WLAN wireless local area network
- WAN wide area network
- the Internet a cellular network, etc.
- Coded instructions 720 corresponding to FIGS. 2-6 may be stored in the local memory 704 , in the volatile memory 706 , in the non-volatile memory 708 , in the mass storage device 712 , and/or on a removable tangible computer readable storage medium such as a flash memory stick, a CD or DVD.
- the disclosed soft calibration models and/or techniques advantageously reduce (e.g., eliminate) the above-described errors that may arise in connection with the use of hard calibration models and/or techniques when the calibration weights take on a relatively large magnitude.
- a data measurement company implementing the disclosed soft calibration models and/or techniques in lieu of traditional hard calibration models and/or techniques may advantageously provide and/or generate calibrated panelist and/or reporting data that more closely matches and/or resembles reference and/or benchmark data.
- the apparatus comprises a calibration engine to execute the calibration model based on a target loss function, a weight loss function, and a budget parameter.
- the apparatus further comprises a calibrated weights determiner to determine calibrated weights resulting from execution of the calibration model.
- the apparatus further comprises a calibration model validator to incorporate a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model.
- the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- the calibration engine is to re-execute the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter.
- the calibrated weights determiner is to re-determine the calibrated weights resulting from re-execution of the calibration model.
- the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- the calibration engine is to execute the calibration model based further on criteria-based input data to be incorporated into the target loss function and unit-based input data to be incorporated into the weight loss function.
- the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables.
- the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variable.
- the calibration engine is to execute the calibration model based further on matrix data.
- the matrix data is based on the criteria-based input data and the unit-based input data.
- the method comprises executing the calibration model based on a target loss function, a weight loss function, and a budget parameter. In some disclosed examples, the method further comprises determining, by executing one or more computer readable instructions with a processor, calibrated weights resulting from the executing of the calibration model. In some disclosed examples, the method further comprises incorporating, by executing one or more computer readable instructions with the processor, a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model. In some disclosed examples, the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- the method further includes re-executing the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter. In some disclosed examples, the method further includes re-determining the calibrated weights resulting from the re-executing of the calibration model. In some disclosed examples, the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- the executing of the calibration model is further based on criteria-based input data incorporated into the target loss function and unit-based input data incorporated into the weight loss function.
- the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables.
- the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variables.
- the executing of the calibration model is further based on matrix data. In some disclosed examples, the matrix data is based on the criteria-based input data and the unit-based input data.
- Tangible machine-readable storage media comprising instructions are also disclosed.
- the instructions when executed, cause a processor to execute a calibration model based on a target loss function, a weight loss function, and a budget parameter.
- the calibration model has relaxed benchmark constraints.
- the instructions when executed, cause the processor to determine calibrated weights resulting from the execution of the calibration model.
- the instructions when executed, cause the processor to incorporate a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model.
- the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- the instructions, when executed, are further to cause the processor to re-execute the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter. In some disclosed examples, the instructions, when executed, are further to cause the processor to re-determine the calibrated weights resulting from the re-execution of the calibration model. In some disclosed examples, the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- the instructions, when executed, are to cause the processor to execute the calibration model based further on criteria-based input data incorporated into the target loss function and unit-based input data incorporated into the weight loss function.
- the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables.
- the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variables.
- the instructions, when executed, are to cause the processor to execute the calibration model based further on matrix data.
- the matrix data is based on the criteria-based input data and the unit-based input data.
Abstract
Description
- This application arises from a continuation of U.S. patent application Ser. No. 16/942,429, filed Jul. 29, 2020, which is a continuation of U.S. patent application Ser. No. 15/583,337, filed May 1, 2017, which claims priority to U.S. Provisional Patent Application Ser. No. 62/346,897, filed Jun. 7, 2016, U.S. Provisional Patent Application Ser. No. 62/378,041, filed Aug. 22, 2016, U.S. Provisional Patent Application Ser. No. 62/379,205, filed Aug. 24, 2016, and U.S. Provisional Patent Application Ser. No. 62/379,517, filed Aug. 25, 2016. The entireties of U.S. patent application Ser. No. 16/942,429, U.S. patent application Ser. No. 15/583,337, U.S. Provisional Patent Application Ser. No. 62/346,897, U.S. Provisional Patent Application Ser. No. 62/378,041, U.S. Provisional Patent Application Ser. No. 62/379,205, and U.S. Provisional Patent Application Ser. No. 62/379,517 are hereby incorporated by reference herein.
- This disclosure relates generally to methods, systems, and apparatus for calibrating data and, more specifically, to methods, systems, and apparatus for calibrating data using relaxed benchmark constraints.
- Traditional models and/or techniques for calibrating data adjust and/or modify projection weights for the purpose of improving upon the accuracy of estimates that may be yielded from applying the projection weights to panelist and/or reporting data (e.g., survey data). The goal of such traditional calibration models and/or techniques is to adjust and/or modify the projection weights such that the weighted totals and/or results of the panelist and/or reporting data closely match known totals and/or results of reference and/or benchmark data (e.g., census data).
- Traditional calibration models and/or techniques are executed and/or performed in a “hard” manner, whereby the projection weights are adjusted and/or modified such that the estimated totals of the panelist and/or reporting data match exactly with the known totals of the reference and/or benchmark data. Such traditional models and/or techniques for calibrating data are accordingly referred to herein as “hard calibration” models and/or techniques.
-
FIG. 1 is a block diagram of an example calibration apparatus constructed in accordance with the teachings of this disclosure. -
FIG. 2 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus ofFIG. 1 to calibrate data using relaxed benchmark constraints incorporated into a soft calibration model. -
FIG. 3 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus ofFIG. 1 to determine criteria-based input data for the target loss function of the soft calibration model to be executed by the calibration apparatus. -
FIG. 4 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus ofFIG. 1 to determine unit-based input data for the weight loss function of the soft calibration model to be executed by the calibration apparatus. -
FIG. 5 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus ofFIG. 1 to determine model parameter data for the soft calibration model to be executed by the calibration apparatus. -
FIG. 6 is a flowchart representative of example machine readable instructions that may be executed at the example calibration apparatus ofFIG. 1 to validate the soft calibration model executed by the calibration apparatus. -
FIG. 7 is an example processor platform capable of executing the instructions ofFIGS. 2-6 to implement the example calibration apparatus ofFIG. 1 . - Certain examples are shown in the above-identified figures and described in detail below. In describing these examples, identical reference numbers are used to identify the same or similar elements. The figures are not necessarily to scale and certain features and certain views of the figures may be shown exaggerated in scale or in schematic for clarity and/or conciseness.
- Clients of data measurement companies expect that panelist and/or reporting data (e.g., data obtained from tightly controlled participants having a particular demographic representation) should reasonably match reference and/or benchmark data (e.g., raw data acquired from the market, such as actual sales dollars from retail stores that may be obtained from scanner data at checkout). In the event that a data measurement company's panelist and/or reporting data is substantially different from the reference and/or benchmark data, the data measurement company's clients may question the accuracy and/or validity of the data measurement company's ability to accurately project sales and/or usage numbers using the panelist and/or reporting data.
- Data measurement companies often calibrate panelist and/or reporting data to reference and/or benchmark data utilizing the traditional hard calibration models and/or techniques described above. At least one drawback of such hard calibration models and/or techniques is that the calibration weights may begin to take on a relatively large magnitude. While some demographic groups of panelist and/or reporting data might align well with the reference and/or benchmark data when such large-magnitude calibration weights are applied, these same calibration weights may result in substantial errors for other demographic groups.
- In contrast to the traditional hard calibration models and/or techniques described above, the methods, systems and apparatus for calibrating data disclosed herein use relaxed benchmark constraints. As used herein, the term “soft calibration” refers to a convex optimized calibration model and/or technique providing a unique solution for calibrating a set of weights (e.g., projection weights) relative to a set of relaxed benchmark constraints. The benchmark constraints that may be implemented in the disclosed soft calibration models and/or techniques are relaxed (e.g., loosened, broadened, and/or made less rigid) relative to the benchmark constraints that might otherwise be implemented in traditional hard calibration models and/or techniques. For example, the disclosed soft calibration models and/or techniques relax the exact matching requirement of the traditional hard calibration process.
- Unlike hard calibration models and/or techniques requiring that estimated totals associated with projection weights match reference and/or benchmark totals precisely, the disclosed soft calibration models and/or techniques advantageously generate adjusted and/or modified projection weights such that the distance and/or deviation between the estimated totals and the reference and/or benchmark totals is smaller than some particular value (e.g., a threshold and/or budget). By relaxing the benchmark constraints associated with traditional hard calibration models and/or techniques, the disclosed soft calibration models and/or techniques advantageously reduce (e.g., eliminate) the above-described errors that may arise in connection with the use of hard calibration models and/or techniques when the calibration weights take on a relatively large magnitude. Thus, a data measurement company implementing the disclosed soft calibration models and/or techniques in lieu of traditional hard calibration models and/or techniques may advantageously provide and/or generate calibrated panelist and/or reporting data that more closely matches and/or resembles reference and/or benchmark data.
- In some examples, the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on criteria-based input data to be incorporated into a soft calibration model. In some examples, the criteria-based input data provides inputs for a target loss function of the soft calibration model.
- In some examples, the criteria-based input data may include criteria associated with panelist and/or reporting data, criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables. As used herein, the terms “criterion” and/or “criteria” refer to one or more panelist and/or reporting data point(s) to be taken into account when executing and/or performing a calibration model and/or technique. In some examples, the criteria may be identified by a unique set of names and/or characters. In other examples, the criteria may be identified by a unique set of numbers. An example indexed set of criteria (M) may be expressed and/or defined as follows:
- In some examples, a set of criteria variables represents panelist and/or reporting values for each criterion. An example set of criteria variables (y) may be expressed and/or defined in scalar and vector notation as follows:
- In the setting of soft calibration, the criteria variables are not identified and/or made to be equal to any specific prescribed value. Instead, the criteria variables are identified and/or made to be as close as possible to a set of ideal values (e.g., reference and/or benchmark data values). As used herein, the terms “target” and/or “criteria target” refer to an ideal value associated with a criteria variable. Targets for the criteria variables are to be provided as input parameters to the soft calibration model. An example set of targets (t) may be expressed and/or defined as follows:
- In some examples, an upper bound and a lower bound may additionally be associated with each criteria variable. The upper bound may be a non-negative number that is greater than or equal to the corresponding target. The lower bound may be a non-negative number that is smaller than or equal to the corresponding target. Setting both the upper and lower bounds equal to the corresponding target causes the soft calibration model and/or technique to mimic a traditional hard calibration model and/or technique. Conversely, setting the lower bound to zero (0) and the upper bound to a large multiple of the target may result in the lower and upper bounds being practically inactive.
- In some examples, the upper and lower bounds associated with respective ones of the criteria variables and/or targets are identified independently. Thus, some criteria variables may be required to match their corresponding targets exactly, while others criteria variables may have very loose bounds relative to their corresponding targets. Upper and lower bounds for a set of criteria variables are to be supplied as input parameters to the soft calibration model. An example set of constraints corresponding to upper bounds (u) and lower bounds (l) to be associated with a set of criteria variables (y) and/or targets (t) may be expressed and/or defined as follows:
- In some examples, respective ones of the targets to be associated with corresponding ones of the criteria variables may have different measurement scales and/or units. For example, demographic targets measured in terms of a number of people in a group may be mixed together with purchased amount targets measured in terms of a local currency. To enable computation of a reasonable target loss, these different scales and/or units of measurement may be converted to measureless comparable quantities. In some examples, such a conversion may be based on relative distances (e.g., ((yi−ti)/ti)) between respective ones of the targets and corresponding ones of the criteria variables. In other examples, a scaling parameter (c) may be applied to respective ones of the criteria variables. Each scaling parameter (c) may be based on a relative standard error (rse) associated with the target (t) corresponding to the respective criteria variable (e.g. ci=rseiti). In examples described herein, the scaling parameters have positive values. Example scaling parameters (c) per criteria variable for computing target loss may be expressed and/or defined as follows:
- In some examples, an importance parameter (v) may be applied to respective ones of the scaled criteria variables. The importance parameters may capture and/or represent preferences associated with respective ones of the scaled criteria variables. In some examples, each importance parameter (v) may be based on a monotonically increasing concave function of the corresponding target (t)
-
- In examples described herein, the importance parameters have positive values. Example importance parameters (v) per criteria variable for computing target loss may be expressed and/or defined as follows:
- In some examples, the criteria-based input data (e.g., criteria associated with panelist and/or reporting data, criteria variables, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables) may be provided by, accessed from, and/or stored in a tab-separated values (TSV) file having multiple columns. For example, a first column of a TSV file for the criteria-based input data may include identifiers (e.g., a set of unique names and/or characters) for each criterion. A second column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each target. A third column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each lower bound. A fourth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each upper bound. A fifth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each scaling parameter. A sixth column of the TSV file for the criteria-based input data may include values (e.g., double precision numerical values) for each importance parameter. In such an example, the respective values of the second column of the TSV file are greater than or equal to corresponding ones of the respective values of the third column of the TSV file, and less than or equal to corresponding ones of the respective values of the fourth column of the TSV file. In some such examples, the respective values of the fifth column and the sixth column of the TSV file are positive, and cannot be zero (0).
- In other examples, the criteria-based input data may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file. For example, the criteria-based input data may be provided by, accessed from, and/or stored in a comma-separated values (CSV) file, or some other type of data file.
- In response to identifying and/or determining the criteria-based input data (e.g., criteria associated with panelist and/or reporting data, criteria variables, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables), distances between respective ones of the panelist and/or reporting values captured by the criteria variables (y), and corresponding respective ones of the targets (t), may be measured and/or determined. As used herein, the term “target loss” refers to a measured distance between (a) a panelist and/or reporting value captured by a criteria variable, and (b) a corresponding target. In some examples, target loss may be expressed and/or defined in terms of a target loss function (F(y)). In some examples, it is desired that the target loss function (F(y)) will express a weighted average of criteria variable distances while at the same time being robust with respect to outliers. Thus, it may be desired that the target loss function (F(y)) is both convex and robust.
- In some examples, the target loss function (F(y)) may be based on a known Huber loss function (Hϕ(⋅)). For example, the target loss function (F(y)) may be expressed and/or defined as follows:
-
- where Hϕ(⋅) is the Huber loss function expressed and/or defined as follows:
-
- Replacing the scalar (z) of the Huber loss function with a vector (z) results in (Hϕ(z)) being applied for each component of (z) with the same value of the parameter ϕ. In such an example, (Hϕ(Z)) is of the same dimensionality as (z).
- In some examples, the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on unit-based input data to be incorporated into the soft calibration model. In some examples the unit-based input data provides inputs for a weight loss function of the soft calibration model.
- In some examples, the unit-based input data may include units associated with panelist and/or reporting data, unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables. As used herein, the term “unit” refers to one or more projection weight(s) to be used and/or calibrated to attain one or more corresponding panelist and/or reporting data point(s) relative to one or more corresponding target(s) when executing and/or performing a calibration model and/or technique. In some examples, the units may be identified by a unique set of names and/or characters. In other examples, the units may be identified by a unique set of numbers. An example indexed set of units (N) may be expressed and/or defined as follows:
- In some examples, a set of unit variables represents the calibrated weight for each unit. An example set of unit variables (x) may be expressed and/or defined in scalar and vector notation as follows:
- Unlike the above-described criteria variables that are made to be as close as possible to the above-described targets, there is no corresponding set of ideal values (e.g., reference and/or benchmark data values) for respective ones of the unit variables. Instead, respective ones of the unit variables are associated with corresponding respective ones of initial weights that are to be adjusted as little as possible. In some examples, respective ones of the initial weights may assume different roles in different domains (e.g., coverage adjustment, weights derived by panel design, etc.). Initial weights for the unit variables are to be supplied as input parameters to the soft calibration model. An example set of initial weights (xo) may be expressed and/or defined as follows:
- In some examples, an upper bound and a lower bound may additionally be associated with each unit variable. The upper bound may be a non-negative number that is greater than or equal to the corresponding initial weight. The lower bound may be a non-negative number that is smaller than or equal to the corresponding initial weight. In some examples, the lower bound may be set to zero (0) and the upper bound may be set to a large multiple of the initial weight. In other examples, the lower bound and the upper bound may respectively be determined by dividing (for the lower bound) and/or multiplying (for the upper bound) the initial weight by a constant.
- In some examples, the upper and lower bounds associated with respective ones of the unit variables and/or initial weights are identified independently. Upper and lower bounds for the unit variables are to be supplied as input parameters to the soft calibration model. An example set of constraints corresponding to upper bounds (u) and lower bounds (l) to be associated with a set of unit variables (x) and/or initial weights (xo) may be expressed and/or defined as follows:
- Unlike respective ones of the targets to be associated with corresponding ones of the criteria variables described above, respective ones of the initial weights associated with corresponding respective ones of the unit variables do not have different measurement scales and/or units. Accordingly, there is no need to convert differing scales and/or units of measurement to measureless comparable quantities in connection with computing weight loss.
- In some examples, a size parameter (s) may be applied to respective ones of the unit variables. The size parameters may capture and/or represent preferences associated with respective ones of the unit variables. In some examples, each size parameter (s) may be based on an average of non-zero contributions of a unit to the criteria. In examples described herein, the size parameters have positive values. Example size parameters (s) per unit variable for computing weight loss may be expressed and/or defined as follows:
- In some examples, the unit-based input data (e.g., units associated with panelist and/or reporting data, unit variables for the units, initial weights for unit variables, lower bounds for unit variables, upper bounds for unit variables, and size parameters for unit variables) may be provided by, accessed from, and/or stored in a TSV file having multiple columns. For example, a first column of a TSV file for the unit-based input data may include identifiers (e.g., a set of unique names and/or characters) for each unit. A second column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each initial weight. A third column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each lower bound. A fourth column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each upper bound. A fifth column of the TSV file for the unit-based input data may include values (e.g., double precision numerical values) for each size parameter. In such an example, the respective values of the second column of the TSV file are greater than or equal to corresponding ones of the respective values of the third column of the TSV file, and less than or equal to corresponding ones of the respective values of the fourth column of the TSV file. In some such examples, the respective values of the fifth column of the TSV file are positive, and cannot be zero (0).
- In other examples, the unit-based input data may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file. For example, the unit-based input data may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- In response to identifying and/or determining the unit-based input data (e.g., units associated with panelist and/or reporting data, unit variables for the units, initial weights for unit variables, lower bounds for unit variables, upper bounds for unit variables, and size parameters for unit variables), distances between respective ones of the currently evaluated weights captured by the unit variables (x), and corresponding respective ones of the initial weights)(x°, may be measured and/or determined. As used herein, the term “weight loss” refers to a measured distance between (a) an evaluated weight captured by a unit variable, and (b) a corresponding initial weight. In some examples, weight loss may be expressed and/or defined in terms of a weight loss function (G(x)). In some examples, it is desired that the weight loss function (G(x)) express a weighted average of unit variable distances. For example, the weight loss function (G(x)) may be expressed and/or defined as follows:
-
- In some examples, the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on matrix data to be incorporated into the soft calibration model. In some examples, a design matrix including the matrix data may be generated and/or derived from the criteria-based input data and the unit-based input data. The design matrix may capture and/or generate relationships between the above-described units and the above-described criteria. For example, the non-expanded contribution of each unit to each criterion may be captured by a design matrix having non-negative entries. An example design matrix may be expressed and/or defined as follows:
- Accordingly, an example relationship formula for relationships between units and criteria may be expressed and/or defined as follows:
-
- In some examples, the design matrix may be provided by, accessed from, and/or stored in a TSV file having multiple columns. For example, a first column of a TSV file for the design matrix may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data. A second column of the TSV file for the design matrix may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the criteria-based input data. A third column of the TSV file for the design matrix may include values (e.g., double precision numerical values) for the contribution of respective ones of the units to respective ones of the criteria without weighting. In such an example, the respective values of the third column of the TSV file for the design matrix are positive and not equal to zero (0). In some such examples, if there is no record for a given pair of unit and criteria identifiers in the TSV file for the design matrix, the data is assumed to be zero (0).
- In other examples, the design matrix may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file. For example, the design matrix may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- In some examples, the apparatus, systems and methods disclosed herein for calibrating data using relaxed benchmark constraints determine a set of calibrated weights (e.g., calibrated projection weights to be applied to panelist and/or reporting data) based in part on model parameter data to be incorporated into the soft calibration model. In some examples, the model parameter data includes a Huber function parameter associated with target loss and a budget parameter associated with weight loss. As described above in connection with the target loss function of Equation 7, a Huber loss function Hϕ(⋅) may be implemented to measure the distance between (a) a panelist and/or reporting value captured by a criteria variable, and (b) a corresponding target. The Huber loss function is parametric in ϕ. Identification of a Huber function parameter (ϕ) applicable across all criteria enables a fixed target loss to be computed.
- Target loss and the weight loss are conflicting objectives of the soft calibration process. For example, the smaller that the value of the target loss becomes, the larger that the value of the weight loss becomes by necessity, and vice-versa. In some examples, to obtain a unique solution, the target loss may be minimized and a constraint may be placed on the value that the weight loss is allowed to obtain. As used herein, the term “budget” refers to a constraint placed on the value that the weight loss is allowed to obtain. In some examples, a budget parameter (θ) is an upper bound (e.g., θ∈ >0) associated with the weight loss. An example constraint corresponding to the budget parameter (θ) may be expressed and/or defined as follows:
- Based on the above-described criteria-based input data (e.g., criteria associated with panelist and/or reporting data, criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables), target loss function, unit-based input data (e.g., units associated with panelist and/or reporting data, unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and size parameters for the unit variables), weight loss function, matrix data, and model parameter data, an example calibration model to calibrate data using relaxed benchmark constraints (e.g., an example soft calibration model) may be expressed and/or defined as follows:
-
min F(y) -
subject to: Ax=y -
G(x)≤θ Equation 17: - The soft calibration model of Equation 17 may be implemented and/or executed to provide a unique solution whereby respective calibrated panelist and/or reporting values from among a set of calibrated panelist and/or reporting values are close in value to, but not necessarily equal to, respective target and/or benchmark values from among a corresponding set of target and/or benchmark values. Implementation and/or execution of the soft calibration model of Equation 17 also results in a determination and/or identification of the calibrated weights (e.g., optimized weights). As the initial weights (xo) are known and fixed, the calibrated weights may be identified and/or determined based on the weight adjustments per unit
-
- In some examples, calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a TSV file having multiple columns. For example, a first column of a TSV file for the calibrated weights may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data. A second column of the TSV file for the calibrated weights may include values (e.g., double precision numerical values) for the weight adjustment (xadj) of each unit or, alternatively, the calibrated weight of each unit. In other examples, calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file. For example, calibrated weights determined as a result of executing the soft calibration model of Equation 17 may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
- Table 1 below provides an example of panelist and/or reporting data to be calibrated:
-
TABLE 1 AgeGroup Sex Population RetailerA RetailerB 15-34 Male 40199347 28.68 1.28 35-54 Male 40945028 82.78 2.32 55+ Male 26054981 21.72 0.46 15-34 Female 38876268 27.97 1.82 35-54 Female 41881451 37.10 2.00 55+ Female 33211456 13.82 0.42 - The panelist and/or reporting data of Table 1 includes two demographic dimensions: (1) an age group dimension having three levels; and (2) a sex dimension having two levels. A population estimate is provided for each intersection of age and group dimensions. The panelist and/or reporting data of Table 1 also includes the average spending per person for two retailers (e.g., RetailerA and RetailerB).
- Criteria-based input data may be associated with the panelist and/or reporting data of Table 1 above. For example, Table 2 below provides an example of criteria-based input data associated with the panelist and/or reporting data of Table 1 in a scenario where the scaling parameters (c) described above in connection with Equation 5 have been set to equal the corresponding targets (t), and where the importance parameters (v) described above in connection with Equation 6 have been set to a value of 1.00 for the demographic targets:
-
TABLE 2 criteria ID target ly uy V c demo::Sex::Male 107199356 107199356 107199356 1.00 107199356 demo::Sex::Female 113969175 113969175 113969175 1.00 113969175 demo::AgeGroup::15-34 79075615 79075615 79075615 1.00 79075615 demo::AgeGroup::35-54 82826479 82826479 82826479 1.00 82826479 demo::AgeGroup::55+ 59266437 59266437 59266437 1.00 59266437 volumetric::RetailerA 9029255672 6566731398 9850097097 90600.30 8208414247 volumetric::RetailerB 310554470 261519553 392279330 18080.36 326899442 - Unit-based input data may also be associated with the panelist and/or reporting data of Table 1 above. For example, Table 3 below provides an example of unit-based input data associated with the panelist and/or reporting data of Table 1 in a scenario where the initial weights (x0) are set to equal the population estimates of the panelist and/or reporting data at intersection level:
-
TABLE 3 unit ID x0 lx ux size 15-34::Male 40199347 20099674 80398694 29.96 35-54::Male 40945028 20472514 81890056 85.10 55+::Male 26054981 13027491 52109962 22.18 15-34::Female 38876268 19438134 77752536 29.79 35-54::Female 41881451 20940726 83762902 39.10 55+::Female 33211456 16605728 66422912 14.24 - Table 4 below provides an example of a design matrix including matrix data associated with the criteria-based input data of Table 2 above and the unit-based input data of Table 3 above:
-
TABLE 4 unit ID criteria ID a 15-34::Male demo::Sex::Male 1.00 35-54::Male demo::Sex::Male 1.00 55+::Male demo::Sex::Male 1.00 15-34::Female demo::Sex::Female 1.00 35-54::Female demo::Sex::Female 1.00 55+::Female demo::Sex::Female 1.00 15-34::Male demo::AgeGroup::15-34 1.00 35-54::Male demo::AgeGroup::35-54 1.00 55+::Male demo::AgeGroup::55+ 1.00 15-34::Female demo::AgeGroup::15-34 1.00 35-54::Female demo::AgeGroup::35-54 1.00 55+::Female demo::AgeGroup::55+ 1.00 15-34::Male volumetric::RetailerA 28.68 35-54::Male volumetric::RetailerA 82.78 55+::Male volumetric::RetailerA 21.72 15-34::Female volumetric::RetailerA 27.97 35-54::Female volumetric::RetailerA 37.10 55+::Female volumetric::RetailerA 13.82 15-34::Male volumetric::RetailerB 1.28 35-54::Male volumetric::RetailerB 2.32 55+::Male volumetric::RetailerB 0.46 15-34::Female volumetric::RetailerB 1.82 35-54::Female volumetric::RetailerB 2.00 55+::Female volumetric::RetailerB 0.42 - Table 5 below provides an example of calibrated weights data resulting from executing the soft calibration model of Equation 17 based in part on the criteria-based input data of Table 2, the unit-based input data of Table 3, and the matrix data of Table 4, in a scenario where the budget parameter (θ) has been set to a value of 0.35 and the Huber function parameter (ϕ) has been set to a value of 0.50:
-
TABLE 5 unit ID xadj 15-34::Male 0.8789425 35-54::Male 1.4060901 55+::Male 0.5486109 15-34::Female 1.1251775 35-54::Female 0.6029897 55+::Female 1.3541228 - In some circumstances, execution of the soft calibration model of Equation 17 described above may result in the soft calibration model performing in an unexpected manner. One such set of circumstances occurs when two conditions are present: (1) the target loss function (F(y)) has a flat region (D) (e.g., a flat region (D) expressed and/or defined as ∃D⊆ ≥0 m: y1,y2∈D⇒|F(yi)−F(y2)|≈0); and (2) the budget parameter (θ) is large enough to allow the minimization procedure of the soft calibration model of Equation 17 to reach the flat region (D) of the target loss function (F(y)).
- For example, in the case of a totally flat region (D) (e.g., where |F(y1)−F(y2)|=0 in condition (1) above), the soft calibration model of Equation 17 does not specify which one of the countless possibilities is to be preferred. Different solvers might choose to implement heuristics picking up one of the edges of the optimal set or, alternatively, a central point of this set. A particular case is when there is a totally flat optimal region for the target loss and all criteria are fulfilled exactly (e.g., where y∈D⇒F(y)=0). This particular case reduces the soft calibration problem to the traditional case of statistical weight calibration. Yet, unlike the traditional calibration approach, the soft calibration model of Equation 17 does not capture a preference for a solution that minimizes the weight loss. As another example, in the case of a nearly flat region (D) (e.g., where |F(y1)−F(y2)| is negligible and/or close to machine precision), implementation heuristics become unreliable, and may result in a progressively increasing weight loss function (G(x)) that provides no meaningful improvement in relation to the target attainment. In some such examples, execution of the soft calibration model may result in calibrated weights that do not provide a unique solution for the executed soft calibration model.
- In some examples, a stability parameter (γ) may be incorporated into (e.g., appended to) the soft calibration model of Equation 17 to remedy and/or prevent unexpected performance of the soft calibration model resulting from the above-described circumstances and/or conditions. The stability parameter (γ) reduces the influence of the budget parameter (θ) on the soft calibration model. For example, incorporation of the stability parameter (γ) may prevent the minimization procedure of the soft calibration model of Equation 17 from reaching the flat region (D) of the target loss function (F(y)). In some examples, the stability parameter (γ) is included as part of the model parameter data described above. In some examples, the stability parameter (γ) may be incorporated into the soft calibration model of Equation 17 by adding the product of the stability parameter (γ) and the weight loss function (G(x)) to the target loss function (F(y)). In such examples, the revised soft calibration model may be expressed and/or defined as follows:
-
min F(y)+γG(x) -
subject to: Ax=y -
G(x)≤θ Equation 18: - In some examples, the stability parameter (γ) may be determined and/or provided at run-time. As the target loss function (F(y)) and the weight loss function (G(x)) are both scaled to be measureless (e.g. both are non-negative, and both are expected to be not much larger than 1), the stability parameter (γ) may preferably have a value between 0.00001 and 0.01 (e.g., comparable to a precision sensitivity associated with the soft calibration model). In some examples, the value of the stability parameter (γ) may be increased until the calibrated weights that result from executing the soft calibration model provide a unique solution for the soft calibration model. Thus, implementation and/or incorporation of the stability parameter (γ) may reduce the occurrence of errors (e.g., the generation of calibrated weights providing non-unique solutions) attributable to and/or arising from an influence of the value of the budget parameter (θ) on the soft calibration model.
- The soft calibration model of Equation 18 may be implemented and/or executed to provide a unique solution whereby respective calibrated panelist and/or reporting values from among a set of calibrated panelist and/or reporting values are close in value to, but not necessarily equal to, respective target and/or benchmark values from among a corresponding set of target and/or benchmark values. Implementation and/or execution of the soft calibration model of Equation 18 also results in a determination and/or identification of the calibrated weights (e.g., optimized weights). As the initial weights (x0) are known and fixed, the calibrated weights may be identified and/or determined based on the weight adjustments per unit
-
- In some examples, calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a TSV file having multiple columns. For example, a first column of a TSV file for the calibrated weights may include identifiers (e.g., a set of unique names and/or characters) corresponding to the first column of the above-described TSV file for the unit-based input data. A second column of the TSV file for the calibrated weights may include values (e.g., double precision numerical values) for the weight adjustment of each unit or, alternatively, the calibrated weight of each unit. In other examples, calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a file having a format that differs from that of a TSV file. For example, calibrated weights determined as a result of executing the soft calibration model of Equation 18 may be provided by, accessed from, and/or stored in a CSV file, or some other type of data file.
-
FIG. 1 is a block diagram of anexample calibration apparatus 100 constructed in accordance with the teachings of this disclosure. Thecalibration apparatus 100 ofFIG. 1 may be implemented to calibrate panelist and/or reporting data using relaxed benchmark constraints incorporated into a soft calibration model (e.g., the example soft calibration model of Equation 17 or the example soft calibration model of Equation 18 described above). In the illustrated example ofFIG. 1 , thecalibration apparatus 100 includes an example criteria-basedinput determiner 102, an example unit-basedinput determiner 104, anexample matrix determiner 106, an examplemodel parameter determiner 108, anexample calibration engine 110, anexample user interface 112, and anexample memory 114. - The example criteria-based
input determiner 102 ofFIG. 1 determines and/or identifies example criteria-basedinput data 118 to be incorporated into an exampletarget loss function 120 of a soft calibration model to be executed by theexample calibration engine 110 of thecalibration apparatus 100 ofFIG. 1 . The criteria-basedinput data 118 determined and/or identified by the criteria-basedinput determiner 102 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. In some examples, the criteria-basedinput determiner 102 ofFIG. 1 determines and/or identifies the criteria-basedinput data 118 by accessing and/or obtaining the criteria-basedinput data 118 from thememory 114 ofFIG. 1 . In other examples, the criteria-basedinput determiner 102 ofFIG. 1 determines and/or identifies criteria-basedinput data 118 received from theexample user interface 112 ofFIG. 1 described below. In such other examples, the criteria-basedinput determiner 102 ofFIG. 1 may cause thememory 114 ofFIG. 1 to store the received criteria-basedinput data 118. - In some examples, the criteria-based
input data 118 to be determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may include criteria associated with example panelist and/or reporting data (e.g., survey data) 116, criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables. In some examples, the criteria of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 1 described above. In some examples, the criteria variables of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 2 described above. In some examples, the targets of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 3 described above. In some examples, the lower and upper bounds of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 4 described above. In some examples, the scaling parameters of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 5 described above. In some examples, the importance parameters of the criteria-basedinput data 118 may be expressed in a manner consistent with Equation 6 described above. In some examples, thetarget loss function 120 into which the criteria-basedinput data 118 determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 is to be incorporated may be expressed in a manner consistent with Equation 7 described above. - The example unit-based
input determiner 104 ofFIG. 1 determines and/or identifies example unit-basedinput data 122 to be incorporated into an exampleweight loss function 124 of the soft calibration model to be executed by theexample calibration engine 110 of thecalibration apparatus 100 ofFIG. 1 . The unit-basedinput data 122 determined and/or identified by the unit-basedinput determiner 104 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. In some examples, the unit-basedinput determiner 104 ofFIG. 1 determines and/or identifies the unit-basedinput data 122 by accessing and/or obtaining the unit-basedinput data 122 from thememory 114 ofFIG. 1 . In other examples, the unit-basedinput determiner 104 ofFIG. 1 determines and/or identifies unit-basedinput data 122 received from theexample user interface 112 ofFIG. 1 described below. In such other examples, the unit-basedinput determiner 104 ofFIG. 1 may cause thememory 114 ofFIG. 1 to store the received unit-basedinput data 122. - In some examples, the unit-based
input data 122 to be determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may include units associated with the panelist and/orreporting data 116, unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables. In some examples, the units of the unit-basedinput data 122 may be expressed in a manner consistent with Equation 8 described above. In some examples, the unit variables of the unit-basedinput data 122 may be expressed in a manner consistent with Equation 9 described above. In some examples, the initial weights of the unit-basedinput data 122 may be expressed in a manner consistent with Equation 10 described above. In some examples, the lower and upper bounds of the unit-basedinput data 122 may be expressed in a manner consistent with Equation 11 described above. In some examples, the size parameters of the unit-basedinput data 122 may be expressed in a manner consistent with Equation 12 described above. In some examples, theweight loss function 124 into which the unit-basedinput data 122 determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 is to be incorporated may be expressed in a manner consistent with Equation 13 described above. - The
example matrix determiner 106 ofFIG. 1 determines, generates, derives and/or identifiesexample matrix data 126 to be incorporated into the soft calibration model to be executed by theexample calibration engine 110 of thecalibration apparatus 100 ofFIG. 1 . Thematrix data 126 determined, generated, derived and/or identified by thematrix determiner 106 ofFIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. In some examples, thematrix determiner 106 ofFIG. 1 determines and/or identifies thematrix data 126 by accessing and/or obtaining thematrix data 126 from thememory 114 ofFIG. 1 . In other examples, thematrix determiner 106 ofFIG. 1 determines and/or identifiesmatrix data 126 received from theexample user interface 112 ofFIG. 1 described below. In such other examples, thematrix determiner 106 ofFIG. 1 may cause thememory 114 ofFIG. 1 to store the receivedmatrix data 126. - In some examples, the
matrix determiner 106 ofFIG. 1 determines, generates, derives and/or identifies thematrix data 126 based on the example criteria-basedinput data 118 ofFIG. 1 and the example unit-basedinput data 122 ofFIG. 1 . In some examples, thematrix data 126 determined, generated derived and/or identified by thematrix determiner 106 ofFIG. 1 may be expressed in a manner consistent with Equation 14 and/or Equation 15 described above. In some examples, thematrix data 126 determined, generated, derived and/or identified by thematrix determiner 106 may capture relationships between the criteria of the criteria-basedinput data 118 and the units of the unit-basedinput data 122. For example, the non-expanded contribution of each unit to each criterion may be captured by thematrix data 126. - The example
model parameter determiner 108 ofFIG. 1 determines and/or identifies examplemodel parameter data 128 to be incorporated into the soft calibration model to be executed by theexample calibration engine 110 of thecalibration apparatus 100 ofFIG. 1 . Themodel parameter data 128 determined and/or identified by themodel parameter determiner 108 ofFIG. 1 may be of any type, form and/or format, and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. In some examples, themodel parameter determiner 108 ofFIG. 1 determines and/or identifies themodel parameter data 128 by accessing and/or obtaining themodel parameter data 128 from thememory 114 ofFIG. 1 . In other examples, themodel parameter determiner 108 ofFIG. 1 determines and/or identifiesmodel parameter data 128 received from theexample user interface 112 ofFIG. 1 described below. In such other examples, themodel parameter determiner 108 ofFIG. 1 may cause thememory 114 ofFIG. 1 to store the receivedmodel parameter data 128. - In some examples, the
model parameter data 128 to be determined and/or identified by themodel parameter determiner 108 ofFIG. 1 may include a Huber function parameter, a budget parameter, and/or a stability parameter. In some examples, the budget parameter of themodel parameter data 128 determined and/or identified by themodel parameter determiner 108 ofFIG. 1 may be expressed in a manner consistent with Equation 16 described above. - The
example calibration engine 110 ofFIG. 1 builds, constructs and/or compiles the soft calibration model to be executed by thecalibration apparatus 100. In some examples, thecalibration engine 110 ofFIG. 1 builds, constructs and/or compiles the soft calibration model based on the criteria-basedinput data 118, thetarget loss function 120, the unit-basedinput data 122, theweight loss function 124, thematrix data 126, and themodel parameter data 128 ofFIG. 1 . In some examples, the soft calibration model built, constructed and/or compiled by thecalibration engine 110 ofFIG. 1 may be expressed in a manner consistent with Equation 17 described above. In other examples, the soft calibration model built, constructed and/or compiled by thecalibration engine 110 ofFIG. 1 may be expressed in a manner consistent with Equation 18 described above. Thecalibration engine 110 ofFIG. 1 executes and/or controls the execution of the built, constructed and/or compiled soft calibration model. - In the illustrated example of
FIG. 1 , thecalibration engine 110 includes an example calibratedweights determiner 130. The calibrated weights determiner 130 ofFIG. 1 determines and/or calculates example calibratedweights data 132 in response to execution of the soft calibration model by thecalibration engine 110. In some examples, in response to thecalibration engine 110 executing the soft calibration model, the calibrated weights determiner 130 ofFIG. 1 determines the calibratedweights data 132 based on the criteria-basedinput data 118, thetarget loss function 120, the unit-basedinput data 122, theweight loss function 124, thematrix data 126, and themodel parameter data 128 incorporated into the executed soft calibration model. In some examples, the calibrated weights determiner 130 ofFIG. 1 may determine and/or calculate the calibratedweights data 132 based on adjusted calibration weights output by the soft calibration model. In some examples, the calibrated weights determiner 130 ofFIG. 1 may cause the calibratedweights data 132 to be presented (e.g., displayed) via theexample user interface 112 ofFIG. 1 . The calibratedweights data 132 determined and/or calculated by the calibrated weights determiner 130 ofFIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. - In the illustrated example of
FIG. 1 , thecalibration engine 110 also includes an example calibratedreporting data determiner 134. The example calibratedreporting data determiner 134 ofFIG. 1 determines and/or calculates example calibrated panelist and/orreporting data 136. In some examples, the calibrated panelist and/orreporting data 136 determined and/or calculated by the calibratedreporting data determiner 134 ofFIG. 1 is based on initial panelist and/or reporting data (e.g., the panelist and/orreporting data 116 ofFIG. 1 ) and the calibratedweights data 132 determined and/or calculated by the calibrated weights determiner 130 ofFIG. 1 . For example, the calibratedreporting data determiner 134 ofFIG. 1 may determine and/or calculate the calibrated panelist and/orreporting data 136 by applying the calibratedweights data 132 to the panelist and/orreporting data 116 ofFIG. 1 . In some examples, the calibratedreporting data determiner 134 ofFIG. 1 may cause the calibrated panelist and/orreporting data 136 to be presented (e.g., displayed) via theexample user interface 112 ofFIG. 1 . The calibrated panelist and/orreporting data 136 determined and/or calculated by the calibratedreporting data determiner 134 ofFIG. 1 may be of any type, form and/or format (e.g., a TSV file, a CSV file, etc.), and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. - In the illustrated example of
FIG. 1 , thecalibration engine 110 also includes an examplecalibration model validator 138. The example calibratedreporting data determiner 134 ofFIG. 1 validates the soft calibration model executed by theexample calibration engine 110 ofFIG. 1 . For example, thecalibration model validator 138 ofFIG. 1 may validate the soft calibration model executed by thecalibration engine 110 ofFIG. 1 by determining whether the calibrated weights resulting from execution of the soft calibration model (e.g., the calibratedweights data 132 determined by the calibrated weights determiner 130 ofFIG. 1 ) provide a unique solution for the executed soft calibration model. - In some examples, if the
calibration model validator 138 ofFIG. 1 determines that the resultant calibrated weights do not provide a unique solution for the executed soft calibration model, thecalibration model validator 138 may determine whether the executed soft calibration model included a stability parameter. For example, thecalibration model validator 138 ofFIG. 1 may determine that a soft calibration model expressed in a manner consistent with Equation 17 described above does not include a stability parameter. If thecalibration model validator 138 ofFIG. 1 determines that the executed soft calibration model does not include a stability parameter, thecalibration model validator 138 may incorporate a stability parameter into the soft calibration model, and may further cause thecalibration engine 110 ofFIG. 1 to re-execute the soft calibration model (e.g., the soft calibration model including the stability parameter) to re-determine the calibrated weights. For example, thecalibration model validator 138 ofFIG. 1 may incorporate a stability parameter expressed in a manner consistent with Equation 16 described above into the soft calibration model expressed in a manner consistent with Equation 17 described above to provide the soft calibration model expressed in a manner consistent with Equation 18 described above (e.g., a soft calibration model including a stability parameter). - The
example user interface 112 ofFIG. 1 facilitates interactions and/or communications between an end user and the criteria-basedinput determiner 102, the unit-basedinput determiner 104, thematrix determiner 106, themodel parameter determiner 108, thecalibration engine 110, thememory 114, the calibratedweights determiner 130, the calibratedreporting data determiner 134, thecalibration model validator 138, and/or, more generally, thecalibration apparatus 100 ofFIG. 1 . Theuser interface 112 ofFIG. 1 includes one or more input device(s) 140 via which the user may input information and/or data to the criteria-basedinput determiner 102, the unit-basedinput determiner 104, thematrix determiner 106, themodel parameter determiner 108, thecalibration engine 110, thememory 114, the calibratedweights determiner 130, the calibratedreporting data determiner 134, thecalibration model validator 138, and/or, more generally, thecalibration apparatus 100. For example, the one or more input device(s) 140 of theuser interface 112 may include a button, a switch, a keyboard, a mouse, a microphone, and/or a touchscreen that enable(s) the user to convey data and/or commands to the criteria-basedinput determiner 102, the unit-basedinput determiner 104, thematrix determiner 106, themodel parameter determiner 108, thecalibration engine 110, thememory 114, the calibratedweights determiner 130, the calibratedreporting data determiner 134, thecalibration model validator 138, and/or, more generally, thecalibration apparatus 100 ofFIG. 1 . - The
user interface 112 ofFIG. 1 also includes one or more output device(s) 142 via which theuser interface 112 presents information and/or data in visual and/or audible form to the user. For example, the one or more output device(s) 142 of theuser interface 112 may include a light emitting diode, a touchscreen, and/or a liquid crystal display for presenting visual information, and/or a speaker for presenting audible information. In some examples, the one or more output device(s) 142 of theuser interface 112 may present and/or display the example calibratedweights data 132 and/or the example calibrated panelist and/orreporting data 136 resulting from execution of the soft calibration model by theexample calibration engine 110 ofFIG. 1 . In some examples, the one or more output device(s) 142 of theuser interface 112 may be directed to present and/or display the example calibratedweights data 132 and/or the example calibrated panelist and/orreporting data 136 by one or more of thecalibration engine 110, the calibratedweights determiner 130, the calibratedreporting data determiner 134, thecalibration model validator 138, and/or, more generally, thecalibration apparatus 100 ofFIG. 1 . Data and/or information that is presented and/or received via theuser interface 112 ofFIG. 1 may be of any type, form and/or format, and may be stored in a computer-readable storage medium such as theexample memory 114 ofFIG. 1 described below. - The
example memory 114 ofFIG. 1 may be implemented by any type(s) and/or any number(s) of storage device(s) such as a storage drive, a flash memory, a read-only memory (ROM), a random-access memory (RAM), a cache and/or any other physical storage medium in which information is stored for any duration (e.g., for extended time periods, permanently, brief instances, for temporarily buffering, and/or for caching of the information). The data and/or information stored in thememory 114 may be stored in any file and/or data structure format, organization scheme, and/or arrangement. Thememory 114 is accessible to one or more of the example criteria-basedinput determiner 102, the example unit-basedinput determiner 104, theexample matrix determiner 106, the examplemodel parameter determiner 108, theexample calibration engine 110, theexample user interface 112, the example calibratedweights determiner 130, the example calibratedreporting data determiner 134, and/or the examplecalibration model validator 138 ofFIG. 1 , and/or, more generally, to the calibration apparatus ofFIG. 1 . - In some examples, the
memory 114 ofFIG. 1 stores data and/or information received via the one or more input device(s) 140 of theuser interface 112 ofFIG. 1 . In some examples, thememory 114 stores data and/or information to be presented via the one or more output device(s) 142 of theuser interface 112 ofFIG. 1 . In some examples, thememory 114 stores panelist and/orreporting data 116. In some examples, thememory 114 stores criteria-basedinput data 118 including criteria associated with the panelist and/orreporting data 116, criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables. In some examples, thememory 114 stores atarget loss function 120 to be incorporated into the soft calibration model to be executed by thecalibration engine 110 ofFIG. 1 . In some examples, thememory 114 stores unit-basedinput data 122 including units associated with the panelist and/orreporting data 116, unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables. In some examples, thememory 114 stores aweight loss function 124 to be incorporated into the soft calibration model to be executed by thecalibration engine 110 ofFIG. 1 . In some examples, thememory 114stores matrix data 126. In some examples, thememory 114 storesmodel parameter data 128 including a Huber function parameter, a budget parameter, and/or a stability parameter. In some examples, thememory 114 stores calibratedweights data 132 determined and/or calculated by the calibrated weights determiner 130 and/or, more generally, by thecalibration engine 110 ofFIG. 1 . In some examples, thememory 114 stores calibrated panelist and/orreporting data 136 determined and/or calculated by the calibratedreporting data determiner 134 and/or, more generally, by thecalibration engine 110 ofFIG. 1 . - While an example manner of implementing a
calibration apparatus 100 is illustrated inFIG. 1 , one or more of the elements, processes and/or devices illustrated inFIG. 1 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example criteria-basedinput determiner 102, the example unit-basedinput determiner 104, theexample matrix determiner 106, the examplemodel parameter determiner 108, theexample calibration engine 110, theexample user interface 112, theexample memory 114, the example calibratedweights determiner 130, the example calibratedreporting data determiner 134, and/or the examplecalibration model validator 138 ofFIG. 1 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example criteria-basedinput determiner 102, the example unit-basedinput determiner 104, theexample matrix determiner 106, the examplemodel parameter determiner 108, theexample calibration engine 110, theexample user interface 112, theexample memory 114, the example calibratedweights determiner 130, the example calibratedreporting data determiner 134, and/or the examplecalibration model validator 138 ofFIG. 1 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example criteria-basedinput determiner 102, the example unit-basedinput determiner 104, theexample matrix determiner 106, the examplemodel parameter determiner 108, theexample calibration engine 110, theexample user interface 112, theexample memory 114, the example calibratedweights determiner 130, the example calibratedreporting data determiner 134, and/or the examplecalibration model validator 138 ofFIG. 1 is/are hereby expressly defined to include a tangible computer-readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the software and/or firmware. Further still, theexample calibration apparatus 100 ofFIG. 1 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIG. 1, and/or may include more than one of any or all of the illustrated elements, processes and devices. - Flowcharts representative of example machine readable instructions which may be executed to calibrate data using relaxed benchmark constraints are shown in
FIGS. 2-6 . In these examples, the machine-readable instructions may implement one or more program(s) for execution by a processor such as theexample processor 702 shown in theexample processor platform 700 discussed below in connection withFIG. 7 . The one or more program(s) may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with theprocessor 702 ofFIG. 7 , but the entire program(s) and/or parts thereof could alternatively be executed by a device other than theprocessor 702 ofFIG. 7 , and/or embodied in firmware or dedicated hardware. Further, although the example program(s) is/are described with reference to the flowcharts illustrated inFIGS. 2-6 , many other methods for calibrating data using relaxed benchmark constrains man alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. - As mentioned above, the example instructions of
FIGS. 2-6 may be stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term “tangible computer readable storage medium” is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, “tangible computer readable storage medium” and “tangible machine readable storage medium” are used interchangeably. Additionally or alternatively, the example instructions ofFIGS. 2-6 may be stored on a non-transitory computer and/or machine-readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term “non-transitory computer readable medium” is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, when the phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended. -
FIG. 2 is a flowchart representative of example machinereadable instructions 200 that may be executed at theexample calibration apparatus 100 ofFIG. 1 to calibrate data using relaxed benchmark constraints. Theexample program 200 ofFIG. 2 may be implemented to calibrate panelist and/or reporting data using relaxed benchmark constraints incorporated into a soft calibration model (e.g., the example soft calibration model of Equation 17 or the example soft calibration model of Equation 18 described above). - The
example program 200 ofFIG. 2 begins when the example criteria-basedinput determiner 102 ofFIG. 1 determines criteria-based input data for a target loss function of the soft calibration model (block 202). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify the example criteria-basedinput data 118 ofFIG. 1 by accessing and/or obtaining the criteria-basedinput data 118 from theexample memory 114 ofFIG. 1 , and/or by receiving the criteria-basedinput data 118 from theexample user interface 112 ofFIG. 1 . In some examples, the criteria-basedinput data 118 determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may include criteria associated with the example panelist and/orreporting data 116 ofFIG. 1 , criteria variables for the criteria, targets for the criteria variables, lower bounds for the criteria variables, upper bounds for the criteria variables, scaling parameters for the criteria variables, and/or importance parameters for the criteria variables. An example process that may be used to implementblock 202 is further described below in connection withFIG. 3 . Followingblock 202, control of theexample program 200 ofFIG. 2 proceeds to block 204. - At
block 204, the example unit-basedinput determiner 104 ofFIG. 1 determines unit-based input data for a weight loss function of the soft calibration model (block 204). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify the example unit-basedinput data 122 ofFIG. 1 by accessing and/or obtaining the unit-basedinput data 122 from theexample memory 114 ofFIG. 1 , and/or by receiving the unit-basedinput data 122 from theexample user interface 112 ofFIG. 1 . In some examples, the unit-basedinput data 122 determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may include units associated with the example panelist and/orreporting data 116 ofFIG. 1 , unit variables for the units, initial weights for the unit variables, lower bounds for the unit variables, upper bounds for the unit variables, and/or size parameters for the unit variables. An example process that may be used to implementblock 204 is further described below in connection withFIG. 4 . Followingblock 204, control of theexample program 200 ofFIG. 2 proceeds to block 206. - At
block 206, theexample matrix determiner 106 ofFIG. 1 determines matrix data for the soft calibration model (block 206). For example, thematrix determiner 106 ofFIG. 1 may determine and/or identify theexample matrix data 126 ofFIG. 1 by accessing and/or obtaining thematrix data 126 from theexample memory 114 ofFIG. 1 , and/or by receiving thematrix data 126 from theexample user interface 112 ofFIG. 1 . In some examples, thematrix determiner 106 ofFIG. 1 determines and/or identifies thematrix data 126 based on the example criteria-basedinput data 118 ofFIG. 1 and the example unit-basedinput data 122 ofFIG. 1 . In some examples, thematrix data 126 determined and/or identified by thematrix determiner 106 ofFIG. 1 may be expressed in a manner consistent with Equation 14 and/or Equation 15 described above. Followingblock 206, control of theexample program 200 ofFIG. 2 proceeds to block 208. - At
block 208, the examplemodel parameter determiner 108 ofFIG. 1 determines model parameter data for the soft calibration model (block 208). For example, themodel parameter determiner 108 ofFIG. 1 may determine and/or identify the examplemodel parameter data 128 ofFIG. 1 by accessing and/or obtaining themodel parameter data 128 from theexample memory 114 ofFIG. 1 , and/or by receiving themodel parameter data 128 from theexample user interface 112 ofFIG. 1 . In some examples, themodel parameter data 128 to be determined and/or identified by themodel parameter determiner 108 ofFIG. 1 may include a Huber function parameter, a budget parameter, and/or a stability parameter. An example process that may be used to implementblock 208 is further described below in connection withFIG. 5 . Followingblock 208, control of theexample program 200 ofFIG. 2 proceeds to block 210. - At
block 210, theexample calibration engine 110 ofFIG. 1 executes the soft calibration model to determine calibrated weights based on the criteria-based input data, the target loss function, the unit-based input data, the weight loss function, the design matrix, and the model parameters (block 210). For example, thecalibration engine 110 ofFIG. 1 may execute the soft calibration model based on the example criteria-basedinput data 118, the exampletarget loss function 120, the example unit-basedinput data 122, the exampleweight loss function 124, theexample matrix data 126, and the examplemodel parameter data 128 ofFIG. 1 . In response to thecalibration engine 110 ofFIG. 1 executing the soft calibration model, the calibrated weights determiner 130 ofFIG. 1 determines the example calibratedweights data 132 ofFIG. 1 based on the criteria-basedinput data 118, thetarget loss function 120, the unit-basedinput data 122, theweight loss function 124, thematrix data 126, and themodel parameter data 128 incorporated into the executed soft calibration model. In some examples, the calibrated weights determiner 130 ofFIG. 1 may determine and/or calculate the calibratedweights data 132 based on adjusted calibration weights output by the soft calibration model. - In some examples, the
model parameter data 128 incorporated into the soft calibration model to be executed atblock 210 includes a stability parameter. In other examples, themodel parameter data 128 incorporated into the soft calibration model to be executed atblock 210 may lack a stability parameter. An example process that may be used to implement block 210 when themodel parameter data 128 incorporated into the soft calibration model may lack a stability parameter is further described below in connection withFIG. 6 . Followingblock 210, control of theexample program 200 ofFIG. 2 proceeds to block 212. - At
block 212, the example calibratedreporting data determiner 134 ofFIG. 1 determines calibrated panelist and/or reporting data based on initial panelist and/or reporting data and the calibrated weights (block 212). For example, the calibratedreporting data determiner 134 ofFIG. 1 may determine and/or calculate the example calibrated panelist and/orreporting data 136 ofFIG. 1 by applying the example calibratedweights data 132 ofFIG. 1 determined and/or calculated by the calibrated weights determiner 130 ofFIG. 1 to the example panelist and/orreporting data 116 ofFIG. 1 . Followingblock 212, control of theexample program 200 ofFIG. 2 proceeds to block 214. - At
block 214, theexample user interface 112 ofFIG. 1 presents the calibrated weights, and/or presents the calibrated panelist and/or reporting data, for display (block 214). For example, theuser interface 112 ofFIG. 1 may present (e.g., display) the example calibratedweights data 132 ofFIG. 1 determined and/or calculated by the calibrated weights determiner 130 ofFIG. 1 . Theuser interface 112 ofFIG. 1 may additionally and/or alternatively present (e.g., display) the example calibrated panelist and/orreporting data 136 ofFIG. 1 determined and/or calculated by the calibratedreporting data determiner 134 ofFIG. 1 . Followingblock 214, theexample program 200 ofFIG. 2 ends. -
FIG. 3 is a flowchart representative of example machinereadable instructions 202 that may be executed at theexample calibration apparatus 100 ofFIG. 1 to determine criteria-based input data for the target loss function of the soft calibration model to be executed by thecalibration apparatus 100. Example operations ofblocks FIG. 3 may be used to implement block 202 ofFIG. 2 . - The
example program 202 ofFIG. 3 begins when the example criteria-basedinput determiner 102 ofFIG. 1 determines criteria associated with panelist and/or reporting data (block 302). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify criteria associated the example panelist and/orreporting data 116 ofFIG. 1 by accessing and/or obtaining the criteria from theexample memory 114 ofFIG. 1 , and/or by receiving the criteria from theexample user interface 112 ofFIG. 1 . In some examples, the criteria determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 1 described above. Followingblock 302, control of theexample program 202 ofFIG. 3 proceeds to block 304. - At
block 304, the example criteria-basedinput determiner 102 ofFIG. 1 determines criteria variables for the criteria (block 304). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify criteria variables for the criteria by accessing and/or obtaining the criteria variables from theexample memory 114 ofFIG. 1 , and/or by receiving the criteria variables from theexample user interface 112 ofFIG. 1 . In some examples, the criteria variables determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 2 described above. Followingblock 304, control of theexample program 202 ofFIG. 3 proceeds to block 306. - At
block 306, the example criteria-basedinput determiner 102 ofFIG. 1 determines targets for the criteria variables (block 306). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify targets for the criteria variables by accessing and/or obtaining the targets from theexample memory 114 ofFIG. 1 , and/or by receiving the targets from theexample user interface 112 ofFIG. 1 . In some examples, the targets determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 3 described above. Followingblock 306, control of theexample program 202 ofFIG. 3 proceeds to block 308. - At
block 308, the example criteria-basedinput determiner 102 ofFIG. 1 determines lower bounds for the criteria variables (block 308). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify lower bounds for the criteria variables by accessing and/or obtaining the lower bounds from theexample memory 114 ofFIG. 1 , and/or by receiving the lower bounds from theexample user interface 112 ofFIG. 1 . In some examples, the lower bounds determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 4 described above. Followingblock 308, control of theexample program 202 ofFIG. 3 proceeds to block 310. - At
block 310, the example criteria-basedinput determiner 102 ofFIG. 1 determines upper bounds for the criteria variables (block 310). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify upper bounds for the criteria variables by accessing and/or obtaining the upper bounds from theexample memory 114 ofFIG. 1 , and/or by receiving the upper bounds from theexample user interface 112 ofFIG. 1 . In some examples, the upper bounds determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 4 described above. Followingblock 310, control of theexample program 202 ofFIG. 3 proceeds to block 312. - At
block 312, the example criteria-basedinput determiner 102 ofFIG. 1 determines scaling parameters for the criteria variables (block 312). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify scaling parameters for the criteria variables by accessing and/or obtaining the scaling parameters from theexample memory 114 ofFIG. 1 , and/or by receiving the scaling parameters from theexample user interface 112 ofFIG. 1 . In some examples, the scaling parameters determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 5 described above. Followingblock 312, control of theexample program 202 ofFIG. 3 proceeds to block 314. - At
block 314, the example criteria-basedinput determiner 102 ofFIG. 1 determines importance parameters for the criteria variables (block 314). For example, the criteria-basedinput determiner 102 ofFIG. 1 may determine and/or identify importance parameters for the criteria variables by accessing and/or obtaining the importance parameters from theexample memory 114 ofFIG. 1 , and/or by receiving the importance parameters from theexample user interface 112 ofFIG. 1 . In some examples, the importance parameters determined and/or identified by the criteria-basedinput determiner 102 ofFIG. 1 may be expressed in a manner consistent with Equation 6 described above. Followingblock 314, theexample program 202 ofFIG. 3 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . -
FIG. 4 is a flowchart representative of example machinereadable instructions 204 that may be executed at theexample calibration apparatus 100 ofFIG. 1 to determine unit-based input data for the weight loss function of the soft calibration model to be executed by thecalibration apparatus 100. Example operations ofblocks FIG. 4 may be used to implement block 204 ofFIG. 2 . - The
example program 204 ofFIG. 4 begins when the example unit-basedinput determiner 104 ofFIG. 1 determines units associated with panelist and/or reporting data (block 402). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify units associated the example panelist and/orreporting data 116 ofFIG. 1 by accessing and/or obtaining the units from theexample memory 114 ofFIG. 1 , and/or by receiving the units from theexample user interface 112 ofFIG. 1 . In some examples, the units determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 8 described above. Followingblock 402, control of theexample program 204 ofFIG. 4 proceeds to block 404. - At
block 404, the example unit-basedinput determiner 104 ofFIG. 1 determines unit variables for the units (block 404). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify unit variables for the units by accessing and/or obtaining the unit variables from theexample memory 114 ofFIG. 1 , and/or by receiving the unit variables from theexample user interface 112 ofFIG. 1 . In some examples, the unit variables determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 9 described above. Followingblock 404, control of theexample program 204 ofFIG. 4 proceeds to block 406. - At
block 406, the example unit-basedinput determiner 104 ofFIG. 1 determines initial weights for the unit variables (block 406). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify initial weights for the unit variables by accessing and/or obtaining the initial weights from theexample memory 114 ofFIG. 1 , and/or by receiving the initial weights from theexample user interface 112 ofFIG. 1 . In some examples, the initial weights determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 10 described above. Followingblock 406, control of theexample program 204 ofFIG. 4 proceeds to block 408. - At
block 408, the example unit-basedinput determiner 104 ofFIG. 1 determines lower bounds for the unit variables (block 408). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify lower bounds for the unit variables by accessing and/or obtaining the lower bounds from theexample memory 114 ofFIG. 1 , and/or by receiving the lower bounds from theexample user interface 112 ofFIG. 1 . In some examples, the lower bounds determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 11 described above. Followingblock 408, control of theexample program 204 ofFIG. 4 proceeds to block 410. - At
block 410, the example unit-basedinput determiner 104 ofFIG. 1 determines upper bounds for the unit variables (block 410). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify upper bounds for the unit variables by accessing and/or obtaining the upper bounds from theexample memory 114 ofFIG. 1 , and/or by receiving the upper bounds from theexample user interface 112 ofFIG. 1 . In some examples, the upper bounds determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 11 described above. Followingblock 410, control of theexample program 204 ofFIG. 4 proceeds to block 412. - At
block 412, the example unit-basedinput determiner 104 ofFIG. 1 determines size parameters for the unit variables (block 412). For example, the unit-basedinput determiner 104 ofFIG. 1 may determine and/or identify size parameters for the unit variables by accessing and/or obtaining the size parameters from theexample memory 114 ofFIG. 1 , and/or by receiving the size parameters from theexample user interface 112 ofFIG. 1 . In some examples, the size parameters determined and/or identified by the unit-basedinput determiner 104 ofFIG. 1 may be expressed in a manner consistent with Equation 12 described above. Followingblock 412, theexample program 204 ofFIG. 4 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . -
FIG. 5 is a flowchart representative of example machinereadable instructions 208 that may be executed at theexample calibration apparatus 100 ofFIG. 1 to determine model parameters for the soft calibration model to be executed by thecalibration apparatus 100. Example operations ofblocks FIG. 5 may be used to implement block 208 ofFIG. 2 . - The
example program 208 ofFIG. 5 begins when the examplemodel parameter determiner 108 ofFIG. 1 determines a Huber function parameter for the soft calibration model (block 502). For example, themodel parameter determiner 108 ofFIG. 1 may determine and/or identify a Huber function parameter for the soft calibration model by accessing and/or obtaining the Huber function parameter from theexample memory 114 ofFIG. 1 , and/or by receiving the Huber function parameter from theexample user interface 112 ofFIG. 1 . Followingblock 502, control of theexample program 208 ofFIG. 5 proceeds to block 504. - At
block 504, the examplemodel parameter determiner 108 ofFIG. 1 determines a budget parameter for the soft calibration model (block 504). For example, themodel parameter determiner 108 ofFIG. 1 may determine and/or identify a budget parameter for the soft calibration model by accessing and/or obtaining the budget parameter from theexample memory 114 ofFIG. 1 , and/or by receiving the budget parameter from theexample user interface 112 ofFIG. 1 . Followingblock 504, control of theexample program 208 ofFIG. 5 proceeds to block 506. - At
block 506, the examplemodel parameter determiner 108 ofFIG. 1 determines a stability parameter for the soft calibration model (block 506). For example, themodel parameter determiner 108 ofFIG. 1 may determine and/or identify a stability parameter for the soft calibration model by accessing and/or obtaining the stability parameter from theexample memory 114 ofFIG. 1 , and/or by receiving the stability parameter from theexample user interface 112 ofFIG. 1 . Followingblock 506, theexample program 208 ofFIG. 5 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . -
FIG. 6 is a flowchart representative of example machinereadable instructions 210 that may be executed at theexample calibration apparatus 100 ofFIG. 1 to validate the soft calibration model executed by the calibration apparatus. Example operations ofblocks FIG. 6 may be used to implement block 210 ofFIG. 2 in instances when themodel parameter data 128 incorporated into the soft calibration model may lack a stability parameter. - The
example program 210 ofFIG. 6 begins when the examplecalibration model validator 138 ofFIG. 1 determines whether the soft calibration model executed by theexample calibration engine 110 ofFIG. 1 includes a stability parameter (block 602). For example, thecalibration model validator 138 ofFIG. 1 may determine that a soft calibration model expressed in a manner consistent with Equation 17 described above does not include a stability parameter. If thecalibration model validator 138 determines atblock 602 that the soft calibration model executed by thecalibration engine 110 ofFIG. 1 does not include a stability parameter, control of theexample program 210 ofFIG. 6 proceeds to block 604. If thecalibration model validator 138 instead determines atblock 602 that the soft calibration model executed by thecalibration engine 110 ofFIG. 1 does include a stability parameter, theexample program 210 ofFIG. 6 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . - At
block 604, the examplecalibration model validator 138 ofFIG. 1 determines whether the calibrated weights resulting from the soft calibration model being executed by theexample calibration engine 110 ofFIG. 1 provide a unique solution for the executed soft calibration model (block 604). If thecalibration model validator 138 determines atblock 604 that the calibrated weights resulting from the soft calibration model being executed by theexample calibration engine 110 ofFIG. 1 do not provide a unique solution for the executed soft calibration model, control of theexample program 210 ofFIG. 6 proceeds to block 606. If thecalibration model validator 138 instead determines atblock 604 that the calibrated weights resulting from the soft calibration model being executed by theexample calibration engine 110 ofFIG. 1 do provide a unique solution for the executed soft calibration model, theexample program 210 ofFIG. 6 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . - At
block 606, the examplecalibration model validator 138 ofFIG. 1 incorporates a stability parameter into the soft calibration model (block 606). For example, thecalibration model validator 138 ofFIG. 1 may incorporate a stability parameter expressed in a manner consistent with Equation 16 described above into the soft calibration model expressed in a manner consistent with Equation 17 described above to provide the soft calibration model expressed in a manner consistent with Equation 18 described above (e.g., a soft calibration model including a stability parameter). Followingblock 606, control of theexample program 210 ofFIG. 6 proceeds to block 608. - At
block 608, theexample calibration engine 110 ofFIG. 1 re-executes the soft calibration model to re-determine the calibrated weights (block 608). For example, thecalibration engine 110 ofFIG. 1 may re-execute the soft calibration model based on the example criteria-basedinput data 118, the exampletarget loss function 120, the example unit-basedinput data 122, the exampleweight loss function 124, theexample matrix data 126, and the examplemodel parameter data 128 ofFIG. 1 , including the stability parameter. In response to thecalibration engine 110 ofFIG. 1 executing the soft calibration model, the calibrated weights determiner 130 ofFIG. 1 determines the example calibratedweights data 132 ofFIG. 1 based on the criteria-basedinput data 118, thetarget loss function 120, the unit-basedinput data 122, theweight loss function 124, thematrix data 126, and themodel parameter data 128 incorporated into the executed soft calibration model, including the stability parameter. The calibrated weights determined (e.g., re-determined) atblock 608 provide a unique solution for the re-executed soft calibration model. Followingblock 608, theexample program 210 ofFIG. 6 ends and control returns to a calling function or process such as theexample program 200 ofFIG. 2 . -
FIG. 7 is anexample processor platform 700 capable of executing the instructions ofFIGS. 2-6 to implement theexample calibration apparatus 100 ofFIG. 1 . Theprocessor platform 700 of the illustrated example includes aprocessor 702. Theprocessor 702 of the illustrated example is hardware. For example, theprocessor 702 can be implemented by one or more integrated circuit(s), logic circuit(s), controller(s), microcontroller(s) and/or microprocessor(s) from any desired family or manufacturer. Theprocessor 702 of the illustrated example includes a local memory 704 (e.g., a cache). Theprocessor 702 of the illustrated example also includes the example criteria-basedinput determiner 102, the example unit-basedinput determiner 104, theexample matrix determiner 106, the examplemodel parameter determiner 108, theexample calibration engine 110, the example calibratedweights determiner 130, the example calibratedreporting data determiner 134, and the examplecalibration model validator 138 ofFIG. 1 . - The
processor 702 of the illustrated example is also in communication with a main memory including avolatile memory 706 and anon-volatile memory 708 via abus 710. Thevolatile memory 706 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any other type of random access memory device. Thenon-volatile memory 708 may be implemented by flash memory and/or any other desired type of memory device. Access to thevolatile memory 706 and thenon-volatile memory 708 is controlled by a memory controller. - The
processor 702 of the illustrated example is also in communication with one or more mass storage device(s) 712 for storing software and/or data. Examples of suchmass storage devices 712 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives. In the illustrated example ofFIG. 7 , themass storage device 712 includes theexample memory 114 ofFIG. 1 . - The
processor platform 700 of the illustrated example also includes a user interface circuit 714. The user interface circuit 714 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface. In the illustrated example, one or more input device(s) 140 are connected to the user interface circuit 714. The input device(s) 140 permit(s) a user to enter data and commands into theprocessor 702. The input device(s) 140 can be implemented by, for example, an audio sensor, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint, a voice recognition system, a microphone, and/or a liquid crystal display. One or more output device(s) 142 are also connected to the user interface circuit 714 of the illustrated example. The output device(s) 142 can be implemented, for example, by a light emitting diode, an organic light emitting diode, a liquid crystal display, a touchscreen and/or a speaker. The user interface circuit 714 of the illustrated example may, thus, include a graphics driver such as a graphics driver chip and/or processor. In the illustrated example, the input device(s) 140, the output device(s) 142 and the user interface circuit 714 collectively form theexample user interface 112 ofFIG. 1 . - The
processor platform 700 of the illustrated example also includes anetwork interface circuit 716. Thenetwork interface circuit 716 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface. In the illustrated example, thenetwork interface circuit 716 facilitates the exchange of data and/or signals with external machines (e.g., a remote server) via a network 718 (e.g., a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), the Internet, a cellular network, etc.). -
Coded instructions 720 corresponding toFIGS. 2-6 may be stored in thelocal memory 704, in thevolatile memory 706, in thenon-volatile memory 708, in themass storage device 712, and/or on a removable tangible computer readable storage medium such as a flash memory stick, a CD or DVD. - From the foregoing, it will be appreciated that methods, systems and apparatus have been disclosed for calibrating data (e.g., panelist and/or reporting data) using relaxed benchmark constraints. Unlike hard calibration models and/or techniques requiring that estimated totals associated with projection weights match reference and/or benchmark totals precisely, the disclosed soft calibration models and/or techniques advantageously generate adjusted and/or modified projection weights such that the distance and/or deviation between the estimated totals and the reference and/or benchmark totals is smaller than some particular value (e.g., a threshold and/or budget). By relaxing the benchmark constraints associated with traditional hard calibration models and/or techniques, the disclosed soft calibration models and/or techniques advantageously reduce (e.g., eliminate) the above-described errors that may arise in connection with the use of hard calibration models and/or techniques when the calibration weights take on a relatively large magnitude. Thus, a data measurement company implementing the disclosed soft calibration models and/or techniques in lieu of traditional hard calibration models and/or techniques may advantageously provide and/or generate calibrated panelist and/or reporting data that more closely matches and/or resembles reference and/or benchmark data.
- Apparatus for generating a unique solution when calibrating data via a calibration model having relaxed benchmark constraints are disclosed. In some disclosed examples, the apparatus comprises a calibration engine to execute the calibration model based on a target loss function, a weight loss function, and a budget parameter. In some disclosed examples, the apparatus further comprises a calibrated weights determiner to determine calibrated weights resulting from execution of the calibration model. In some disclosed examples, the apparatus further comprises a calibration model validator to incorporate a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model. In some disclosed examples, the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- In some disclosed examples, the calibration engine is to re-execute the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter. In some disclosed examples, the calibrated weights determiner is to re-determine the calibrated weights resulting from re-execution of the calibration model. In some disclosed examples, the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- In some disclosed examples, the calibration engine is to execute the calibration model based further on criteria-based input data to be incorporated into the target loss function and unit-based input data to be incorporated into the weight loss function. In some disclosed examples, the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables. In some disclosed examples, the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variable. In some disclosed examples, the calibration engine is to execute the calibration model based further on matrix data. In some disclosed examples, the matrix data is based on the criteria-based input data and the unit-based input data.
- Methods for generating a unique solution when calibrating data via a calibration model having relaxed benchmark constraints are disclosed. In some disclosed examples, the method comprises executing the calibration model based on a target loss function, a weight loss function, and a budget parameter. In some disclosed examples, the method further comprises determining, by executing one or more computer readable instructions with a processor, calibrated weights resulting from the executing of the calibration model. In some disclosed examples, the method further comprises incorporating, by executing one or more computer readable instructions with the processor, a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model. In some disclosed examples, the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- In some disclosed examples, the method further includes re-executing the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter. In some disclosed examples, the method further includes re-determining the calibrated weights resulting from the re-executing of the calibration model. In some disclosed examples, the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- In some disclosed examples, the executing of the calibration model is further based on criteria-based input data incorporated into the target loss function and unit-based input data incorporated into the weight loss function. In some disclosed examples, the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables. In some disclosed examples, the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variables. In some disclosed examples, the executing of the calibration model is further based on matrix data. In some disclosed examples, the matrix data is based on the criteria-based input data and the unit-based input data.
- Tangible machine-readable storage media comprising instructions are also disclosed. In some disclosed examples, the instructions, when executed, cause a processor to execute a calibration model based on a target loss function, a weight loss function, and a budget parameter. In some disclosed examples, the calibration model has relaxed benchmark constraints. In some disclosed examples, the instructions, when executed, cause the processor to determine calibrated weights resulting from the execution of the calibration model. In some disclosed examples, the instructions, when executed, cause the processor to incorporate a stability parameter into the calibration model in response to determining that the calibrated weights do not provide a unique solution for the executed calibration model. In some disclosed examples, the stability parameter is to reduce an influence of the budget parameter on the calibration model to enable the generation of a unique solution.
- In some disclosed examples, the instructions, when executed, are further to cause the processor to re-execute the calibration model based on the target loss function, the weight loss function, the budget parameter, and the stability parameter. In some disclosed examples, the instructions, when executed, are further to cause the processor to re-determine the calibrated weights resulting from the re-execution of the calibration model. In some disclosed examples, the re-determined calibrated weights are to provide a unique solution for the re-executed calibration model.
- In some disclosed examples, the instructions, when executed, are to cause the processor to execute the calibration model based further on criteria-based input data incorporated into the target loss function and unit-based input data incorporated into the weight loss function. In some disclosed examples, the criteria-based input data includes criteria variables, targets for the criteria variables, upper and lower bounds for the criteria variables, scaling parameters for the criteria variables, and importance parameters for the criteria variables. In some disclosed examples, the unit-based input data includes unit variables, initial weights for the unit variables, upper and lower bounds for the unit variables, and size parameters for the unit variables. In some disclosed examples, the instructions, when executed, are to cause the processor to execute the calibration model based further on matrix data. In some disclosed examples, the matrix data is based on the criteria-based input data and the unit-based input data.
- Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/745,608 US20220277244A1 (en) | 2016-06-07 | 2022-05-16 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662346897P | 2016-06-07 | 2016-06-07 | |
US201662378041P | 2016-08-22 | 2016-08-22 | |
US201662379205P | 2016-08-24 | 2016-08-24 | |
US201662379517P | 2016-08-25 | 2016-08-25 | |
US15/583,337 US10776728B1 (en) | 2016-06-07 | 2017-05-01 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
US16/942,429 US11361264B2 (en) | 2016-06-07 | 2020-07-29 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
US17/745,608 US20220277244A1 (en) | 2016-06-07 | 2022-05-16 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/942,429 Continuation US11361264B2 (en) | 2016-06-07 | 2020-07-29 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
Publications (1)
Publication Number | Publication Date |
---|---|
US20220277244A1 true US20220277244A1 (en) | 2022-09-01 |
Family
ID=72425507
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/583,337 Active 2039-06-01 US10776728B1 (en) | 2016-06-07 | 2017-05-01 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
US16/942,429 Active 2037-07-12 US11361264B2 (en) | 2016-06-07 | 2020-07-29 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
US17/745,608 Pending US20220277244A1 (en) | 2016-06-07 | 2022-05-16 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/583,337 Active 2039-06-01 US10776728B1 (en) | 2016-06-07 | 2017-05-01 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
US16/942,429 Active 2037-07-12 US11361264B2 (en) | 2016-06-07 | 2020-07-29 | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints |
Country Status (1)
Country | Link |
---|---|
US (3) | US10776728B1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190095557A1 (en) * | 2017-09-15 | 2019-03-28 | Credit Suisse Securities (Usa) Llc | Modelling apparatuses, methods, and systems |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2354387B (en) * | 1996-09-09 | 2001-05-09 | Dennis J Dupray | Wireless location using multiple simultaneous location estimators |
EP1066582A4 (en) | 1998-03-20 | 2004-10-20 | Mcgraw Hill Companies Inc | System, method, and computer program product for assessing risk within a predefined market |
US7512542B1 (en) | 1999-06-10 | 2009-03-31 | A.C. Nielsen (Us), Inc. | Method and system for market research data mining |
WO2001069329A2 (en) * | 2000-03-10 | 2001-09-20 | Cyrano Sciences, Inc. | Control for an industrial process using one or more multidimensional variables |
US7287000B2 (en) * | 2000-11-15 | 2007-10-23 | Jda Software Group, Inc. | Configurable pricing optimization system |
WO2005020031A2 (en) | 2003-08-22 | 2005-03-03 | Mastercard International Incorporated | Methods and systems for predicting business behavior from profiling consumer card transactions |
US20050096950A1 (en) * | 2003-10-29 | 2005-05-05 | Caplan Scott M. | Method and apparatus for creating and evaluating strategies |
US7873529B2 (en) | 2004-02-20 | 2011-01-18 | Symphonyiri Group, Inc. | System and method for analyzing and correcting retail data |
US20080288889A1 (en) | 2004-02-20 | 2008-11-20 | Herbert Dennis Hunt | Data visualization application |
US20090006156A1 (en) | 2007-01-26 | 2009-01-01 | Herbert Dennis Hunt | Associating a granting matrix with an analytic platform |
US7580852B2 (en) | 2004-04-15 | 2009-08-25 | Sap Ag | System and method for modeling non-stationary time series using a non-parametric demand profile |
MX2007001251A (en) | 2004-07-30 | 2008-02-14 | Nielsen Media Res Inc | Methods and apparatus for improving the accuracy and reach of electronic media exposure measurement systems. |
WO2006058274A2 (en) | 2004-11-29 | 2006-06-01 | Arbitron Inc. | Systems and processes for use in media and/or market research |
US20060149616A1 (en) * | 2005-01-05 | 2006-07-06 | Hildick-Smith Peter G | Systems and methods for forecasting book demand |
US20080294996A1 (en) | 2007-01-31 | 2008-11-27 | Herbert Dennis Hunt | Customized retailer portal within an analytic platform |
EP2168089A4 (en) | 2007-06-14 | 2020-09-09 | The Nielsen Company (US), LLC | Methods and apparatus to weight incomplete respondent data |
JP2010002370A (en) | 2008-06-23 | 2010-01-07 | Fujitsu Ltd | Pattern extraction program, technique, and apparatus |
US20120123372A1 (en) * | 2010-11-12 | 2012-05-17 | David Harry Melik | Elastomeric compositions that resist force loss and disintegration |
US20140089051A1 (en) | 2012-09-25 | 2014-03-27 | Frank Piotrowski | Methods and apparatus to align panelist data with retailer sales data |
US8797668B1 (en) | 2013-03-13 | 2014-08-05 | Lsi Corporation | Systems and methods for penalty based multi-variant encoding |
US10275545B2 (en) * | 2013-03-15 | 2019-04-30 | Ventana Systems, Inc. | Modeling and simulation |
US9306600B2 (en) | 2014-01-06 | 2016-04-05 | Micron Technology, Inc. | Read threshold calibration for LDPC |
US9953330B2 (en) | 2014-03-13 | 2018-04-24 | The Nielsen Company (Us), Llc | Methods, apparatus and computer readable media to generate electronic mobile measurement census data |
US10496927B2 (en) * | 2014-05-23 | 2019-12-03 | DataRobot, Inc. | Systems for time-series predictive data analytics, and related methods and apparatus |
US10366346B2 (en) * | 2014-05-23 | 2019-07-30 | DataRobot, Inc. | Systems and techniques for determining the predictive value of a feature |
US10558924B2 (en) * | 2014-05-23 | 2020-02-11 | DataRobot, Inc. | Systems for second-order predictive data analytics, and related methods and apparatus |
RU2705012C1 (en) * | 2015-11-03 | 2019-11-01 | ВЕЗЕРФОРД ТЕКНОЛОДЖИ ХОЛДИНГЗ, ЭлЭлСи | Systems and methods for estimating and optimizing efficiency of stimulation using bypass devices |
US10221649B2 (en) * | 2015-11-03 | 2019-03-05 | Weatherford Technology Holdings, Llc | Systems and methods for intelligent diversion design and application |
US10360517B2 (en) * | 2017-02-22 | 2019-07-23 | Sas Institute Inc. | Distributed hyperparameter tuning system for machine learning |
-
2017
- 2017-05-01 US US15/583,337 patent/US10776728B1/en active Active
-
2020
- 2020-07-29 US US16/942,429 patent/US11361264B2/en active Active
-
2022
- 2022-05-16 US US17/745,608 patent/US20220277244A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US10776728B1 (en) | 2020-09-15 |
US20210012259A1 (en) | 2021-01-14 |
US11361264B2 (en) | 2022-06-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Plagborg‐Møller | Bayesian inference on structural impulse response functions | |
Khare et al. | A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees | |
Fushiki | Estimation of prediction error by using K-fold cross-validation | |
Fritz et al. | A comparison of algorithms for the multivariate L 1-median | |
Dogan | Bootstrapping for confidence interval estimation and hypothesis testing for parameters of system dynamics models | |
US20220277244A1 (en) | Methods, systems and apparatus for calibrating data using relaxed benchmark constraints | |
Giuzio et al. | Sparse and robust normal and t-portfolios by penalized Lq-likelihood minimization | |
Bekiros et al. | Bayesian forecasting with small and medium scale factor-augmented vector autoregressive DSGE models | |
Escanciano et al. | Nonparametric euler equation identification and estimation | |
Duan et al. | Non-Gaussian bridge sampling with an application | |
Woźniak | Testing causality between two vectors in multivariate GARCH models | |
Juselius et al. | Balassa-Samuelson and wage, price and unemployment dynamics in the Spanish transition to EMU membership | |
Hallin et al. | A simple R-estimation method for semiparametric duration models | |
Guolo | Pseudo-likelihood inference for regression models with misclassified and mismeasured variables | |
Aeberhard et al. | Saddlepoint tests for accurate and robust inference on overdispersed count data | |
de Silva et al. | Modeling and analysis of functional method comparison data | |
Wu et al. | Calibrating generalized predictive distributions | |
Luo et al. | Markov chain Monte Carlo estimation of default and recovery: dependent via the latent systematic factor | |
Bee et al. | Approximate maximum likelihood estimation of the Bingham distribution | |
Hainy et al. | Likelihood-free simulation-based optimal design | |
Holden | Estimation of dynamic models with occasionally binding constraints. | |
Pal et al. | A bayesian approach for analyzing data on the stiefel manifold | |
JP2012173537A (en) | Feature amount enhancement device, feature amount enhancement method, and program therefor | |
Plagborg-Møller | Supplement to “Bayesian inference on structural impulse response functions” | |
Holden | Tractable estimation and smoothing of highly non-linear dynamic state-space models |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: THE NIELSEN COMPANY (US), LLC, NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DAEMEN, LUDO;SMITH, ROBERT C.;WONG, EDMOND;AND OTHERS;SIGNING DATES FROM 20160711 TO 20161004;REEL/FRAME:059946/0298 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063560/0547 Effective date: 20230123 |
|
AS | Assignment |
Owner name: CITIBANK, N.A., NEW YORK Free format text: SECURITY INTEREST;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063561/0381 Effective date: 20230427 |
|
AS | Assignment |
Owner name: ARES CAPITAL CORPORATION, NEW YORK Free format text: SECURITY INTEREST;ASSIGNORS:GRACENOTE DIGITAL VENTURES, LLC;GRACENOTE MEDIA SERVICES, LLC;GRACENOTE, INC.;AND OTHERS;REEL/FRAME:063574/0632 Effective date: 20230508 |