Demystify the KPIs in cross selling “support, confidence and lift”!
Since the early 90th Rakesh Agrawal and others developed “fast algorithms for mining association rules in large databases” (http://en.wikipedia.org/wiki/Apriori_algorithm). So discovering association rules and large itemsets in market basket analysis is around for quite a long time. IBM SPSS is one tool having 2 additional algorithms (carma, sequence) beside the aprioi algorithm in place to do “specific” market basket analysis (MBA) (http://www-01.ibm.com/software/analytics/spss/products/modeler/index.html).
Algorithm and formula usage (cross selling analysis itself) and cross selling KPI generating is one important subject but bridging the results into retail LOBs thinking and actions is another important part.
One challenge in using cross-selling expertise systematically in retail is continuous experimental optimization (or doing better over time) with focus on cross selling measures and their practical day-to-day use on the Line-of-Business (LOB) level. Successful Category managers, store managers or shelf planner in retail have to get familiar with the core measures of cross-selling.
In order to maximize consumer satisfaction and category profitability the range planning is the process of determining an optimal product offering, within a category and “across” (when allowing cross selling to be used creative and active across established categories) and the space planning and optimization provides the link between assortment management and store execution of merchandise presentation.
Having a look on the analytic algorithms for cross selling like apriori or carma algorithms you cannot expect a retail expert to use or to get familiar with the formulas and functions described. See an example on “Apriori - Find Frequent Item Sets and Association Rules with the Apriori Algorithm” (http://www.borgelt.net//doc/apriori/apriori.html#suppset).
To work with the analytic (algorithm) output “support, confidence and lift” – the key measures, it is important to follow a 4 step approach to enable LOB experts and to earn acceptance on a working level that triggers actions and that is guiding the way to success for all LOBs involved.
A proposed 4 step approach has to cover:
Step 1: Get the basics right (transactions analyzed = market baskets)
Define what is the foundation defined for the company’s cross-selling analytics (long lasting definitions, only a few changes over time).
Definitions can be or has to cover for each analytical (data mining) run:
1.Time frame (transactions for 4 weeks, including 4 weekends) or other “useful” timeframe
dependent on the retail format like food (hypermarket), non-food (fashion, DIY, electronics) or mixed assortments
2.Store level transactions
possibly in addition with comparable stores / whole company benchmarks
3.SKU (Items) or / and category level or / and whole taxonomy,
without price changes or promo (caution if new SKU no.), quantities per single SKU in baskets
Step 2: Identify Consumer or Sales Hot Spots in the store on item and itemset level
Core input is the measure “support” (analysis or algorithm output) – this measure can be defined and named in different ways if suitable for the LOB experts.
A simple definition can be:
- for support
“Based on all transactions or market baskets (defined in step 1 and analyzed) you find out or see how often – as a percentage or as proportion of all transactions - an item (item “A”, item “B”, item “C”… or an itemset (itemset “A+B”, itemset “A+C”….all relevant combinations of items) is found in the analyzed transactions or market baskets”
These analytics can cover the whole taxonomy like item – item group – category – department etc.
Pic 1: Explaining support for items or itemset
(Note: Consumer, Buyer, Customer are used as synonyms. Antecedents and succedents can be items or large itemset – in graphics rule is used for simplification on level item <> item)
Step 3: Identify Cross-Selling Details and evaluate directed rules (Cross-Selling Rules discovery and understanding)
Core input is the measure “confidence” (analytic or algorithm output) – this measure can be defined and named in different ways as well if suitable and needed for the LOB experts.
A simple definition can be
- for confidence
“Based on transactions or market baskets containing an item “A” ( ! subset of all transactions ! ) you will find an item “B” in X percentage (or as proportion) of the transactions having “A” in basket. Important is the direction you measure the occurrences.”
Example: Based on market baskets containing “Frozenmeal“– Antecedent - (! subset of all transactions !) you will find “Beer” – Succedent - in 56,291 % (or as proportion) of the transactions having “Frozenmeal” in basket. Important is the direction you measure the occurrences (direction or view from frozenmeal to beer).”
This rule is found based on xx no. of market baskets analyzed. The antecedence has an occurrence of 30,2 % (support) and the rule has a support of 17 % based on all baskets (results from Step 2)
Step 4: Identify Cross-Selling Details and do relative evaluation (Cross-Selling Rules discovery and understanding)
Core input is the measure “lift” (analytic or algorithm output) – this measure can be defined and named in different ways as well if suitable and needed for the LOB experts.
A simple definition can be
- for lift:
“Based on the support measured in step 2 for item “B” you expect this item “B” with a measured proportion of X percentage in all baskets (support for item “B”).
To compare this value – support item “B” - with the measured confidence value (step 3 in rules discovery) you will get a factor – called lift – that shows if the confidence (step 3) value is higher (or lower) than the support value (step 2).
Below or above your “expectations” (below or above lift value 1) you will see based on the lift value if the identified rule is “worth” to focus on!
Example (taken from “IBM SPSS Modeler” demo data):
Step 2: Antecedent “Frozenmeal” > Succedent “Beerl” (Example Rule 1, Line 1, below)
>> Antecedent (“Frozenmeal”) Support 30,2 %
>> Succedent (“Beer”) Support 29,3 %
>> Rule or Itemset (“Frozenmeal + Beer”) Support 17,0 %
Step 3: Antecedent “Frozenmeal” > Succedent “Beer”
>> Confidence for Rule (“Frozenmeal > Beer”) 56,291 %
(56,291 % of “Frozenmeal Buyers” are buying “Beer” – direction is important!)
Step 4: Antecedent “Frozenmeal” > Succedent “Beer”
>> Lift (Factor) for Rule (“Frozenmeal > Beer”) 1,921 (above 1 = positive cross selling)
(“Comparing” support of Beer 29,3 % with confidence of rule (Frozenmeal > Beer) = 56,291 %
>> 56,21/29,3 = 1,921 Lift
Example / Pic 4: (Screen shot taken from “IBM SPSS Modeler” – my demo data):
Example (taken from “IBM SPSS Modeler” my demo data, explained):
To be read from Antezedens (Antecedent) –column 2- to Sukzedens (Succedent) –column 1-
Line 1 (rule examples):
>>> 56,291 % of “Frozenmeal Buyers” buy “Beer”, Confidence for this rule
(Beer is found in 29,3 % of all baskets ; Support for Beer to be found in line 2)
>>> “Frozenmeal – Beer” itemset is found in 17 % of all baskets; Support)
>>> Comparing Beer support (29,3 %) with Confidence of rule (56,291 %)
– 56,21/29,3 = 1,921 Lift
>>> “Frozenmeal Buyers” buy “Beer” 1,921 times more often – compared to all buyers
- Translation > German – English < version:
- Unterstützung % (= Support % for Antezedens, Antecedent)
- Konfidenz % (= Confidence %)
- Regelunterstützung % (Support % for Rule)
Example / Pic 5: (Adoption of analysis results in spread-sheet format for LOB users – my own layout)
Derived visualization, transformation, enrichment and formation of cross selling analysis results for LOB in spread-sheet format (further adjustments for visualization should be directed by LOB)
To work on the “simplification” for these core measures: “support, confidence and lift” it is essential to get acceptance in the different LOBs. Even a new naming make sense (example: support = consumer acceptance value / confidence = cross-selling value / lift = cross-selling factor).
Association rule algorithms “automatically find” the associations and more measures than the above described. In addition visualization techniques (even simple graphics) help to identify the essential cross-selling rules. Having 100.000+ rules as the output of a mining run (market basket analysis results) it is key to focus on the set of cross-selling rules to be used for experimental optimization in the store (via placement, promotion etc.) easily and with confident. Category Managers for example will get a completely different and much better view on their managed assortment, merchandising activities or listing strategy when enhancing decision processes with cross selling analysis KPIs.
From my experience adaptation aids with regard to cross selling measures are an important core success factor to get LOBs on the “creative” cross-selling path!