Description:FIELD OF INVENTION
[001] The present invention generally relates to the field of graph based user platforms or interfaces. The present invention specifically relates to a method for measuring the impact of multiple interventions in e-commerce platform. The method measures impact of multiple test programs or interventions of different platform drivers on one or multiple outcome metrics without the need to maintain different hold out populations and interaction guardrails by the product owners.
BACKGROUND OF THE INVENTION
[002] Network-connected software applications (e.g., web applications, e-commerce platforms and hybrid applications) and websites are a valuable resource for many organizations. An E-commerce platform is a platform for providing online transaction negotiation for enterprises or individuals. E-commerce platform can be described as an interconnected network of various nodes comprising of modalities like search, recommendation, cart check out experience, payment method, buy-now button, price, quality, speed, selection, and return. Product or design innovations can be made in different nodes, monetary or non-monetary in nature to drive customer conversion. Product owners often run multiple A/B experiments to measure the impact of the individual interventions and the ones producing significant favorable change in the chosen metrics are prioritized to roll out for the target groups.
[003] However, often the relations between the chosen isolated metrics specific to the individual interventions do not add up to the company’s revenue due to complex mutual interactions among the interventions. For example- more discount on a product may lead to increased demand (especially during sale events) and that may put added pressure on the supply chain resulting in reduced delivery speed. Reduced delivery speed in turn, may negatively impact conversion and customer sentiment. Thus, reduced delivery speed in turn may also increase return to order events. As a result, often the expected individual impact of each of the interventions do not transform into equivalent revenue impact and hence it becomes difficult to measure the contribution of multiple interventions and test programs to the true change in the company’s bottomline. Measuring isolated impacts on chosen target metrics is insufficient to measure the impact of multiple interventions on interconnected nodes.
[004] There are several patent applications that provides A/B testing. One such Patent Application US8296643B1 discloses a Methods, systems and apparatus, including computer program products, for performing multiple tests on a test web page. -measuring the impact of each intervention on the target metrics by creating a system where test units (example- web pages) are configured to perform multivariate experiments all at a time. Different subsets of units are subjected to multivariate intervention subsets. Impact of each intervention is calculated by measuring the lift of the target metric of the subsets of units which have the concerned intervention present as one of the treatments. However, the prior art relates to testing variations in web page contents. Thus the prior art unable to measure impact of multiple product interventions in E-commerce platform.
[005] US20110161825A1 discloses a related Multi-page Testing framework where every user is assigned to a test group with a version of the intervention (Eg- web page designs) having one or more unique design features. The user’s behavior on the delivered intervention and subsequent interactions with the platform are tracked and eventually aggregated with all the other users and test groups to analyze and determine test results. However, the prior art relates to testing of a group with a version of intervention having one or more unique design features. Thus the prior art unable to measure impact of multiple product interventions in E-commerce platform
[006] US20150287050A1 discloses a spilt testing framework. The framework where every incoming request can be evenly balanced to different application variants, initiating a plurality of application variants. The framework essentially describes how to measure the effect of each variation on the user behaviors included in the test scenarios based on logging data associated with the plurality of application variants and interactions with the users.
[007] In view of the above problems associated with the state of the art, there is a need a framework or method or technique that can accurately measure impact of multiple product interventions in E commerce platforms or websites.
OBJECTIVES OF THE INVENTION
[008] The primary objective of the present invention is to provide a method for measuring the impact of the multiple interventions in E-commerce platform.
[009] Another objective of the present invention is to measure the total revenue impact of multiple interventions.
[010] Another objective of the present invention is to develop a method which can provide product owners or users with accurate data about conversion details of customers in case of multiple interventions.
[011] Yet another objective of the present invention is to provide easy to use method for product owners to test and measure impact of multiple interventions without tactually performing multiple A/B tests.
[012] Other objects and advantages of the present invention will become apparent from the following description taken in connection with the accompanying drawings, wherein, by way of illustration and example, the aspects of the present invention are disclosed.
SUMMARY OF THE INVENTION
[013] The present invention relates to a method for measuring the impact of multiple interventions in E-commerce platform. In the present invention, the method proposes an additional capability on top of the A/B user platform which aims to learn the causal structure of an E-commerce platform. The method includes steps: verifies login credentials of one or more users over a user platform installed in a computing device; receives plurality of test scenarios from the one or more users over the user platform, each of the test scenario comprises plurality of configured A/B tests, domain of customers, specification of nodes, intervention nodes and time slots; storing parameters into one or more memory units by a parameter extraction module, the parameters corresponding to the plurality of A/B tests along with plurality of interventions applied to corresponding nodes or the intervention nodes; retrieves real world historical data from a database of an E-commerce platform by a data fetching module; creates a data frame on the customers and a list of intervention parameters or variables for each customer Identity (ID) from the stored parameters. Further a compute module is adapted to identify graphical structure of the non-intervened observational graph of the E-commerce platform based on real word historical data, the compute module is configured to: learn interventional and non-interventional data from the data frame and utilize the A/B tests to learn a graphical structure of the user platform in presence of the plurality of interventions, determine joint probability distribution based on estimated edge weights of the non-intervened observational graph and the graphical structure of the user platform in presence of the plurality of interventions by maximizing the two likelihood functions for the non-intervention data and interventional data, simulate outcome variables under an interventional distribution and no-intervention distribution from the joint probability distribution an importance sampling module, estimate final uplift data caused due to multiple interventions in a uplift data by calculating average value of the simulated outcome variables under the interventional and non-interventional data using, and generating a report by the user platform displaying the estimated final uplift data to accurately measure impact of multiple interventions in the E-commerce platform.
BRIEF DESCRIPTION OF DRAWINGS
[014] The present invention will be better understood after reading the following detailed description of the presently preferred aspects thereof with reference to the appended drawings, in which the features, other aspects and advantages of certain exemplary embodiments of the invention will be more apparent from the accompanying drawing in which:

[015] Figure 1 illustrates a flowchart representing a method for measuring the impact of multiple interventions in E-commerce platform.

[016] Figure 2 illustrates a flowchart representing an interface framework of E-commerce platform with different connected nodes and multiple interventions.

[017] Figure 3 illustrates a layered A/B test framework along with a compute module for measuring the impact of multiple test programs.

DETAILED DESCRIPTION OF THE INVENTION
[018] The following description describes various features and functions of the disclosed system with reference to the accompanying figures. In the figures, similar symbols identify similar components, unless context dictates otherwise. The illustrative aspects described herein are not meant to be limiting. It may be readily understood that certain aspects of the disclosed system can be arranged and combined in a wide variety of different configurations, all of which have not been contemplated herein.

[019] Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[020] Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

[021] The terms and words used in the following description are not limited to the bibliographical meanings, but, are merely used to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention are provided for illustrative purpose only and not for the purpose of limiting the invention.

[022] It is to be understood that the singular forms “a”, “an” and “the” include plural referents unless the context clearly dictates otherwise.

[023] It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, steps, components or groups thereof. The equations used in the specification are only for computation purpose.

[024] The present invention relates to a method for measuring the impact of multiple interventions in E-commerce platform. The method measures impact of multiple test programs or interventions of different platform drivers on one or multiple outcome metrics without the need to maintain different hold out populations and interaction guardrails by the product owners.

[025] Referring to figure 1, a method for measuring the impact of multiple interventions in e-commerce platform is illustrated. The method is performed via a computerized system. The computerized system includes but not limited to, one or more computers or computing devices, each computer including a non-transitory memory storing instructions for execution by one or more processor, one or more memory units. In some embodiments, multiple compute processors, the memory units or memory systems to perform the functions on input data to generated desired output data by executing the computer programs written in non-volatile memory devices- internal hard disk, magnetic disks or other form of semiconductor memory devices. The processors adapted to receive instructions encoded in the computer programs from a memory- read only or random access type, in order to fetch required data from one or more mass storage devices. The functional operations described in the specification can be implemented in a combination of computer software’s and hardware’s. The steps involved in the specification can be implemented as a collection of computer programs encoded in executable form in a suitably designed execution environment. In an exemplary embodiment, a display device, a keyboard is utilized by a user to give input on the different A/B tests and receive the output results after the computer programs complete execution and generate the results. The processor, memory, display, keyboard and mass storage devices are connected such that at any point of time the user can query on the store data, view the logs of the computer programs, read the intermediate results or give or receive other form of feedbacks to and fro to the system.

[026] Referring to figure 3, the processor runs a sampling algorithm to fetch data from the mass storage device or memory units where all the E-commerce platform data are stored. Two separate data frames are formed - one for the real world customer data under no-intervention and another data frame for the data on multiple live test programs. The statistical likelihood optimization written in equation (c) and (d) for the observational data and the interventional data are executed from the corresponding computer program with the help of the processor and memory units. The intermediate outputs of the optimisations are written in the form of run time logs displayed to the user along with the reported optimised value over different time stamps during the code execution. The user can terminate the process, across though the result any point of time by giving suitable input to the system. After the optimization is complete with convergence, the parameter estimates are recorded in the memory units for next stage of executions. The processor perform importance sampling of desired sample size from the two estimated probability distributions using the estimated models temporarily stored in the memory. The final estimated uplift is written along with other performance statistics and metrics in the memory and shown on the visual display device.

[027] The computing device includes but not limited to, one or more processors, one or more memory units, and one or more communication modules. The computing device is selected from but not limited to, a smartphone, tablet, handheld computing device, laptop, a computer or supercomputer, and a server. In an embodiment, the computing device is installed with a user platform or user interface. The user interface or user platform may be an application installed in the computing device. In some embodiments, the computing device has a touchpad or a touch-sensitive display (also known as a “touch screen” or “touch screen display”). The user interface or user platform allow users (hereinafter interchangeably refer as product owners) or product owners to interact with the interface through finger contacts and gestures on the touch-sensitive surface of the display and access the user interface to store or modify or delete or extract information and execute different functions on the interface. In an embodiment, the user platform/user interface is a Graphical User Interface (GUI).

[028] The user platform treats the E-commerce platform as a graphical network of interconnected nodes, whose structure can be learned from Data using the state of the art statistical methods. The nodes are selected from a group of price, speed, cart page, search algorithm, payment method, and cart add button. The only inputs required into the user platform is from the end users, supposedly the product owners of test programs. Each test program is essentially a collection of one or more interventions designed on different nodes of the E-commerce platform. Each intervention is designed in the form of an A/B test on the suitable node where a random sample of data is treated with the intervention and a random sample of data is kept as control.It is to be noted that, A/B testing is a controlled experiment to test user experience where two different product or website versions simultaneously run by the executor and see which one performs better. A collection of such interventions can be one test program and a platform may have multiple test programs running simultaneously by different product teams or business functions of the organisations. The inputs of the platform are the specifications of the nodes of the E-commerce network where the interventions are made and the outcome metric measured at a chosen outcome node. The outcome metric can be customer conversion rate, revenue per unit investment, return to order, delivery cost etc. The outcome metric is defined depending on the choice of the outcome node for which impact of the designed interventions need to be measured. For Example: Rate of conversion is defined at final conversion node, and return to order is defined at return node, and similarly delivery cost is defined at speed node. It is to be noted that the product owners of different test programs can specify the nodes for the interventions are made for respective test programs along with the chosen levels of the interventions. Similarly the outcome nodes are also needed to be specified for each test program along with the chosen outcome metrics.

[029] If the interest is to learn the impact of multiple levels of drivers designed through the interventions on the outcome node of different test programs, the levels are needed to be specified by the user(s) through the interface. In test programs with active interventions to collectively measure the total impact of multiple interventions on the outcome node correcting for the experimental overlaps. In absence of test programs, if the interest is to learn the impact of different levels of multiple drivers on the outcome node from real world observational data, the values of the drivers are also needed to be specified by the user(s) through the interface. In an embodiment, a historical platform dataset includes subsets of data with different levels of the drivers. In absence of test programs (i.e. user or product owner’s input) or active interventions, the method use the subsets of data and compare the outcome metrics aggregated within the subsets after correcting for confounding biases following the same statistical modeling strategy. The impact of values of the different drivers on the outcome nodes are essentially the causal uplifts estimated from historical observational data by correcting for confounding bias caused by different confounder variables.

[030] A method for measuring impact of multiple interventions configured as different A/B tests in E-commerce platform is explained as follows:

a) At first, the product owners have to login in the user platform with login credentials. The login credentials includes but not limited to, login id, password, email id or test program id. Further, the user platform verifies login credentials of one or more users. Upon successful verification of the product owner, the product owners need to provide inputs and configured experiments into the user platform as per their requirements. The inputs may include the specifications of the nodes applied with interventions, values of intervention variables, time period for the interventions continue to be applied over the nodes and customer Identity (IDs) assigned to test(s) and control groups of each of the A/B test corresponding to different interventions. Each of said test scenario essentially comprising plurality of A/B tests, domain of customers, intervention nodes, and time slots. The domain of customers include a subset of the platform traffic of customer relevant to a given A/B test. For Example- A pricing intervention is designed for customers from metro cities only for grocery products. The pricing intervention identifies a subset of all customer traffic and call it a domain of customers. It is to be noted that, different domains may specified for different interventions partially or completely overlaps different interventions. The intervention variables, values of intervention variables, the domain of customers participating in different tests etc. together constitute of different parameters of the A/B tests.
b) A parameter extraction module records all the parameters corresponding to different configured A/B tests along with the corresponding nodes applied with the interventions. A data frame is created to record all the intervention variables for each of the customer IDs participating in different A/B test programs.
c) Further, the parameters corresponding to the plurality of A/B tests along with plurality of interventions applied to corresponding nodes of the intervention nodes are stored into the one or more memory units by the user platform. The parameters corresponding to the A/B tests are configured by the product owners for different test programs and are taken as input by the method. Each test program may also consist of multiple A/B tests on related interventions on similar nodes.
d) A data frame on the customers IDs along with the test program IDs, wherein each of the customers are part of is created along with time stamps and test program durations. Another data frame is created on the test programs with test program IDs and parameters corresponding to the plurality of A/B tests on interventions applied to corresponding intervention nodes. The customer data frame is joined with the test program data frame, left joined on test program IDs giving out customers IDs participating in different test programs along with parameters corresponding to the plurality of A/B tests.
e) Further, a data fetching module retrieves or fetches real world historical data from a database of an E-commerce platform by the user platform. The real word historical data include details of all the historical browsing of the customers, clicks, page views and conversion events before the interventions for the different A/B Test programs start happening (or conversion history). The real world historical data is essentially historical non-interventional data on multiple variables corresponding to different nodes of the E-commerce platform along with the events created at different time stamps due to customer activities spread over the chosen time frame before the interventions start. The data fetched on real world non-interventional data on different variables are stored in a data frame.
f) Along with real world non-interventional historical data, the data fetching module also fetches post intervention data corresponding to different A/B test programs by joining the platform data with the data frame created on the customer IDs and intervention variables created in the previous step. The created data frame is an additional data frame on interventional data on the same set of platform variables and customer events but observed only during the intervention time window.
g) A compute engine or compute module is set up for all the data operations, statistical modeling, optimization and simulation. In some embodiments, multiple compute processors and memory systems are utilized to perform the functions on input data to generated desired output data by executing the computer programs written in non-volatile memory devices- internal hard disk, magnetic disks or other form of semiconductor memory devices. The processors adapted to receive instructions encoded in the computer programs from the memory units i.e. read only or random access type and fetches required data from one or more mass storage devices or memory units. The compute module is used to perform all the joining operations involved to create the data frame as described in the previous steps (e and f).
h) The compute module installed in the user platform runs a set of computer programs encoded in executable form in a suitably designed execution environment. The computer program involves all the data joining and filtering operations. The program also includes the statistical modeling steps to learn the non-interventional and interventional data distribution from the data frame created in the previous step by maximising the statistical data likelihood function.
i) The statistical data likelihood function is the mathematical representation of the platform data with the underlying structure of the E-commerce platform incorporating the different variables. The unknown quantities in the likelihood function are estimated maximising the likelihood function, the likelihood function signifies what values of the unknown parameters are most likely to generate the data that have been observed while fetching the platform data. The estimated values of the unknown quantities characterise the underlying data distribution. The net uplift observed due to multiple interventions under various test programs as compared to the non-interventional distribution is essentially a function of the estimated unknown parameters maximising the two likelihood functions- the non-interventional one and the interventional one.
j) The different variables in the E-commerce network are connected by directional edges, indicating how the different pairs of variables are causally associated. The edges have weights indicating the strength of the causal connection. The graphical form and edge weights may change depending on the intervention applied on a node of the graph. Therefore two sets of directed graphs are generated, one under the intervention and one under no intervention. Consequently two different likelihood functions are configured i.e. one for the non-interventional data and one in presence of multiple interventions on the platform. The two likelihood functions are written in equation (a) and (b) for the interventional data and the non-interventional data are optimised by executing the corresponding computer program with the help of the processors and the random access memory.
k) The compute module is configured to:
i. Utilize the data frame and A/B tests under the plurality of interventions to learn a graphical structure of the user platform in presence of the plurality of interventions.
ii. Determine joint probability distribution based on estimated edge weights of the non-intervened observational graph and the graphical structure of the user platform in presence of the plurality of interventions by maximising the two likelihood equations given as equation (a) and (b). The non-intervened observational graph is a graph of nodes that are not applied with interventions.
iii. The maximum likelihood estimated parameters used to recover the probability distributions associated to the non-interventional data and interventional data. In both the situations, the outcome variables are connected to the other platform variables based on the directed graph with the estimated parameters being the directional edge weights. The joint distributions of the outcome variables and the platform variables under non-interventional data and interventional data used to simulate the outcome variables under the two different scenarios. The influence of the platform variables including the intervened variables on the outcome variable is taken care of by the maximum likelihood optimised data distributions through the directional edges and edge weights.
iv. Simulate outcome variables under an interventional data and non-intervention data from the joint probability distribution mentioned in the previous step using an importance sampling algorithm/module. The method of importance sampling and the mathematical details as given in equation (d);
v. To estimate the final uplift caused due to multiple interventions by calculating the average value of the simulate outcome variable under the interventional data and non-intervention data and take the differences.
vi. The final uplift generated due to multiple interventions is estimated as the average of the simulated outcome variables under multiple interventions and no intervention respectively. The user platform generates a report, the report displays the estimated uplift data to accurately measure impact of multiple interventions in the E-commerce platform. The uplift data include the difference of the estimated outcome metrics under the non-interventional and interventional distribution as given in equation (d). The uplift data may include the error bound for the estimated uplift, the confidence interval and statistical p-value. The outcome metric uplift can be estimated over a fixed time window repeatedly to create a profile for the estimated uplift with time due to the interventions from multiple test programs and shown as a graphical plot in the report. It is possible to compute the change in the other secondary variables, not necessarily the intervened ones, due to multiple interventions on a number of primary variables and report the secondary variables as a part of a reporting dash board. The change in the outcome variables in the form of estimated uplifts, the change in the secondary variables can be also shown for various segments of data for example, specific customer segments, specific product segments or time window. The change in the outcome variable reports the overall goodness due to the multiple interventions, whereas the change in the secondary variables often provide directional insights on impact of the interventions on other business, tech or experience metrics of the platform. The metrics often need to be observed and guard-railed to make sure the performance of the E-commerce platform remains consistent with respect to various factors over time. Reporting the results, within various segments of data give insights on relative goodness of the interventions within various customer cohorts, product groups or other important data segments- such as geography, market and other demographics. Therefore, helps to take prioritisation calls on where it makes more sense to roll out different product changes as a result of the A/B test programmes.

[031] E-commerce platform with key modalities as described in the Figure 2. The various metrics tracked for different nodes of interest can be as follows: for example, suppose a product organisation is running multiple interventions on the nodes as described below and multiple A/B tests running for each of these interventions separately with a respective randomised control group for each test.

[032] Explanation of the components disclosed in the fig 2 is as follows: Selection Node: Intervention A (Example- boosting newer lifestyle brands at a lower price band) and Intervention B (Example- Introducing New T-shirt selection with certain trendy design pattern. Speed Node: Intervention C (Example- Improving Lifestyle product Speed by one Day). Cart Page Node: Intervention D (Example- A “Buy now pay later” call out in the Cart page). Payment Node: Intervention E (Example- Option of a new Pay later using a Digital Wallet).

[033] Each of these interventions may have a number of direct or indirect impacts on different metrics as denoted by the figure 2 based on the directed paths from the intervention nodes to the outcome nodes. The Directional arrows from one node to the other in the Graph (as shown in fig 2) are causal connection routes describing effect of changing the source node causes the destination node to change. Example- Changing the speed node causally impact the Product Rating and supply chain cost. A directional causal path between the Speed node and conversion node in the sense that, changing speed cause rating to change, which in turn cause more products being added to the cart. Thus lead to conversion through payment methods (Prepaid/Cash on delivery) on the card check out page. The speed offered on a product can causally influence the Return through the same causal path. Example- a slower speed possibly increase the chance of order cancellation and return.

[034] Fig 3 illustrates a layered A/B test framework along with a compute engine for measuring the impact of multiple test programs. In the fig 3, left portion (i.e. A/B experiments configuration layer) of the fig 3 represents running of multiple A/B tests simultaneously and right portion of the fig 3 represents a compute engine for measuring the impact of multiple test programs.

[035] A layered test framework (Fig 2-Left) is often the choice of large internet organisations for running such A/B test programs. The platform (as described in the fig 2). The platform include multiple parameters corresponding to different nodes of the E-commerce network, eg- price, speed, search algorithm, payment method, cart add button, etc. All the parameters are divided into a number of subsets of parameters, so that each subset of parameters together consist of one layer. The layered A/B framework consists of multiple such layers. In the layered A/B test, the parameter subsetting is done in such a way, that parameters from different layers do not interact with each other, but parameters from same layers generally interact with each other. In case of multiple buckets of customers in each layer, where customers are randomly assigned to each bucket. Hence for the A/B test program with multiple experiments, with at most one experiment per layer, the experiments remain none overlapping. Example: An Ad Ranking algorithm and an ad banner positioning are parameters from two different layers which not interact with each other, however, the ad banner positioning and the banner size may have interactions with each other as both the ad banner positioning and banner size belong to same layer. Standard statistical tests like 2-sample t-test, Pearson's Chi-squared tests etc. are performed on the test results to measure the uplift and association between the intervention and outcome along with the reported p-value, secondary metrics change, and group summary statistics etc.

[036] Steps to Measure Impact of Multiple A/B tests: In order to learn joint impact of the multiple interventions, a causal network associated with the E-commerce platform need to be learned in two different circumstances i.e. under no intervention and in presence of multiple such interventions. In the method, the compute module is used learn the joint impact of multiple test programs on the outcome metrics correcting for the mutual interactions among overlapping experiments. The steps involve learning the causal network of the different nodes of the platform along with the joint probability distribution associated with the nodes. The joint probability distribution efficiently estimated using Markov factorization associated with the causal structure of the network and optimization of the data likelihood. The final step involves estimating the uplift in the outcome variable under two different distributions i.e. the interventional one and the one in absence of all interventions.

[037] The framework of the user platform and experimental data is mentioned as follows: In an embodiment, the user platform act as a directed acyclic graph of a number of nodes (V) and directed edges (E) between the nodes, structure of the graph is required to be learned from the data. The data include real world historical data under no-intervention and under multiple interventions according to the designed A/B test programmes fetched from the E-commerce platform database or data storage. The data include details of all the historical browsing of the customers, clicks, page views and conversion events before the interventions for the different A/B Test programs start happening. The data is essentially historical non-interventional data on multiple variables corresponding to different nodes of the E-commerce platform along with the events created at different time stamps due to customer activities spread over the chosen time frame before the interventions start. Along with real world non-interventional historical data, the data also include post intervention data corresponding to different A/B test programs for different customer IDs and intervention variables. A number of variables observed and recorded for both the datasets including the variables which are intervened and the variables which are not intervened. Each variable corresponds to some node of the platform linked to the final outcome such as conversion, revenue etc. At the simplest form, speed, selection, cart page etc. are different nodes of the platform network as shown in fig 3. The nodes may be defined at a more granular level of features such as specific aspects of selection or specific widgets of cart page. The observations corresponding to different nodes of the platform network are recorded as observed values corresponding to a |V| dimensional random vector denoted by

[038] One of these variables correspond to an outcome node, the outcome node can be cart conversion, order cancellation etc. An outcome variable Y is measured corresponding to the outcome node, for example: Conversion rate, return rate, Cart value per transaction, Cost per order cancellation etc. In order to measure the impact of the interventions on the outcome variable Y where the interventions are applied on a subset of the platform nodes. Several variables are observed on the platform nodes are given in X. Some of these variables are impacted due to multiple interventions at several platform nodes. The platform variables have influence on the outcome variable via several directional causal paths described by the platform network based on the directional weighted edges between platform variables and outcome variables.

[039] Some of the nodes corresponding to the outcome variables are intervened by a set of binary randomised A/B tests. The goal is to learn the joint effect of these A/B tests on the outcome variable Y. The underlying joint probability distribution corresponding to the random vector X is given by PX i.e. is the distribution corresponding to the observational data. Let us assume K Binary Interventions are applied on K Nodes of the platform network and the node intervened in the lth intervention is given by Rl(r), where the value of intervention is r.
The joint interventional distribution of the platform network for the intervention is denoted by

Where Pa(Xj) denotes the parents of the node corresponding to the variable Xj in the Intervened Causal Graph. Similarly the joint observational data distribution of the platform is denoted by

[040] The interventions described here are assumed to be perfect interventions in the sense that the intervention on a specific node j does not depend on its parents. In some cases the intervention can be a function of the parents, which is the case of a non-randomised A/B and is called an imperfect intervention. Example- one day faster speed is applied for products with higher price. Note that price is a parent node of the speed node and hence such intervention is an imperfect non-randomised intervention. The key difference between the interventional distribution in equation (a) and that of the observational data distribution in equation (b) is simply the fact that, when a given node is intervened by a suitably designed treatment value r, the parents corresponding to the intervened node unable to influence the intervened node anymore and hence the edges flowing from the parents to the intervened node are removed and the likelihood function is recomputed incorporating the removed inward edges of the intervened node.
[041] Denoted the No-interventional distribution and the Interventional distribution by P(Y,X) and PInt(Y,X) respectively, where Y is the outcome variable. The final step involves estimating the expected value of the outcome variable under the Interventional distribution and comparing it with that of the no-interventional distribution for estimated uplift due to the joint effect of all the K interventions. The estimated uplift is given by the following integral operation given in (c). The first integral represents the average value of the outcome variable Y under the intervention distribution and the second integral represents the average value of the outcome variable Y under observational data in absence of any interventions.

[042] It is to be noted that the quantity can be estimated with an importance sampling approach. The first integral in particular, needs to be computed using the interventional data distribution. However, observations drawn from real world observational data under no-intervention can be used to approximate its average in interventional data distribution using an importance sampling module. The importance sampling approach involves drawing observations on the outcome variable and the platform variables (Y, X) from the observational data distribution and reweighing the observations depending on generation of observations from the interventional distribution in comparison to the non-interventional data distribution. The reweighing gives the following value of the uplift due to multiple interventions which is an approximation of the integral in (c) given by

[043] While this invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.
, Claims:WE CLAIM:

1. A method for measuring the impact of multiple interventions in E-commerce platform, comprising of:
a) verifying login credentials of one or more users over a user platform installed into a computing device, the login credentials comprising of user ID, login ID, password, and program ID;
b) receiving plurality of test scenarios from the one or more users over the user platform, each of said test scenario comprising plurality of configured A/B tests, domain of customers, specification of nodes, intervention nodes, and time slots;
c) storing, into one or more memory units by a parameter extraction module, parameters corresponding to the plurality of A/B tests along with plurality of interventions applied to corresponding nodes or the intervention nodes;
d) retrieving real world historical data from a database of an E-commerce platform by a data fetching module; and
e) creating, by the user platform, a data frame on the customers and a list of intervention parameters or variables for each customer Identity (ID) from the stored parameters;
wherein,
I. a compute module installed in the user platform adapted to identify graphical structure of the non-intervened observational graph of the E-commerce platform based on real word historical data, the compute module is configured to:
i. learn interventional and non-interventional data from the data frame and utilize the A/B tests to learn a graphical structure of the user platform in presence of the plurality of interventions;
ii. determine joint probability distribution based on estimated edge weights of the non-intervened observational graph and the graphical structure of the user platform in presence of the plurality of interventions by maximizing the two likelihood functions for the non-intervention data and interventional data;
iii. simulate outcome variables under an interventional distribution and no-intervention distribution from the joint probability distribution an importance sampling module;
iv. estimate final uplift data caused due to multiple interventions in a uplift data by calculating average value of the simulated outcome variables under the interventional and non-interventional data using; and
II. generating a report by the user platform displaying the estimated final uplift data to accurately measure impact of multiple interventions in the E-commerce platform.

2. The method as claimed in claim 1, wherein the real word historical data include details of all the historical browsing of the customers, clicks, page views and conversion history.

3. The method as claimed in claim 1, wherein the domain of users includes a subset of the user platform traffic of the customers relevant to an A/B test assigned to the customer.

4. The method as claimed in claim 1, wherein the parameters includes intervention variables for each of the customer ID, values of intervention variables, and the domain of customers participating in different A/B tests.

5. The method as claimed in claim 1, wherein the compute module is configured via a Bayesian network using Markov factorisation.

6. The method as claimed in claim 1, wherein the E-commerce platform is a graphical network of interconnected nodes.

7. The method as claimed in claim 6, wherein the nodes are selected from a group of price, speed, cart page, search algorithm, payment method, and cart add button.

8. The method as claimed in claim 1, wherein the non-intervened observational graph is a graph of nodes that are not applied with interventions.

9. The method as claimed in claim 1, wherein the estimated final uplift data comprises difference of the average outcome variable under multiple interventions and no-intervention, simulating the outcomes from respective probability distributions.

10. The method as claimed in claim 1, wherein the report comprises the uplift data, that is the difference of the total estimated outcome metrics due to the multiple test programmes as estimated under the non-interventional and interventional distribution, the error bound for the estimated uplift, the confidence interval and statistical p-value.