{"id":879,"date":"2019-09-26T10:44:04","date_gmt":"2019-09-26T08:44:04","guid":{"rendered":"http:\/\/blog.zhaw.ch\/splab\/?p=879"},"modified":"2019-09-26T10:44:05","modified_gmt":"2019-09-26T08:44:05","slug":"cloud-cost-estimation-forecasting-engine","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/splab\/2019\/09\/26\/cloud-cost-estimation-forecasting-engine\/","title":{"rendered":"Cloud Cost Estimation & Forecasting Engine"},"content":{"rendered":"\n

The occasion of the 3.3.2 release of the rating-charging-billing solution for cloud software and platform providers, Cyclops<\/a>, is a good opportunity for a deep dive into the new forecasting engine, the how and the why of its functionality and how to use it.
<\/p>\n\n\n\n

First, a bit of news. Active maintenance and further updates to Cyclops will now be found under the repository https:\/\/github.com\/serviceprototypinglab\/cyclops<\/a>. The primary new addition is the forecasting engine. It helps SaaS\/PaaS\/CaaS\/…XaaS providers to not only charge customers for their services, but also predict a revenue flow for deciding about future investments.
<\/p>\n\n\n\n\n\n\n\n

Core Estimation & Forecasting Engine
<\/h3>\n\n\n\n

The purpose of the Estimation Engine is to provide revenue prediction functionality and to allow the user, in this case a manager or administrator, to test potential changes to the pricing strategies they employ, and see how they will affect future revenue generation.
<\/p>\n\n\n\n

Thus, the engine works in two stages. The first stage of the engine is the forecast generator, which is based on the principles of timeseries forecasting and integrated into Cyclops\u2019 UDR microservice. Early versions of the engine relied on the ARIMA<\/a> model for forecast generation, though some problems in its Java library implementation lead to a simpler forecasting method being used. At this stage, the forecast is generated in the following way:<\/p>\n\n\n\n

  1. Existing usage data is collected from the usage table of the Cyclops UDR database<\/li>
  2. Data is organized by user and usage type (eg CPU allocated or external IPs used)<\/li>
  3. For each user and for each type of usage, the average hourly usage is calculated, then extrapolated over the desired forecast duration. This may seem like a downgrade from the ARIMA model used in previous iterations, though test-runs have shown the results are within \u22135% of what the model predicted and as stated before the Java library implementing that model was problematic<\/li><\/ol>\n\n\n\n

    The second element of the estimation engine is the ability to test pricing rules different to the ones currently used in production to generate the revenue forecast. Special \u2018testing\u2019 rules can be added to the Coin engine, which rely on a property of the forecast generator to \u2018tag\u2019 the generated records in order for them to trigger specific rules. Put simply, rules can test records for a specific tag, so that when a forecast request is made with that tag, the records will have that tag and trigger these specific targeted rules.
    <\/p>\n\n\n\n

    Using the Engine
    <\/h3>\n\n\n\n

    To demonstrate the features of the Estimation Engine, we\u2019ll walk through some simple examples. This will be made even easier with the use of the graphical dashboard.
    <\/p>\n\n\n\n

    We will actually be following the stages of the engine backwards, since we will start by deploying our test rules. First, we navigate to the Rule Management page in the dashboard. Here, we can see all rules deployed to both instances of the Coin engine.
    <\/p>\n\n\n\n

    By going to the New Rule tab, we can add new rules to the engine. Let\u2019s look at an example of a testing rule:
    <\/p>\n\n\n\n

    import ch.icclab.cyclops.facts.Usage;\nimport ch.icclab.cyclops.facts.Charge;\n\nrule \"Test 1 rule for storage_allocated (12:36:0.4 20\/Aug\/2019)\"\nsalience 60\nwhen\n  $usage: Usage(metric == \"storage_allocated\" && account.contains(\"test1\"))\nthen\n  Charge charge = new Charge($usage);\n  charge.setCharge(0.00000000004 * $usage.getUsage());\n  charge.setCurrency(\"CHF\");\n\n  retract($usage);\n  insert(charge);\nend\n<\/code><\/pre>\n\n\n\n
    The important part to note here is account.contains(“test1”) , and then the fact that the rule has a salience of 60 , where production rules have 50. The first item means that this rule will only be fired if the account field of the usage record contains the \u2018test1\u2019 substring. When a forecast is requested with the \u2018test1\u2019 tag, all records generated will contain this substring in the account field, so they will trigger the rule if the other conditions are met. Thus, these tags can be used to group rules together into entire pricing models. The importance of salience being higher that the production rule is simply to ensure the rule is checked first, as a rule firing means the record is deleted from the queue.
    <\/p>\n\n\n\n
    After all the rules are deployed we can move to the forecasting page to generate some forecasts and compare our models. Let\u2019s review the options given.
    <\/p>\n\n\n\n
    For the type of forecast, we can select:<\/p>\n\n\n\n