Custom Charts in Facilita Forecast Web

This article will describe how to create your own custom realtime graphs in the Facilita Forecast Web Controller.   When running load tests, one of best places to get a quick overall idea of how a current test is executing is the charts window in the Facilita Forecast Test Controller. This is place where you can see various graphs for all the metrics you are recording.

There are various metrics already collected about all your tests – VU Concurrency; Transaction and Timing response times and rates; HTTP request statistics; and few more. But what do you do when you need to monitor some custom metrics?

I had such a problem. In the system I was testing there were some transactions where the responses contained error messages. For example – The application under test had various error messages with different codes, e.g. error-1004error-1007error-1009  etc…

There is a in-built way to record new metrics, you simply need to think of a name for what you are recording and add a line to your code.

The function that records this is this:

void  recordMetric(const string& name,long value);

So to use my example to record error I would use code like this:

recordMetric(errorCode, 1);

Where errorCode is extracted from response,  this included the error name (e.g. error-1004).

This worked fine,  but there were multiple errors and each error would create a new metric chart. In fact 3 new metrics were created because every individual metric now had 3 charts – average, rate & sum.

This isn’t exactly what I wanted – what I really needed was to have one chart for all the errors.  Which technically would have resulted in 3 charts (avarage, rate, sum).

This is an undocumented feature of the Forecast software.  Fortunately Facilita provided some help (thanks Chris!). This is how you consolidate all of the Generic Viagra errors into a single chart:

After you have run a test (you have to make sure the target metric(s) are recorded at least once), a record is created for this metric in the $WORKSPACE_DIR$/counters.json. If you had 3 type of errors (error-1004, error-1007, error-1009) and they have been successfully actioned and recorded while running tests, your counters.json file will contain these lines:

{
      "Title":"error-1004 (Average)",
      "Label":"",
      "CounterDefinitions":[
         {
            "Title":"error-1004 (Average)",
            "Description":"",
            "Label":"error-1004",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1004"
            ]
         }
      ]
   },
{
      "Title":"error-1007 (Average)",
      "Label":"",
      "CounterDefinitions":[
         {
            "Title":"error-1007 (Average)",
            "Description":"",
            "Label":"error-1007",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1007"
            ]
         }
      ]
   },
{
      "Title":"error-1009 (Average)",
      "Label":"",
      "CounterDefinitions":[
         {
            "Title":"error-1009 (Average)",
            "Description":"",
            "Label":"error-1009",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1009"
            ]
         }
      ]
   }

And another similar 3 records for Sum and Rate. What this does is create 3 different type of charts (error-1004 (Average), error-1007 (Average) and error-1009 (Average)), it is unlikely you want this format.

So how can you join them together in one chart? To unite them you need to do some editing (copy & paste) by hand. You need to join the various errors into one chart. For my example you would need to change all that text (json code) into this:

{
      "Title":"Error count (Average)",
      "Label":"",
      "CounterDefinitions":[
         {
            "Title":"error-1004 (Average)",
            "Description":"",
            "Label":"error-1004",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1004"
            ]
         },
         {
            "Title":"error-1007 (Average)",
            "Description":"",
            "Label":"error-1007",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1007"
            ]
         },
         {
            "Title":"error-1009 (Average)",
            "Description":"",
            "Label":"error-1009",
            "CounterType":0,
            "CalculatorType":0,
            "IsRequestCounter":false,
            "Factor":1,
            "RawMetricNames":[
               "error-1009"
            ]
         }
      ]
}

This will then create a single  chart with three different series for each of the errors.

You can follow this principle with the Sum and Rate charts too (there is slight difference in values at CounterType and CalculatorType).

Note that counters.json file removes all white-spaces. This means you will have to use a JSON Formatter to make editing easier.

I successfully used this technique for displaying error counts, however the techniques can be applied to solve a host of different problems.

Leave a Reply

Your email address will not be published. Required fields are marked *