Manipulating chart data

Each chart type in amCharts family has a specific requirement for structure its data should come in. Mostly it comes in a form of an array of objects. However sometimes you don't have control over format of source data. This tutorial will explore how we can pre-process data to reshape it into an amCharts-suitable form.

Let's look at various possible scenarios and how to tackle them.

Serial data

Let's say we need to plot a raw log data that contains measurements for multiple devices:

[
  { device: "dev1", date: new Date(2018, 0, 1), value: 450 },
  { device: "dev2", date: new Date(2018, 0, 1), value: 362 },
  { device: "dev3", date: new Date(2018, 0, 1), value: 699 },

  { device: "dev1", date: new Date(2018, 0, 2), value: 269 },
  { device: "dev2", date: new Date(2018, 0, 2), value: 450 },
  { device: "dev3", date: new Date(2018, 0, 2), value: 841 },

  { device: "dev1", date: new Date(2018, 0, 3), value: 700 },
  { device: "dev2", date: new Date(2018, 0, 3), value: 358 },
  { device: "dev3", date: new Date(2018, 0, 3), value: 698 }
]

We want to plot separate Series for each individual device, but obviously we can't use a data like that, because:

a) Dates repeat.
b) There's no way to bind Series to a specific value field because all entries use the same name of the value field: "value".

If we can't change our incoming source data, we need to reshape it before it is used by chart.

There are a couple of approaches to do that.

Consolidating data items

In this chapter we'll see how we can turn the raw data above into something that can be used by chart and targeted by specific Series:

[
  { date: new Date(2018, 0, 1), dev1: 450, dev2: 362, dev3: 699 },
  { date: new Date(2018, 0, 2), dev1: 269, dev2: 450, dev3: 841 },
  { date: new Date(2018, 0, 3), dev1: 700, dev2: 358, dev3: 698 }
]

With data like this we can easily create three separate Series, binding one for Device #1 to "dev1", etc.

Before chart parses data, it fires off a "beforedatavalidated" event. This gives us a prefect chance to modify data in any way, before chart sees it.

chart.events.on("beforedatavalidated", function(ev) {
  let source = ev.target.data;
  let dates = {};
  let data = [];
  for(let i = 0; i < source.length; i++) {
    let row = source[i];
    if (dates[row.date] == undefined) {
      dates[row.date] = {
        date: row.date
      };
      data.push(dates[row.date]);
    }
    dates[row.date][source[i].device] = row.value;
  }
  ev.target.data = data;
});
chart.events.on("beforedatavalidated", function(ev) {
  var source = ev.target.data;
  var dates = {};
  var data = [];
  for(var i = 0; i < source.length; i++) {
    var row = source[i];
    if (dates[row.date] == undefined) {
      dates[row.date] = {
        date: row.date
      };
      data.push(dates[row.date]);
    }
    dates[row.date][source[i].device] = row.value;
  }
  ev.target.data = data;
});
{
  // ...
  "events": {
    "beforedatavalidated": function(ev) {
      var source = ev.target.data;
      var dates = {};
      var data = [];
      for(var i = 0; i < source.length; i++) {
        var row = source[i];
        if (dates[row.date] == undefined) {
          dates[row.date] = {
            date: row.date
          };
          data.push(dates[row.date]);
        }
        dates[row.date][source[i].device] = row.value;
      }
      ev.target.data = data;
    }
  }
}

That's it. Now our custom function will kick in and mutate the source data in a way that can be used by chart.

See the Pen amCharts 4: Pre-processing data (consolidating) by amCharts team (@amcharts) on CodePen.24419

Filtering series-specific data items

Now let's look at another option: filtering data individual items for each series.

As you may or may not know, you can set data not just directly on a chart object, but also for each series individually.

This way we can use the same source data for all series, and just filter out irrelevant data items using the "beforedatavalidated" event.

Except this time we'll be setting the event on series, because the data is set directly on series, too.

series.data = data;
series.events.on("beforedatavalidated", function(ev) {
  let source = ev.target.data;
  let data = [];
  for(let i = 0; i < source.length; i++) {
    let row = source[i];
    if (row.device == id) {
      data.push(row);
    }
  }
  ev.target.data = data;
});
series.data = data;
series.events.on("beforedatavalidated", function(ev) {
  var source = ev.target.data;
  var data = [];
  for(var i = 0; i < source.length; i++) {
    var row = source[i];
    if (row.device == id) {
      data.push(row);
    }
  }
  ev.target.data = data;
});
{
  // ...
  "series": [{
    // ...
    "events": {
      "beforedatavalidated", function(ev) {
        var source = ev.target.data;
        var data = [];
        for(var i = 0; i < source.length; i++) {
          var row = source[i];
          if (row.device == "dev1") {
            data.push(row);
          }
        }
        ev.target.data = data;
      }
    }
  }, {
    // ...
    // Reapeat same for other series
  }, {
   // ...
    // Reapeat same for other series 
  }]
}

Now each series will start off with identical data. "beforedatavalidated" will kick in for each individual series, and will filter items relevant for that series only.

Here's how it looks like on a live chart:

See the Pen amCharts 4: Pre-processing data (filtering by series) by amCharts team (@amcharts) on CodePen.24419

Custom-structured data

Now let's see at a more exotic scenario. Say our source is some complex structure, containing actual data hidden a few levels deep.

Say we are trying to build a heat map on a World map. our source data looks like this:

{
  something: "Useless",
  maybe: {
    here: {
      values: [
        { id: "US", value: 60 },
        { id: "MX", value: 50 },
        { id: "CA", value: 70 }
      ]
    }
  }
}

Obviously, we can't use this directly on a MapPolygonSeries' data. We just need the array contained deep inside the object, under "values".

Our good old friend "beforedatavalidated" comes to our rescue again:

polygonSeries.events.on("beforedatavalidated", function(ev) {
  var source = ev.target.data;
  if (source.maybe) {
    ev.target.data = source.maybe.here.values;
  }
});

polygonSeries.data = {
  something: "Useless",
  maybe: {
    here: {
      values: [
        { id: "US", value: 60 },
        { id: "MX", value: 50 },
        { id: "CA", value: 70 }
      ]
    }
  }
};
polygonSeries.events.on("beforedatavalidated", function(ev) {
  var source = ev.target.data;
  if (source.maybe) {
    ev.target.data = source.maybe.here.values;
  }
});

polygonSeries.data = {
  something: "Useless",
  maybe: {
    here: {
      values: [
        { id: "US", value: 60 },
        { id: "MX", value: 50 },
        { id: "CA", value: 70 }
      ]
    }
  }
};
{
  // ...
  "series": [{
    // ...
    "events": {
      "beforedatavalidated", function(ev) {
        var source = ev.target.data;
        if (source.maybe) {
          ev.target.data = source.maybe.here.values;
        }
      }
    },
    data: {
     something: "Useless",
      maybe: {
        here: {
          values: [
            { id: "US", value: 60 },
            { id: "MX", value: 50 },
            { id: "CA", value: 70 }
          ]
        }
      }
    }
  }]
}

Here's a live map:

See the Pen amCharts 4: Pre-processing map data by amCharts team (@amcharts) on CodePen.24419

External and dynamically updated data

The beauty of using "beforedatavalidated" approach is that we absolutely don't need to care where our data came from: was it set directly in-line, loaded via dataSource, or updated live on a live chart via API: the event will kick in regardless.