JM Spate Web Analytics

The difference between data and information

May 10

Posted by Jess in Technical notes | No Comments

Data is valuable. Nobody holds data in higher esteem than data miners and web analysts, but there is a world of difference between data and information.

Take the famous (to devotees of data mining) and much admired (by devotees of data mining) Wallmart customer basket database. The implementation of barcode scanners meant that each collection of products bought by a Wallmart customers could be recorded, and they did just that. Back in the early years of the new century this database was probably the biggest in the world. It ran into TeraBytes back when that was an impressive achievement (even to devotees of data mining). Depending on how you reckon 1 TB, it’s either 1000000000000 or 1099511627776 bytes. The good folks at the University of California, Berkeley, reckon that amount of data at 50000 trees or a good sized forest worth of paper.   To put it another way, Dr Jess’ PhD thesis weighed in at 2464KB, or one 405844th of a TeraByte.

Cute arithmetic aside, a TB is a lot of data, and there are plenty of databases of that size around today. Many of the server logs we deal with are well into GB or larger. Wading through all those numbers is the job of data mining algorithms and not people, and that’s as it should be.

It’s the job of web analysts to sort through the data spat out by programs like Google Analytics and Webalizer, sometimes using algorithms and software tools and sometimes their own judgment.

The task isn’t just to build a giant database, it’s to convert that data into usable information. Wallmart doesn’t find out whether coffee and sugar are bought together by reading through their data, and nobody trying to make money from a website needs to wade through masses of facts and figures and sieve out what’s important. That’s our job.

We know what to look for and how to extract information from data, and we also know how to present that information in an easily accessible form, which is half the battle when dealing with large databases.

Tags: , ,

Musings on metadata

Apr 15

Posted by Jess in Technical notes | No Comments

Years ago when I was working with earth scientists on a regular basis, a hydrologist asked me to take a look at a couple of his time series. A usually reliable rainfall-runoff model was refusing to calibrate properly, and when forced, producing results that were obviously nonsensical.

The problem was easy enough to spot. Of the two series of daily values, peaks were appearing in the runoff (river level) before the rainfall event was recorded. Conceptual models get upset when things like that happen.

A quick look at the meta data revealed the cause. The data wasn’t necessarily poor or even unrepresentative of real world behaviour, but the measurements were not taken in the same place or by the same person. Rainfall was measured every day at 9am, using a day defined from 0900 to 0859. Runoff was automatically logged at midnight every night, 0000 to 2359. You can see what happened from there.

Could be go back and re-collect a decade of data? No. Could we interpolate one series to fudge the definition of a day? Absolutely. Problem solved and we had a happy and useable conceptual model.

However, it’s worth noting that almost all small rural rainfall measurements are taken 9am to 9am, and almost all runoff values assume the day begins at midnight. The problem exists in many, many more datasets than we applied the correction algorithm to. All the subsequent modelling done with uncorrected rainfall-runoff data could have been improved in accuracy and utility had more people taken the time to interrogate the metadata and see that all was not well.

More recently I’ve noticed a similar issue crop up in web analytics. Instead of differing definitions of what constitutes a daily period, the trick is in the time zones used by various data collection devices and logs. If you’re seeing an inconsistency between two or more datasets or series, it could be something as simple as that.

Particularly in analytics, it might not be that simple (and often won’t be), but the first check you make should always be in the metadata. If you’ve got it, check it, and if you haven’t, consistency problems may well be the least of your worries!

Tags: ,

The significance of significant figures

Apr 7

Posted by Jess in Technical notes | No Comments

At this point in time, the use of significant figures is almost exclusively confined to scientific disciplines. It’s a pity, because these days so many businesses are relying in analytics to provide the basis for some serious decisions and the figures they have are rarely as accurate as they might appear.

The significant figures (or significant digits) from any number are those that contribute meaningfully once uncertainty is taken into account. At DrJess we like to get stuck in and really investigate uncertainty, but the first step is to realise it’s there.

Say Google Analytics says drjess.com/blog site has seen 1621483 unique viewers in the last month. A touch on the generous side, but in thought experiments we are allowed to be optimistic.  Neither the Goog nor almost any other free web analytics suites have much to say on the subject of accuracy, but presenting in any halfway scientific context a number with that many significant figures implies a very high confidence in its accuracy, percentage-wise.

Web analytics pros know that Google Analytics and its tag-based friends tend to under count traffic by rather more than 10%. More on the reasons why in another post, but let’s be generous and assume a maximum 10% error for the sake of a simple thought experiment. Applying that to my imaginary unique user count of somewhere between  1621483 and 1783631.3.

If I had to pick a single number to represent that range, it would be 1700000, not 1621483. Not only is it probably more accurate, it gives anyone looking at it a much better indication of what the uncertainty is.

Now, if I was going to base some serious and potentially very expensive business decisions based on analytics numbers, I’d want to know just how good my figures were. So would most people who want analytics done, if it occurred to them to wonder if precision might be lacking in the first place. With the over-precise numbers spat out by most analytics programs, the question is rarely posed unless inconsistencies crop up.

I’m not blaming Google. The simple truth is that given the choice between a web analytics program that spits out 1700000 and one that spits out 1621483, the overwhelming majority of users will perceive the latter as more accurate.

Legend states that the first surveyor of Mount Everest measured the height at 29000ft exactly. It was reported as 29002ft, supposedly because the surveyor didn’t think anyone would believe his rounder figure had a reasonable degree of precision.

That was back in 1856. Unfortunately it still seems we’re having this kind of problem. The only solution is to state clearly what the uncertainty is. Bring it out of hiding and discuss error, accuracy, and precision before making decisions.

Tags: , ,

Welcome to Dr Jess

Mar 19

Posted by Jess in Administration | No Comments

Welcome to DrJess.com, the new home of JM Spate Web Analytics. Our new site will have a lot more functionality and will eventually host more content than the previous one, and it already has greater bandwidth. We’ll be moving content over to this site over the next few months.