Wow! Something from IT that actually works.

As many of you know at the beginning of the year I was seconded into IT for 3 days a week (now only 1 day per week) to help them with their business reporting.

It has been a very frustrating time for me as we’ve been using the new data warehouse, in principle it’s a great idea having all of your data connected and in the same place but in practise I have rarely seen it work.

I am using COGNOS to interrogate the data and have found it so slow and unpredictable. It almost brings me to tears when it takes 30 seconds to run a report one day then the next it takes more than 30 minutes.

With the particular data set that I use, a major problem is that the data warehouse stores more than 100 fields of data on a transactional basis. This means that we have over a million records with 100+ bits of info each. Simple maths tells me that that’s over 100,000,000 single pieces of data. It is no doubt that it takes so long to summarise the data. In fact as the company will only ever get more information it will only become more of an issue.

In reality, to do my job (this part of it) I can do 95% of my analysis using only 20 fields and a grouped up level of data. A much more manageable 100,000 lines of data, for those with a maths GCSE that’s 2,000,000 bits of info, 50 times less than before.

Anyway… I suggested this nearly 7 months ago and they’ve finally done it for me. A report that I’ve created in the old system used to take between 10 minute and an hour now took less than 1 minute to run on my first attempt. You might think that that is impressive but that’s not the half of it. The development and testing of these reports will be much more time efficient, rather than running a report to see if it works then twiddling my thumbs and hanging around on the off chance that the report works. I will now know practically instantly.

Thank you IT. I never thought I’d hear myself saying that.

I think my experience in IT would have been much better had they listened to me from the beginning but hey, I’m sure I’ll get over it…