Now holistically speaking, its the same process but being measured through two different measurements systems. And the process I am referring to is a customer’s interaction with the organization (phone, chat, email etc). The two measurement systems in this context are the QA system in the organization and the CSAT program. In an ideal environment, the output from these systems should tell the same story. But then we don’t live in a perfect world, do we? And what generally happens is that organizations tend to measure totally different things through these systems and expect the results to align, which would not be possible by any stretch of imagination.  A typical example would be that customers are asked various questions on a CSAT form  which would have to do nothing with compliance, where the QA process would always measure compliance along with other customer experience drivers. Compliance can be in terms of regulatory bindings or may very well be internal to the organization (what the company wants the agents to do or not to do and then also measure through QA process, for example the agent is not supposed to offer more than $25 worth of discount and also would measure how agents are doing in this regard). Now things like these don’t help at all towards alignment of the QA and CSAT outcome.

Now let’s try to explain what I mean by the term “alignment”. If you believe that its one single process (customers interacting with agents) and there are two different measurement systems (QA and CSAT), you would also agree that these two measurements should present fairly identical picture of what is going on (unless of course it’s intentional on your part to measure different things through these systems). So if overall CSAT is showing an upward trends in the last six months, let’s say, QA should also indicate an improvement. But what I absolutely don’t mean by alignment is that both numbers should be identical – they can never be and are driven by respective threshold performances. In a perfect world, QA should work as a predictive model for CSAT, which essentially means that you can control QA to control your CSAT. In simpler terms, if the customer rates a call as a “failed” call (gives a rating of 1 or a 2 on a scale of 5), that call should also fail the internal QA test. So if everything is aligned, proportion defectives on the CSAT side should match proportion defective on your QA side. And I am talking about statistical match here, not each and every transaction would match for obvious reasons. And then proportion defectives on a QA side are a function of your threshold as well, so we have to be sensitive to that as well.

So how to achieve this alignment?

There are 6 major steps which are a combination of advanced analytic, listening to calls and understanding customer emotions -

1. Key Driver Analysis –

• Analyze transaction level CSAT results to figure out what actually drives the overall customer satisfaction
• Understand opportunities in the current CSAT format (too many or too less questions, format of questions, rating scale etc
• Understand the overall model fit and relative impacts of each of the shortlisted drivers
• Tools – Regression Modeling (mostly ordinal logistic regression because CSAT data is ordinal), DOE, multivariate analysis

2. CSAT / QA alignment verification – Analyze both CSAT and QA performance for the last 6 months

• Statistically prove if QA is a predictive model for CSAT (they move together in the same direction)
• If not, what specific areas?
• Which specific drivers are missing on the QA form (or not properly laid on the QA form)

3. Capturing customer behavior – Why customers rate the way they rate?

• Listen to a significant sample of calls with available CSAT scores
• Connect customer behavior to call handling process and agent skills

4. Translating CSAT drivers to QA attributes – what can be done on the QA side to control (and improve) a CSAT driver performance?

• More statistical analysis
• Converting each driver into corresponding attribute(s) on the QA form

5. Operational definitions and weights – what does an attribute on the QA form means and how important it should be?

• Define each attribute in a way that it captures customer behavior
• Use PI matrix technique to assign weights to attributes (each attribute has a different impact)
• Balance the QA form (include attributes important for compliance and other business objectives)

6. New QA form pilot and tweak

• Listen to a significant sample and compare CSAT and new QA ratings
• Modify and tweak based on the results
• Complete rollout

But then its rare that you will get everything right in the first go, there would have to be multiple iterations based on the results. But once you get there, it is absolutely worth the effort. Imagine you being able to drive significant variation in your CSAT through your QA efforts.

Customer service & product performance —> Customer Satisfaction —-> Customer Loyalty (probability to stay)

First relation can be proved statistically fairly easy, and can be done easily using few months worth of customer satisfaction data. The second relation is somewhat of a debate. Studies say that even if the customers are satisfied, certain portion of them incline to move away but that number isn’t significant compared to ones who have a the probability to stay. So that does mean that if you control satisfaction, you’d ensure that you do control a significant portion of loyalty, others you really can’t do much about (or would you even want to spent time and energy for whom loyalty isn’t in terms of satisfaction).

The term I use to refer CSAT analysis is called “key driver analysis”, which is a pretty generic term but is used extensively in the area of customer satisfaction research. The idea is to create a predictive model by keeping the “overall satisfaction index” on the CSAT questionnaire as the output and all other supporting questions as inputs or “drivers”. Modeling is through techniques like binary or ordinal logistic regression based on the type of data gathered from the questionnaire. Now one important thing to understand is that now we are working at a layer lower than what we have discussed in the previous paragraph. The assumption is that since CSAT index drives loyalty, we are now trying to find what in turn drives CSAT – factors which can be controlled through process improvement or re-engineering.

Based on the numerous satisfaction questionnaires I have analyzed (across various industries and verticals), what I found was that one of the key drivers of satisfaction has always been customer effort. Customer Effort can be defined as the overall effort the customer had to put in to get a resolution for his concern. But what I have also observed is that Customer Effort isn’t the only driver of CSAT index, there are a lot of other things too (Relative impacts of the drivers can be figured out from statistic like odds ratio in the output of regression analysis). And the beauty is that those drivers more or less remain the same irrespective of the industry/vertical but their relative impact varies to a degree.

So the point I am trying to drive here is that Customer Effort is important, but not the only defining factor in deciding how satisfied a customer would be (and in turn driving the loyalty). Every organization needs to collect data not just to figure out short term fixes, but to also to develop long term strategies – which might include changing/modifying how things would be carried out from a business standpoint.

1. Add your own mysql data source

2. Create and publish reports using report designer wizard (next post)

Adding your own mysql data source -

Just by adding the data source in the admin console did not work so I tried searching for some help. I did found a wiki page which is step by step guide to setup your own data source for tomcat (Pentaho 2.0 is on tomcat instead of Jboss so the configuration is different). Though the wiki did help but it still didnt work, I guess there due to incorrect file name to be modified (just my thoughts, I tried this way and it worked). So here are the steps -

A. Configure the datasource in admin console

Adding mysql data source using admin console

Replace localhost with the ip of your server and click on test to ensure the connection is working.

B. Modify the biserver\tomcat\webapps\pentaho\WEB-INF\web.xml file to add a connection -

<resource-ref>

<description>classicmodels</description>

<res-ref-name>jdbc/classicmodels</res-ref-name>

<res-type>javax.sql.datasource</res-type>

<res-auth>container</res-auth>

</resource-ref>

Classicmodels is the name of my mysql database running on localhost. If you intend to access the server from remote, you should use the ip address of the server machine whereever I might use localhost.

C. Add an element to the biserver\tomcat\webapps\pentaho\meta-inf\context.xml (the guide I have mentioned above mentions the modifications to be done to the server.xml file which did not work for me). Add the following lines -

<Resource name=”jdbc/classicmodels” auth=”Container” type=”javax.sql.DataSource”
maxActive=”20″ maxIdle=”5″ maxWait=”10000″ username=”gaurav” password=”password“
factory=”org.apache.commons.dbcp.BasicDataSourceFactory” driverClassName=”com.mysql.jdbcdriver”
url=”jdbc:mysql://localhost/classicmodels” />

Replace with your own mysql db name in the url (do not forget to use the SAME resource name as you have used in web.xml file)

This definition is for a mysql database but you can use other db sources and modify driverclass and url accordingly.

You are done! Through these steps you have added your own mysql datasource to Pentaho 2.0 and now you are (hopefully) ready to create and publish your own reports.

Next Post – Creating and publishing reports on Pentaho 2.0

And then I saw the demo for the brand new Pentaho 2.0 on the website which was a complete makeover. Everything looked so classy (maybe because they moved from Jboss to tomcat but not sure). On top of that they also launched an admin console where you can configure a lot of stuff which was otherwise all manual.

So the first thing I did was to download the new server (its not a suite this time but a standalone server) but good thing is that I can use all other older modules (read report designer, Kettle, etc) without any hastle – they are in any case
“standalone”. I deployed it on my test server and could also access it from other remote machines on the network – phew this is such a pleasure as I don’t have to modify various different files to get Pentaho working as a real server on the network. It has a slick interface with all your reports and solutions lined up on the left hand panel (sample reports for the time being). Adding your data source is easy through the admin console but I wasn’t sure of whether it will work once I start to publish the reports through report designer (adding a datasource in ealier versions of Pentaho was comparatively a pain). I will explain and discuss the complete process of creating and publishing a report on 2.0 in my next post (including the pain I had to go through). User creation is another life saving feature available in the admin console where you create new roles and assign users accordingly (I haven’t explored the feature much I will keep you posted).

So all in all, a fantastic new release which would make lives of people like me a lot easier.

Next Post – Adding mysql datasource to Pentaho 2.0

In my last post , I described various steps to get away from a spreadmart (read spreadsheets) and using an open source database mysql as a central data repository. After you’ve installed mysql, designed your database, created relevent tables in mysql and dumped some time worth’s data, it is time now to instal Pentaho. If you dont know what Pentaho is, here is an excerpt from their site -

“Pentaho is the commercial open source alternative for business intelligence (BI). Pentaho Open BI Suite provides comprehensive reporting, OLAP analysis, dashboards, data integration, data mining and a BI platform that have made it the world’s leading and most widely deployed open source BI suite. Pentaho’s commercial open source business model eliminates software license fees, providing support, services, and product enhancements via an annual subscription. In the years since Pentaho’s inception as the pioneer in commercial open source BI, Pentaho’s products have been downloaded more than three million times, with production deployments at companies ranging from small organizations to The Global 2000.”

You can visit them at http://www.pentaho.com

There are different ways in which you can proceed for Pentaho Installation (I will refer to windows installation)-

1. Through a bit rock installer

2. Preconfigured Installation

A bit rock installer is probably the easiest way to instal Pentaho (though it became available on recently). You just have to download the executable and run it. Make sure that your mysql services are up and running because it will create some installation databases. After you run the demo, it will ask what type of installation you would want. Select the 2nd option (or other based on your choice) which would allow to create a new repository in an existing mysql setup. It will then ask for user credentials which you might have to manually create. It for some reason did not take “empty” password for root account. After providing all this information, it takes a while to instal tne entire BI suite (if you have selected all the BI suite modules to be installed).

Here is one thing that might confuse (if confused me ok) – when you start the installer, it would give you 3 options for mysql setup – install mysql server and create a new respository, create a new repository on an existing mysql setup and use an existing repository on a mysql system. I chose the last option thinking that it was referring to my datasource (my own business intelligence database in mysql). But what it meant was that Pentaho has to instal its own database repository in the form of 3 different databases (hibernate, quartz and sampledb). So if you select the 3rd option, you have to manually create these databases first and assign a user to them. So I would advice that you should go ahead and use option number 2.

A pre-configured configuration comes as a zip file with all the options being preconfigured. There are 2 pre configured options – with a HyperSonic (included) database or a mysql (required) database. The one with Hypersonic is easy to get up and running because the bundle comes with inbuilt hypersonic databases which launches itself everytime Pentaho BI suite is launched. If you have to instal the mysql version, you would have to instal the databases first (I would recommend that you refer installation documentation for more information).

My first deployment was a hypersonic version and I had to browse through numerous documents to figure out how to plugin my own database into the whole system. The BI suite would run just fine with its own sample database, but to get your own data stream into the server was a pain.

In my next post, I will talk about how to plug in your own mysql database into Pentaho BI suite and start working on it.

Till then, Enjoy!!!

Its a shame that I’ve not been blogging for so long. But some of my business trips outside the country and my other committments have kept me busy. Anyways, I will try to get back to a habit of regular posts based on my experiences.

As I have mentioned earlier in my post, the biggest challenge I am facing with a spreadsheet based BI system is the consistency of data and offcourse the time lag between recieving a request and delivering the output. There are tonns of other pitfals of otherwise easy to manage system (which I am trying to get rid of, hopefully soon). So my first obvious step was to move away from spreadsheets and create a central data repository – and for a small or a mid sized organization, there is nothing better than mysql. It has the features of a robust commercial database and yet it comes for free.

Step 1 – The first step towards this initiative was to understand the business needs (from a BI perspective) and design someting on a piece of paper that would be later converted into a formal database structure. I am not a database guru, but there are tonns of resources available online to get started. I am not going to talk much about how I designed and implemented my database (this is something you have to learn online and implement). But after this excercise, I had list of tables in mysql ready to be fed with data.

Step 2 – The next step was obviously to get some test (or real) data into the database to see everything is working as per the earlier design. If in case the data is on the worksheets, some sort of ETL (Extract, transform and Load) software has to be used to transfer data from spreadsheets to mysql. There are lot of software available on the internet, but I guess Navicat worked best for me. It is fairly easy to map your spreadsheet columns to the fields in your tables in mysql, rest everything is taken care by Navicat. Do remember to check the data in mysql after every transfer process to ensure that everything was transferred correctly or not.

With test data in the database (I would recommend atleast 3 months of data in all tables), you are ready to jump on to the next big level – deployment of BI Solution (Pentaho in my case).

Though the deployment process was a little complex earlier, but the inclusion of a bitrock windows installer has made things a lot easier now. Just download the executable and start installing. I will talk a little about the installation piece in my next blog.

Until then, have fun deploying

Apart from my other normal duties, I also have the responsibility of handling the Business Intelligence function in the organization. This function started off as a 2 member MIS team with MS Excel being the only tool at their disposal (And this would be true for almost all small to medium sized organizations). I somehow don’t like Excel for some obvious reasons (Though its a great tool) and have been trying get away from the spreadsheet business “spread” all over the place.

My primary goals:

1. A centralized repository for all my data

2. Ability to create a reporting layer with some canned reports without much of a manual coding work

3. Dashboarding

4. A possible web based interface for my users with customized home pages

5. Analysis of business data

All these requirements though seem wonderful, but is a daunting challenge to be even anywhere near to them (Specially if you are a small company with almost no money to spend on pricey software).

I have been trying various things which I thought might click (Without really thinking of how to do them eventually), like for example a test deployment of Palo technology (Open source OLAP). Problem with PALO is that it helps in my first goal of a central data repository, but everything else is so difficult (Specially for someone with very less coding skills). Though OLAP provides a very nice functionality of drill down analysis, everything seems a waste if I can’t create reports on top of it (Without much of a coding that is).  So I had to backtrack.

Then somehow I received an invitation for a webinar on Open Source BI technologies. I attended and came to know about two major players in the Open Source BI market -

Pentaho BI Suite

JasperSoft

After months of research and feasibility analysis,  I came to the conclusion that Pentaho is best suited for my needs for some specific reasons -

1. A combined suite with reporting, dashboarding, analysis and other modules built in (Including Mondrian OLAP and Kettle ETL).

2. Its increasing work and documentation with mysql (Makes me smile as I have the basic understanding of mysql)

3. GUI report building which almost anyone can learn (With some basic knowledge of query language)

So I finally made the decision of moving my entire BI function onto Pentaho (And I have only started). As I stated in my first post, I am going to keep my promise of giving back to the Open Source community by documenting each and every step of my battle towards automating my spreadsheet based BI function. I have already some initial inroads, and would discuss in my next posts. If you think you are one like me, please support this project and keep me motivated (A comment on this post is what I consider as a “support”).

In my free time, I have been trying to help the organization with its automation drive (We still are a small company and its kind of difficult to invest hundred thousand dollars in commercial software). Everytime I google for any of my requirement, I am amazed by the kind of work that is being carried in the Open Source world. There is alwayswh someone (Or a group or a company) who are working in some specific area in the Open Source domain. For an SME like mine, it makes absolute sense not to invest too much in software, specially when requirements can be met easily with the kind of Open Source software available online.

I’ve have see almost all functions in my organization battling out in a manual environment just because they cannot lay their hands on that fancy expensive software. The only thing they fail to realize is that world doesnt ends with proprietry software – its way beyond that. If they can invest even little (In terms of manpower not money) in exploring the possibilities, they can get decent sofware working for them – at zero cost. But the first question that pops up in their mind – how can something like that be for free – they need to be educated.

Some of the Open Source software I have installed at my organization are:
1. CORE: Open source project management
2. dotproject: Open source project management
3. knowledgetree: Open source DMS
4. Mamook: Open source LMS
5. OrangeHrm2: Open source HRIS
6. phpesp: Open source survey software
7. phpscheduleIt: Open source scheduler
8. Joomla: Open Source CMS

And there are lot of others in the pipeline. Open Source is an obvious path for the SMEs to go for, and sooner the managers realize this, more profitable and efficient they will be. I am trying to contribute back to the community by writing this blog, and I will continue by writing reviews of what I have seen, tweaks, hacks etc. I want others to benefit – the way I have till now