Measuring Customer Satisfaction.aspx

suggest edit
Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

44 responses

  1. Avatar for Joshua Freislich
    Joshua Freislich February 3rd, 2010
  2. Avatar for Galilyou
    Galilyou February 3rd, 2010

    The most honest way to tell if a product is successful is the number of clients using it. Checking out how many sites are built with MVC is an obvious way to count the number of users. Afterwards, support tickets, polls, even posts on the blog-sphere can provide some help.
    For a personal feedback, though, as a client who built a commercial site with MVC 1.0, i'm pretty satisfied.

  3. Avatar for haacked
    haacked February 3rd, 2010

    @Galilyou the number of clients using it is a difficult number to measure with any accuracy. I think the point of these other metrics (this blog post only addresses one metric, but there are others) is to provide extra corroborating evidence.

  4. Avatar for Chris Hardy
    Chris Hardy February 3rd, 2010

    I would go with getsatisfaction.com - from what I remember it seemed to work pretty well for Silverlight 3/4.

  5. Avatar for haacked
    haacked February 3rd, 2010

    Perhaps one metric to focus on is am I and my team responding to feedback? If I show how many bugs were accepted as a result of customer feedback, that would indicate we're effectively collecting customer feedback.

  6. Avatar for Louis DeJardin
    Louis DeJardin February 3rd, 2010

    Hmm... I think there's a hidden trap there in the word "quantitatively."
    I started thinking - maybe a focus group? But then thought you could end up optimizing something like how people felt about their relationship with the framework's brand, or some other tangent.
    Occured to me then even with the best of intentions any measure of a nebulous concept like performance or satisfaction runs the huge risk of becoming an end of its own rather than the means to measure something else.
    Like when outsourcing a call center. By the numbers - minutes per call, percent of issues closed on first call, etc. You could be great on paper. But when you seek to optimize those numbers you lose site of the qualitative values. Like the actual service you get is awful.

  7. Avatar for Louis DeJardin
    Louis DeJardin February 3rd, 2010

    For example, you wouldn't want to be in a position where an increase in the number of bugs get from customer feedback year over year equals better service on paper. ;)

  8. Avatar for Romage
    Romage February 3rd, 2010

    Actions speak louder than words. Check on the number of downloads of MVC. I think a separate quantitative measure could be by creating some "how to" vids, and monitor the downloads. You can also see how many twitter / blog followers you have etc.

  9. Avatar for Filip Ekberg
    Filip Ekberg February 3rd, 2010

    You can't really measure success by counting the downloads, I bet there are a lot of downloads that are never really installed. Even those who are satisfied and re-download them.
    So without using any of those "Poll"-sites out there, maybe there should be a public ( even more public one ) bug-tracker where the community actually can help each other out.
    And re-use the "Did this help you"-button from MSDN on all your videos and put a little "1 - 5"-star-rater next to it.
    That would probably cover a little bit more than just doing an online survey..

  10. Avatar for Kyle Hodgson
    Kyle Hodgson February 3rd, 2010

    Wonder if MVC would be visible in say a Netcraft survey. Does IIS indicate MVC somehow in HTTP headers?

  11. Avatar for Craig Stuntz
    Craig Stuntz February 3rd, 2010

    Phil,
    I ran a query against the Stack Overflow public data dump which shows a steady and continuous increase in MVC questions per month:
    http://stackql.net/default.aspx?qid=42654
    SO just released another dump, but it doesn't look like it's on stackql.net, yet, so this only goes through mid-September 2009.
    Still, I hope it's helpful.

  12. Avatar for Mat Roberts
    Mat Roberts February 3rd, 2010

    I haven't read this book, but it sounds like the type of thing your trying to do:
    www.amazon.co.uk/.../ref=sr_1_1

  13. Avatar for Javier Lozano
    Javier Lozano February 3rd, 2010

    Dude, give them this link: http://bit.ly/a8bz3u Thats should do the trick. :)

  14. Avatar for Eric
    Eric February 3rd, 2010

    An attempt to do a Twitter search for tweets like this and this.

  15. Avatar for Andrei Rînea
    Andrei Rînea February 3rd, 2010

    @Kyle Hodgson : There is (at least in MVC 1.0) an HTTP header called "X-AspNetMvc-Version".

  16. Avatar for Emmanuel
    Emmanuel February 3rd, 2010

    I say check the message boards, discussion groups, stackoverflow, etc. Not how many questions are being asked, but how many users participating, answering each other questions, recommending different solutions. How vibrant is the community, how many other user are helping other users. If you're satisfied with a product, you'll recommend it to your friend, you'll help others implement it.

  17. Avatar for Travis Illig
    Travis Illig February 3rd, 2010

    I'd avoid using polls since the participants aren't a statistically random set of people. You'll probably end up with either an improbably large number of negative results (since people love registering their discontent) or an improbably large number of positive results (very vocal supporters... which would help you from a corporate perspective but wouldn't accurately represent satisfaction). Online polls where you voluntarily participate aren't really objective.
    getsatisfaction.com is probably the best suggestion I've seen. More accessible than your typical issue tracker and allows people to leave both positive and negative feedback.

  18. Avatar for Portman
    Portman February 3rd, 2010

    Yes, you can measure customer satisfaction. Airlines, Amazon.com, and the American Medical Association do it all the time. And that's just the "A"s.
    But the prerequisite to measuring is collecting data. And that's your problem with Microsoft developer tools like MVC.

    how can I measurably demonstrate that [...] my team is responding to customer feedback?


    Where is the customer feedback coming from? Current answer: all the frickin internet. It's on blog comments, Twitter, MSDN, Connect, StackOverflow, private emails, etc. There's no one official place for MVC developers to provide feedback. Therefore you can't collect any data, and you can't measure.
    You need to pick one place where your customers are supposed to go to complain. Then you can measure how responsive you're being to feedback and how satisfied your customers are. Talk to some PMs with Amazon's customer service team - they are very good at this.

  19. Avatar for ciro
    ciro February 3rd, 2010

    I'm a big fan of immediate feedback and I like the idea to give to the users some intruments to express their opinions and make them feel that they are listened. I like put an informal rudimental rating system directly inside the 'About' dialog and possibly show in real time the number of downloads to build up a sense of community. I envision a (possible not) introusive way where customers can rate single features directly inside the software, or at least prepare a special 'beta' version with the feedback process directly engraved in my projects

  20. Avatar for Andrew Hay
    Andrew Hay February 3rd, 2010

    There are services that measure the tone and mood of the conversation on the Internet. The goal of the service is to discover conversations about your area of interest and then measure the mood, influence and trend. Corporations and government organizations really get into this stuff when they stand to gain/lose a lot based on public perception.
    It's not too far of a stretch to turn something like this, http://waggeneredstrom.com/nn/demo/index.html, into a measurement device for conversations around ASP.Net, et al. There are other providers that offer this service too.
    The license might be out of your budget range, but it would be a sweet graph to see.

  21. Avatar for John
    John February 4th, 2010

    Good luck.
    I'm not a cynical person, but my first take on this is that this will just be another number for bean counters to pass to their bosses who pass to their bosses, etc.
    The reason that I call them numbers for bean counters is that they're arbitrarily created and evaluated, but don't do actually do anything. Suppose you score an 8 out of 10, how much would getting a 9 affect MVC use? A 10? How much money would Microsoft potentially lose if it drops to a 6? A 5? How does the number affected investments?
    The real challenge is to find metrics that can be related higher level metrics, which comes from a methodology known as The Balanced Scorecard. You should be able to relate the success of your efforts to the success of your customer metrics. And you should then be able to relate your customer metrics to profitability.
    Find a customer measure that can be continuously measured. How, in turn, does satisfaction affect adoption and, presumably, profitability? Maybe you want to look at adoption rate among alpha developers. Maybe you want to do analysis of people actually using MVC -- are they using anti-patterns or bad practices, because they don't know about better ones?
    Oops, sorry, I forgot that Steve Ballmer is in charge there. (I still remember Ballmer declaring Vista a tremendous success and the iPhone a flop, because it didn't have a keyboard). In that case, send out a survey to all MVPs.

  22. Avatar for Roberto Sebestyen
    Roberto Sebestyen February 4th, 2010

    Take a look at ITIL and Service Desk and Incident Management. Although that may be overkill for a small project, it is indispensable to large corporations. Check out http://www.thinkitsm.com/

  23. Avatar for rickj
    rickj February 4th, 2010

    Check out how many people are using the forum the more people using it the more problems there are that need to be solved I think you guys are doing a great job and I allways get the support
    and access to training to get me through put in one positive vote for me
    Thanks

  24. Avatar for Dave
    Dave February 4th, 2010

    I would just like to add that we only build sites
    in mvc now but to really know the world wide
    opinion your best bet is prob a simple poll
    cant beat em for instant results and voting on
    features. Keep up the good work

  25. Avatar for Craig McGuff
    Craig McGuff February 4th, 2010

    Instead of trying to think outside the square, try thinking inside it - Microsoft is a big company, with lots of other software projects that probably have the same or similar measurement/reporting requirements. Without knowing the internal politics, I would try and speak to someone else in a similar role that has already done this (in a way that satisfies the powers that be, and is productive). Reuse is not just about software :)

  26. Avatar for Pablo Cifuentes
    Pablo Cifuentes February 4th, 2010

    Exposed additional training,video and blogs, then gage the number of hits and comments. I believe the number of potencial learners and audience can provide you with an important hint to the number of people joining the MVC club. Always include two side to every asp net forum question, adding more information of solution on both platforms.
    Continue with the great work and people with eventually value the product as yet again an important tool to our daily work.

  27. Avatar for Klaus Graefensteiner
    Klaus Graefensteiner February 4th, 2010

    You only hear from a satisfied customer when she buys your product or service again.

  28. Avatar for Asad Ali Butt
    Asad Ali Butt February 4th, 2010

    using downloads as metrics could be just a part of the measurement stretegy.
    There are customers who will download, install, test and use.
    Then there are clients which download , install and test.
    Then there are clients which download and test only.
    Then those who download and install only, but never have time for it and then those who only download and then forget untill after there PM announces that some project is going to have a touch with ASP.NET MVC.
    I feel all downloads account for only 25 / 30 % of real users
    your real figures should consist of real customer. A client is one who is your client for life time of the product, not who just downloads.

  29. Avatar for NC
    NC February 4th, 2010

    IM NOT SATISFIED!!!!
    Your making something extra for me to learn. Im not happy.
    /end

  30. Avatar for Anthony
    Anthony February 4th, 2010

    Maybe also by how many emails you and your team get over asp.net mvc. If you are getting lots of emails and helping people out then thats a good thing. It takes time for people to write emails asking questions, so I would think that this is something worth looking at.
    Also maybe the number of questions and answer on stackoverflow... it shows how many questions are asked for particular topics, have a look and see what people are taking about.
    Just a few thoughts.

  31. Avatar for Andri Haraldsson
    Andri Haraldsson February 4th, 2010

    Wow. I'm actually working on a product that in part addresses exactly this problem, using MVC as the front-end.
    The comments so far have pointed out a lot of the short-comings. The one that talked about balanced scorecard even went so far as to try to talk about meaning of the counted things, as opposed to just how to count things.
    Here's what I'm working on. Ask people what is important to them. Get real priorities (that's the magic sauce we got). Let them tell you which is more important, features, or stability, or standard's compliance, or extendibility, or whatever. Because if you don't know what's most important to people, you cannot really measure their satisfaction. I mean you can get a number, but not much insight. If everyone tells you they are really happy with the decor in your take-out pizza joint, that's probably not going to contribute as much to actual satisfaction as the quality of your toppings. And if you just asked about decor and toppings -- and then averaged the two to come up with one number. That would be almost as bad. And it's what most people do.
    So, what our product does (beta coming out in March -- you can try it for free) is that it creates a simple matrix. You ask customers to prioritize the issues that are most important to them, and then tell you how well you are performing on those issues. Combined with demographic info on respondents, these answers can give you very deep insight into what's going on. We can even track info over time to see if there is a noticeable response/chance after a new release.
    If you're interested, I'd be happy to share more. Just email me at my firstname@mylastname.com

  32. Avatar for Tarik Guney
    Tarik Guney February 4th, 2010

    Frankly, me and thousands of people like me don't like to fill out surveys as they are boring. The best way would be a integrated environment into products which collects data about how customers are using the specific products and on which parts they are mostly spending their times. I don't know it would be violating the privacy policies though. When people are working, if they allow, system can ask short questions about the part they are using and send them to the creator team along with the data collected during system use.
    For example, lets say, a customer is using Visual Studio. The customer is writing codes within it, he/she probably uses the Intellisense. What if he doesn't like it ? He will probably disable this feature and here is the data you should collect. While he is disabling, system can ask a quick question about the reason and provides a little text box to allow him to write some suggestions. It would be easy and mostly unboring way instead of handing over 100 question surveys.

  33. Avatar for Dhananjay Goyani
    Dhananjay Goyani February 4th, 2010

    I am not saying anything fundamentally different than what is been talked already in the comments. Apart from no. of downloads, blogs, connect, etc here are my two cents:
    1) Google search results for '.NET MVC' or something. More results means more voice meaning more coverage. You can also present .NET MVC book stat, may be. I believe that people positive-negative expression ration is 80-20. On the net people want to share their experiences and specifically what went well (positive part - 80%) rather than what went poor (neg part - 20%). So, more google results (and increasing...) obviously means good thing. Say, how many people actively consuming Card Space and in turn how the product is performing. Compare google data over a period. (Obviously, you cant do that right away, its a continuous process...)
    2) Stackoverflow. Yes, it can turn as powerful tool on your side. I know you (and your team) actively follow the threads and try to answer queries in as satisfying manner as you can. You can present your team scores (and also include non MS people but active in MVC community) as to show how much effort you put answering community - and its outcome.
    Thanks!
    Dhananjay

  34. Avatar for Mahyar Pasarzangene
    Mahyar Pasarzangene February 4th, 2010

    Hi !
    I think you can not even accomplish this task , i was someday in Razi university of Kermanshah Iran , and our computer science professor Dr.Montazeri always have problem with unmeasurable world of computer and he was saying because of this fact we should never call the "computer science" , the "computer engineering" !

  35. Avatar for Abdu
    Abdu February 5th, 2010

    Measuring the number of downloads doesn't mean much. Most it can say is a lot of people are downloading it. A number has to be a percentage of another number or a rank to be meaningful. How many people who downloaded it are actually using it?
    The amount of "flood" of incoming bugs is an indication. Feeling the general sentiment in the forums on the web is another. Are people in general complaining or happy and satisfied?
    Are the common pain points people talk about attack narrow or wide aspects of your product?

  36. Avatar for Adam Ramshaw
    Adam Ramshaw February 6th, 2010

    Before you ask yourself how to measure customer satisfaction (and there are many ways to do it) you need to ask why you want to measure it. If you're thinking that customer satisfaction is a driver of product take up you're probably going to be disappointed. There is quite a lot of research to show that customer sat is not greatly aligned with product take-up or revenue growth (or anything really).
    If you want to measure a variable that is more often aligned to growth then I suggest that you look towards Net Promoter Score. This has some good support.
    How to measure it? I'd suggest sending an email to downloaders a few days after they had downloaded and ask them the NPS question: How likely would you be to recommend... to a friend or colleague" along with "Why do you say that". Just two questions. You could use Survey Monkey to do it if you like. Then do the NPS maths and look at the comments.
    This is a pretty robust way to start down the path towards really understanding what is driving user loyalty. We use it a lot for our clients.

  37. Avatar for Visual C# Kicks
    Visual C# Kicks February 8th, 2010

    I like polls but they are a hassle since you need a very large number of people to get a realistic sample.

  38. Avatar for cttoy
    cttoy February 8th, 2010

    The ASP.NET MVC is good, but the windows & database are expensive, I am satisfied with ASP.NET MVC, but it is not so easy to make a start business website.
    So, when to make a linux/unix version of ASP.NET MVC? (mono project is slower than MS)
    No limits, Just Use.

  39. Avatar for Glenn
    Glenn February 10th, 2010

    Just like Adam, I too believe that NPS is the best way to gauge how customers view your product and willingness to act on feedback.

  40. Avatar for Nour
    Nour February 13th, 2010

    You may think to create a special custom feedback system for getting developers feedback. You may need to use third-party feedback systems like: Zodiac.NET to create some special kind of surveys.

  41. Avatar for SEOINKÖLN
    SEOINKÖLN March 8th, 2010

    intressanter beitrag

  42. Avatar for Jeremy
    Jeremy March 17th, 2010

    I would start by reading this book: www.amazon.com/.../188516730X/
    This will show you what the right thing to measure is, and if I remember correctly give a couple of suggestions how to measure it.

  43. Avatar for Arvid de
    Arvid de September 19th, 2010

    @ Joshua, thank for your tip. Getsatisfaction.com is a great site, I think this is a good answer for this post.

  44. Avatar for Sean Carty
    Sean Carty December 29th, 2011

    Like others have said, I think there are a bevy of ways to get some measurable customer satisfaction data, Including GetSatisfaction.com, and since this question has been asked a year ago a number of other great resources have blossomed. My own site, StepByStepMarketing.com, has its own section dedicated just to this topic at www.stepbystepmarketing.com/... We have highlighted a good number of high quality ideas and external services you can use. I know I am a little late for the original poster, but hopefully this will help others with similar questions that end up here.