Working Differently
  • Home
  • Working Differently Blog
  • Learning Ovations
  • Book Reviews
  • Presentations & Testimonials
  • Bio and Links
  • Contact Jay

Understanding Measures: Moving from Counting to Accomplishing

6/11/2013

4 Comments

 
Picture
A tsunami of measuring things has hit our community efforts over the past decade.  For the most part, this "run it like a business" mantra has done little harm, but it also has not gotten us to the promised land of transformative change.

There have been critics on both sides of the exercise.  Some will talk about their belief that important things, by their very nature, do not lend themselves to be measured.  While others will identify the insufficiency of the measures chosen.  Still more have called it a distracting fad that takes providers away from the important work of good people doing good things.

Perhaps the best place to start our thinking about how best optimize and tame this measurement fever is the fact that 90% of what we are required to measure in community -- be it because of grant mandates, donor or Board requests, or other well meaning impositions -- fail to measure those aspects of our work, especially our collaborative work, which truly represents the path to transformative and sustainable outcomes.

This 2x2 matrix shows the four quadrants of types of measures available to monitoring our work and progress.  Measures can be thought of in terms of quantity or quality of the work being measured AND the amount of effort or effect such work requires or delivers.

The vast majority of the measures, indicators, mileposts that we have used in community are stuck in the first quadrant --  counting quantity and effort.  Here we look at number of clients served, seats available, units built.  We are focused on what we do, assuming that will be transformative in and of itself.  Most of the public policy and advocacy work in this quadrant is about access and funding more of these same types of activities.  

The dilemma here is multi-fold: 
(1) It tends to solidify our definition of success around what we are already doing, i.e., though, we've failed to achieve our desired outcomes over the past quarter-century, we would be successful if we only did more of it.  Einstien's definition of insanity certainly comes to mind -- doing more of the same, hoping for a different result.
(2) By emphasizing the count versus the result, we tend to heighten the competition around the action -- agency A is better than agency B because their overhead is lower.  
(3) It tends to crowd out innovation be it in activity, result or data because we built momentum around maintaining where we have already been, as opposed to where we want to go.  If fact, in most communities the definition of the ideal future is a better funded past.

Quadrant #2 moves us beyond counting activities, yet it is still limited in scope by framing what matters around the quality of those who are performing the activities.  Imagine: This community runs the very best "Dare" program!  Irrespective of whether there is any evidence that "Dare" programs causally impact teen drug use.

Quadrant #3, though back to counting, is at least looking outside the provider frame for an assessment of impact.  Unlike the first two quadrants, since we have now moved to the part of the matrix dealing with effect, we need to consider more directly whether our activities are in fact the path to improved outcomes.  The first two quadrants were about agency, we are now shifting to impact.  Yet, we are still a level removed from sustainable change.  We are taking a community vessel and filling it with good things without the ultimate connection between whether these "good" things create a good lasting result in the community.  "Self-Esteem," "High School Graduation Rates,"  "Worker Retraining Skills"  are all examples of Quadrant #3 measures, which are not sufficient to tell us whether we have achieve the tipping point in community success.

It is only Quadrant #4 measures which begin to outline the collaborative, collective impact our communities deserve.  The other three quadrants are about counting, whereas here we are measuring the heavy lifting of a community actually accomplishing something. 

As you move from Quadrant #1 to #4, not only are you shifting your collective focus from good people doing good thing to good people accomplishing good results, but you are also declaring what it is that you need all of those other quadrant measures aligning to.  This quadrant thus becomes the ultimate community accountability.

A very worthwhile test for your organization, partnership or community is to assess all measures you are presently using against Quadrant #4.  If you are NOT tracking measures here, you are simply being busy.  


To be tweeted links to my new posts -- blog, book reviews (both nonfiction and fiction), data or other recommended tools -- either go to Twitter.com and follow me @jcrubicon, or just go to my Home page and click on the Twitter button on the right, just above the tweet stream, and follow me @jcrubicon.
4 Comments
Jim Toscano link
6/16/2013 12:49:04 pm

Lots of good analysis here. What you are getting at are three very important things. One: we can measure almost anything.
Two: Measurement needs to be comparative. Three: measurement needs to allow continuous quality improvement.
See the literature on performance management for some complementary ideas.

Reply
Jay Connor link
6/20/2013 02:55:55 am

Thanks, Jim.

To your third requirements re continuous improvement should also be added the flexibility to shift measures over time as you discover through your measures better ways of quantifying the required steps to the outcome.

Reply
Nathan link
3/24/2014 02:06:35 am

Good article! I visited many sites, only the clear expression in addition to the author's experience. Read your article, I learned a lot of knowledge

Reply
Millwork Gresham link
9/15/2022 12:09:40 pm

Great bblog

Reply



Leave a Reply.

    Author

    Jay Connor.  In working with over 75 communities in North America, I came to a growing recognition of the need to develop evidence-based tools in order to achieve transformative outcomes in our community systems – most notably education.  This is a driving consideration in my work and in this blog. 

    Connor Bio

    Archives

    January 2014
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013

    Categories

    All
    A2i PreK 12 Education Outcomes
    A2i - PreK-12 Education Outcomes
    Community Outcomes
    Educational Outcomes
    Tools

    RSS Feed

Powered by Create your own unique website with customizable templates.