TEN THINGS YOU NEED TO KNOW ABOUT EVALUATION

Every so often I get phoned up by someone wanting to undertake an evaluation for the first time. One day as an exercise, I decided to jot down everything I tend to say to them.





NETWORKS, COORDINATION AND PARTNERSHIPS

Networks, coordinations, collaborations and partnerships between government and non-government agencies have become very trendy in New Zealand. It will be interesting to see how this trend pans out. The commitment seems to be there, but the last fifteen years of compulsory competition will take some while to work its way out of the system. I only hope the various initiatives are given appropriate time, tasks, support and resources to demonstrate their potential.

As part of a job evaluating a particular collaborative venture, I was asked to find out what had been written about networks, partnerships and all that. The original version of this download was product of that. Since then with the help and support of other colleagues, I've edited it down to the rather more coherent piece you can get here.

Since this was written several other references have come my way. This includes a more formal literature search undertaken by the New Zealand Ministry of Social Development, and a book edited by Myrna Mandell called Getting Results Through Collaboration: Networks and Network Structures for Public Policy and Management , Quorum Books. There is also an on-line "collaboration inventory" here where you can assess how well your particular collaborations are doing. This framework is produced by the Wilder Foundation and claims to be based on research into collaborations. It certainly seems to include all the major factors.

My thanks to the several dozen people who helped me with this summary. My particular thanks to those whose work is quoted in this summary, and to Judi Altinkaya, then Chief Advisor for Adult & Community Education here in New Zealand who gave permission for the original document to be made widely available.  Thanks also to Penny Hawkins then of the NZ Department of Child Youth and Family, and Patricia Rogers of CIRCLE for their support in developing this document.



KEY FEATURES OF ACTION RESEARCH

I was asked a while ago to describe the key features of action research. And this was the result. I still don't think it quite gets there, but it's a good start. Actually I think THE key feature is the quality of relationships. "Ordinary" research tends to emphasise more the quality of questions. As someone said to me the other day when I made that distinction, good quality relationships actually allow you to ask better questions. Nice point I thought.




PERFORMANCE INDICATORS - THE TYRANNY OF NUMBERS

From time to time I get hot under the collar about the current trend for measuring everything and the unquestioning use of "performance indicators". Here's something I wrote when the temperature singed my shirt. It draws from a variety of sources, including a small bit of research by yours truly. The article was originally published in the Australasian Evaluation Society (AES) "News and Comment".




ZEN AND THE ART OF CONTRACT MAINTENANCE

This piece has a long history. Back in the early 90's Mary-Jane Rivers and I were asked to help negotiate contracts between a government agency and some community organisations. After some head scratching, this is how we approached the job. First we decided that contracts were fundamentally about relationships. We then asked ourselves what were the building blocks of good relationships ? We identified six. We then used these six core features to design a form of strategic planning which we put the various parties through. The actual contract document then fell out of this process.

For a while the approach was very successful, and we trekked around New Zealand helping people develop these contracts. Then the New Zealand public sector was thrust into a contracting environment that was the antithesis of what our approach sought to achieve. It was competitive, divisive, distrustful, and exploitative. The idea of contracts being based on good relationships was a goner, and with it went our little process. I was so bloody angry with what I saw happen as a result that I wrote this article for the "Public Sector" - the NZ Public Sector Association's journal. It was picked up and published in several other journals.

The download is the final version, written for the Australian Community Health Association. Apart from the general discussion of contracting it also includes a sample based on a real contract. I think the article has stood the test of time, although I'd probably make a better job if it today. I recently heard that some organisations still use this contracting approach ten years on.


SURVIVORS GUIDE TO SELF EMPLOYMENT

I've been self employed for over two decades, and I've often wondered how I've done it. Actually quite a lot of people wonder the same thing, and regularly phone me up to find out. So this is what I send them these days. My thanks to Mike Sherry who added a few more tips.

OTHER PEOPLE'S WRITINGS

As promised here is a space for things written by other people that are not exactly "tools" but worth a place in anyone's brain.  As always please acknowledge the original source if you ever get to use them.

TOOLS FOR ORGANISATIONAL CHANGE AND LEADERSHIP - THE CULTURAL FACTOR

The organisational development consultant Dick Axelrod (http://www.axelrodgroup.com) and I were talking one day about national cultural archetypes and the implications for organisational development.  Dick told me of a conversation with the Japanese manager of a Japanese automobile assembly plant located in the southern USA.  The manager mentioned to Dick that when he set a tolerance range in Japan - indicating the range by spreading apart his finger and thumb - Japanese workers would hit right in the middle, whereas in the States the engineers would give him anything within the tolerance range. "So what do you do ?" asked Dick.   "Oh that's easy" replied the manager smiling.  And with that he reduced the gap between his finger and thumb until they almost met.

The relationship between culture, motivation and change has always interested me.  As a migrant who has worked in several countries I'm acutely aware that what motivates people in one country doesn't do the same in another.  Even words have different meaning.  I'm far from the only one to make that observation.  In the mid 80's the US based  AT&T were puzzled why Quality Assurance measures imported from Japan didn't seem to work in the US.  As the previous paragraph suggests it wasn't because US workers couldn't produce quality work.  So they wondered whether US workers engaged with the idea of "quality" differently from their Japanese colleagues.  The research they did on this basically said "you have no idea how profound that difference is" and changed the way they promoted quality in their US workplaces.   Australian consultant Colin Pidd (and his one-time colleague John Evans) took these ideas and generalised them beyond "quality" into a series of tools that could be used in Australian and New Zealand workplaces to help with strategy, motivation, change processes and leadership development.  Their ideas are some of the most valuable on this website, and have got me out of no end of difficult situations.



INTRODUCTION TO THE WORK OF ARGYRIS AND SCHON

Chris Argyris and Don Schon's work has been enormously influential on my thinking and practice (e.g. the Values in Action workshop). Here is Bob Dick and Tim Dalmau's elegant attempt to explain their work and it's significance.

ACCOUNTABILITY, RESPONSIBILITY AND CORRUPTION: MANAGING THE PUBLIC PRODUCION PROCESS

This is not a piece of my own writing but a chapter by New Zealand academic Bob Gregory.  It was one of the first critiques  of  "New Public Management" that really pressed home the  theoretical impossibility as well as the practical impossibility of some of the "accountability" assumptions that underpin NSM.  Based on a framework by public sector management academic James Wilson, Bob postulates that "accountability" can only be guaranteed under certain conditions; conditions that are rarely present in much public sector activity.  The public sector essentially depends on practitioners having a sense of responsibility for their actions rather than accountability to someone for their actions.  Furthermore he claims - and cites case studies to show - that attempts to force public sector activity from "responsibility for" into "accountability to" frameworks actually corrupts the public sector process.  This is an extremely important but oddly overlooked contribution to the NPM and performance management debate.  Bob gave me permission to copy and circulate this chapter.

GENUINE LEADERSHIP WITHIN CONTEXT

A few years ago a large package appeared unannounced in my mailbox.  It was the complete draft manuscript of a book being developed by Robert (Bob) Terry on how do decide what kind of leadership fits which kinds of situations.  I'd met Bob at a conference in Texas for about five minutes and emailed him once a few months later for some clarification.   His response to my request was to send me the entire draft manuscript of a book.  Now there is someone who understands what leadership is all about.  Here's good summary of his ideas {and my thanks to Gene Shackman for hunting this out}.  The book that eventually emerged shortly before his untimely death is called ""Seven Zones for Leadership: Acting Authentically in Stability and Chaos" (2001). He was not a single model snake oil man flogging off his particular take on leadership.  He believed passionately that nearly all models of leadership are relevant somewhere.  The trick is to pick where and most importantly when.  In the manuscript and in the book he examines the major leadership models and shows how each fits a particular situation (zone) and then offers his own model, "authentic leadership," as the integrating approach.  [My thanks to Michael Patton for telling me about the final publication.

Return to the Resources page