Experienced Nurse Rotation Scheme
Phase one evaluation

Section Two
Methodology
Table of Contents
Previous
Next

Section Two — Methodology

2.1 Introduction

As discussed in 1.1, this study is one segment of an Action Research study. As is the case with many other research approaches, Action Research can be carried out from different philosophical perspectives. Since the focus of a study will vary accordingly the various perspectives will be briefly discussed before an explanation of Action Research and the perspective taken are explained.

Following that, the approach to this segment of the research programme will be discussed together with the research methods of data collection and analysis that were used.

2.2 Research Perspectives

Through our actions and through our culture we all develop interrelated sets of beliefs and practices about the world and the things that we and others do in it. Hanson (1958) argued that this is invariably the case and is part of our nature as intelligent actors. Both Habermas (1971) and Lather (1991) go further and argue that in research the perspectives we use inevitably reflect our beliefs about the world we live in and the world we want to live in.

2.2.1 Positivist Perspectives

The perspective that is most commonly used in social and health related research and which is committed to by several schools of philosophy, is the scientific or positivist one which posits a single objective reality in which causal links between variables are identified and behaviour can be predicted and controlled. There is also the claim that all phenomena, including human mental states, can be reduced to more basic, unobservable causes (i.e. using reductionist models).

Lincoln and Guba (1985) and Bryman (1988) claim that even among those who are most knowledgeable about positivism there are a wide range of definitions. Lucock (1996) acknowledges this but provides a number of claims that, she suggests, provide an approximation to its characteristics.

So for the positivists ‘the truth is out there’ although latterly neopositivists admit than only approximations to the truth can be achieved. And there are certain rules to which positivists and neopositivists conform to ensure that the truth is achieved or almost achieved. They are

There are a number of perspectives other than positivism or neo-positivism (Lucock 1996), and as is the case with the positivist perspectives, they reflect philosophical beliefs underlying research as to what sort of things there are in the world and how we might go about finding out about them.

Very broadly, what unites all of these perspectives is a refutation of the claims and methods of positivism. They reject the notion of one social reality and point to the importance of culture, context and history. They reject the claim that for research to be credible it must follow the research designs laid down over the last century by positivists and have developed new research designs that are interactive, contextualised and frequently involve all participants in the exploration of research issues (Lather 1991).

They reject the notion of value free research and claim that researchers can never stand at a neutral distance from what is being examined and thereby not affect it.

They have found other ways than those of positivism to ensure the trustworthiness of their work (for this study see 2.5.1) and they reject the notion of generalisability claiming that, because of possible differences in culture, context and history a study can only provide pointers for others to consider for their own use.

These perspectives will be briefly examined and later used to explain the choice that was made for Action Research and for the evaluation of the Experienced Nurse scheme.

2.2.2. Constructivist and Social Constructionist Perspectives

Although they have their origins in earlier work, both the constructivist and the social constructionist approaches have developed in the recent past and are still developing. There are a number of variations developed from the basic notion that human beings construct the meaning of their world. The differences between these variations reflect either different disciplines or slightly different assumptions about what it is to be human.

For example, Guba & Lincoln (1989) take a constructivist perspective which suggests that they believe that the human world consists of individuals who construct meanings for themselves. A somewhat different perspective is that of social constructionism which argues that meaningful reality is constructed out of the social context and social relationships rather than by individuals. Both approaches tend to deal with conscious deliberation about meaning.

2.2.3. Interpretivist perspectives

Phenomenology is usually seen as descriptive rather than interpretive but the stance being taken here, and which is made clear in the following discussion of hermeneutics, is that we, researchers and non-researchers alike always interpret.

An important aspect of phenomenology lies in its belief that the basis for knowledge is immediate subjective experience. There are two forms of phenomenology, Husserlian, which advocates minimising the constraints of one’s preconceptions in order to allow the experiences of others to be communicated and hermeneutical which argues that such minimisation is not possible; instead one should acknowledge one’s preconceptions as discussed below.

Interpretive Hermeneutics is more concerned with the taken for granted knowledge that people acquire throughout the course of their lives. It is knowledge gained from social interaction but at an accepting rather than at a deliberating or challenging level.

Hermeneutics is concerned with the study of texts but there are different versions which have changed over time. Here Gadamer is being followed. In his book ‘Truth and Method’ (1989) he claimed that hermeneutics is the study of the universal phenomenon of human understanding. His argument is that hermeneutics is the fundamental form of human understanding and as such it is not confined to texts but is applied also to conversation. This implies that people in their everyday life and researchers in their work use the same way of understanding although the latter are expected to pay more conscious attention to this in their analysis of their data.

Hermeneutic understanding, Gadamer argues, is achieved through interpretation. Interpretation will differ from person to person depending largely on an individual’s culture and experience. However, important for researchers is the argument that although we all may have what he calls ‘…our present horizon of understanding’ (1989 p 307) that horizon can be enlarged to enable us to encompass the understandings of others.

Hermeneutics is strongly criticised by those of a positivist persuasion. They argue that we cannot get into other people’s head and see their meaning so we must depend on observing their behaviour, including the behaviour of answering written tests.

In response to these arguments Gadamer said,

“When we understand a text we do not put ourselves in the place of the other, and it is not a matter of penetrating the spiritual activities of the author… The meaning of hermeneutical inquiry is to disclose the miracle of understanding texts or utterances and not the mysterious communication of souls. Understanding is a participation in the common aim.”

In hermeneutics there is the concept of the hermeneutic circle. It is argued that no knowledge is possible without presuppositions and that to gain a full understanding of a text or conversation hermeneutic researchers must first gain an holistic understanding of it and then use this understanding as a basis for understanding the parts of the text or conversation. The researchers gain knowledge from proceeding from the whole to the parts and then back again and where there are discrepancies between part and whole this leads to a reconceptualisation of the meaning of the text or conversation. It is in this way that they can come to understand their own presuppositions and grasp those of the author of the text.

Each of these perspectives, together with those of constructivism and social constructionism takes the same view of how to find out about the social world. The desire is to go beyond discovering facts and to discover meanings. To do that, they argue, the researcher must ask people about the meanings their world and their experiences have for them rather than to try to measure them or collect ‘facts’ about them.

2.2.4 Critical Theory and Critical Feminist Perspectives

Critical theory and feminist perspectives are similar to the perspectives discussed in 2.2.2 and 2.2.3 but they go further and rather than simply interpreting the meanings that people give to their situations and experiences, they look to both surface them and to help people to liberate themselves from the ideology those meanings reflect.

Power and influence are seen as being unequally distributed and the aim is to uncover the distortions and constraints that impede equal and uncoerced participation in society. By doing so, they aim to facilitate change that will liberate ordinary people, and in the case of critical feminist research, women in particular, from the political, economic and social conditions that constrain them.

To argue that there are such distortions and constraints is a value judgment, and to want to facilitate change implies other values and norms which suggest some ideal form of being. In other words these perspectives specifically introduce values and norms into the research process. We have seen above (2.2.1) that research cannot be value free, but from the critical theory perspective, you do not simply acknowledge this; you go further and let your values guide the research process.

This is not to say that the perspectives discussed in 2.2.2 and 2.2.3 are wrong but rather to suggest that they do not go far enough and the critical theory perspective extends it by taking a different attitude towards meanings. By arguing that values and norms have social functions (they maintain particular sets of social relations), critical theory perspectives go beyond understanding to criticize meanings and attempt to transform them.

Habermas (1971), a leading proponent of critical theory, argues that what separates the natural and the social or human sciences is that the natural sciences are concerned with explanation, while the social or human sciences are concerned with understanding. Habermas argues that this simple separation ignores the ways in which relations of power and dominance shape and, perhaps, distort consciousness. He suggests that it is more useful to distinguish three basic forms of scientific interest in knowing about the world: namely ‘technical’, ‘practical’ and ‘emancipatory’ interests. These interests appear again in 2.3 in the discussion of Action Research.

Technical interests are a form of means/ends rationality. Research here is concerned with how most efficiently and effectively to achieve some particular state of affairs. Such aims are reflected in the empirical-analytic (positivist) perspective, in which we seek to know in order to control social and natural realities.

Practical interests are reflected in what Habermas calls the ‘hermeneutic-historical perspective’, in which the concern is to interpret and understand our social realities.

Emancipatory interests are needed to consider the way in which the systems of social labour and of power have the potential to distort consciousness.

2.3 The Action Research Approach

Action Research is discussed here in order to contextualise the evaluation study being discussed in this report. The purpose of Action Research will be discussed as will three modes of Action Research which are underpinned by different philosophical perspectives. The methods of data collection and analysis will not be discussed except to say that these can be very varied and the methodology discussed in 2.4 below is one example.

2.3.1 The purpose of Action Research

Kurt Lewin, a German social psychologist of the Gestalt school, who worked in America in the late 1930s and until his death in 1947 is generally accepted to be the person who introduced and named Action Research studies. Lewin and his colleagues were very concerned with social issues, including education, minority problems, racism and fascism and their concern was to find ways of inducing more democratic behaviour.

As a gestalt psychologist, Lewin was interested in learning and one aspect of his work in this field was his finding that when we learn in a group and, as members of that group, commit ourselves to a change in behaviour, the behavioural change which occurs lasts longer than when we learn individually and commit ourselves to change (Lewin 1947 a). These findings led Lewin to argue that changes to more democratic behaviour should be attempted in groups.

However, Lewin was not only concerned with changing behaviour. As a social scientist he wanted to be able to demonstrate that changes had occurred and, perhaps more importantly, he also wanted to understand the processes that brought about that change. He argued for Action Research as a form of social experiment, but a field experiment under controlled conditions rather than a laboratory experiment. (Lewin 1946) (Lewin 1947b)

Three aspects of the experiment would be studied, the nature of the relevant situation at the start of the experiment, some particular happening or event, such as a series of workshops, designed to bring about a specific change, and the end situation to see the effect of the happening or event on the beginning situation. He argued that a careful analysis of the situation both before and after the change event would allow for an accurate definition of the change created. Furthermore, a careful and accurate description of the change event would enable the researcher to find out as much as possible about the characteristics of the factors which brought about change.

Lewin argued that the whole of social life, in which he presumably included working life, is made up of circular processes in which items or events follow each other in a circular way. Many of the items or events are interdependent in that as one finishes the next begins. He suggested that some of these processes act in the same way as engineering ‘feedback’ systems, which show some kind of self-regulation. It was Lewin’s plan to use both the notion of circular processes and the aspect of ‘feedback’ in the social system in a conscious and deliberate way during an Action Research experiment and thus create a spiral of circular activities in which the feedback from one step would act to correct or change the next step. He saw the steps as developing in the following way.

Planned social research, Lewin (1947b) argued, begins with a rather vague idea, no more than a dream or a wish, which requires a plan for it to become real. However, the development of what he called a ‘general plan’ to precede the planned change, involves the clarification of the objective in the minds of the researchers, a decision by them as to the way in which the plan should be put into action, the means available to achieve the plan, and the development of a strategy of action. Implicit here is the notion that the situation involved is susceptible to improvement.

Lewin (1946, 1947b) further argued that to facilitate the development of the general plan, ‘fact-finding’ is essential. Without fact-finding it would be impossible to confidently structure the planned change, the change’s relationship to the total setting, and the means of bringing about the change. He argued that, very often, in the light of the findings from the fact-finding stage of Action Research there is some alteration to the planned change. However, the fact-finding stage does not stand alone and is not a one-off event. To be realistic, it has to be part of the feedback system which will thus link fact-finding to action throughout the Action Research project.

Furthermore, the organisation or group within which the change is to take place must be involved in fact-finding. For the change to be effective, those responsible for putting the changes into practice, be it an organisation or group, have to be part of the feedback system which links fact-finding to action

Lewin argued that the general plan for action must be flexible. Furthermore, although the general plan can be seen as a blue print for action that does not mean that all further steps in the planned change are fixed at this stage. Only the first step of the change is fixed at this point in the Action Research cycle.

Each step in the process of change must be carefully observed and once it has taken place the situation must be evaluated and reflected upon, both to determine whether the general plan needs to be changed and to provide a basis for the next step. Each of the above activities continue throughout the life of the Action Research process. Consequently, and in line with the notion of feedback, each step in the process of change becomes the first step for the rest of the project.

Although there have been many changes in various other aspects of Action Research, the requirements put forward by Lewin that it should be focused on social practices which are susceptible to improvement, that those responsible for practice should be involved, and that the process of Action Research proceeds through a spiral of planning, acting, observing and reflecting, are still at the heart of Action Research. What has changed over time is the extent to which practitioners are involved in, or are responsible for, the whole research process, the way in which the philosophy which underpins any given Action Research approach may differ from that underpinning other Action Research projects, and, consequently, the values which inform the processes of decision making and data collection and analysis. The following quotations demonstrate the extent to which Action Research, almost sixty years on, adheres to the form first proposed by Lewin.

McCutcheon & Jung (1990), having examined the work of a number of writers to find the salient points of Action Research argue that:

Action Research is characterised as systematic enquiry that is collective, collaborative, self-reflective, critical and undertaken by the participants of the enquiry. The goals of such research are the understanding of practice and the articulation of a rationale or philosophy of practice in order to improve practice.

Unfortunately, this definition, although substantially correct, fails to fully capture the ‘action’ part of Action Research. Grundy (1982) does so. She tells us that the conditions which are ‘individually necessary and jointly sufficient for Action Research to exist are:

  1. the project takes as its subject-matter a social practice, regarding it as a strategic action susceptible to improvement.
  2. the project proceeds through a spiral of cycles of planning, acting, observing and reflecting, with each of these activities being systematically and self-critically implemented and interrelated; and
  3. the project involves those responsible for the practice in each of the moments of the activity, widening participation in the project gradually to include others affected by the practice and maintaining collaborative control of the process.

Rapoport (1970) provides further insight to Action Research when he says:

Action Research aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework.

2.3.2 Modes of Action Research

Modes of Action Research have been discussed by a number of writers but reference here will be mainly to Grundy’s 1982 paper because it is considered to be the most influential. Reference will also be made to a later paper by Holter and Schwartz-Barcott’s (1993) The names that Grundy gave to each mode will be used here.

There are three modes of Action Research which Grundy, very much in line with Habermas (1971) (see 2.2.4), defines as technical, practical and emancipatory. It is important to note that the mode used in any one particular study can change over time depending on the circumstances.

Discussion papers on Action Research sometimes imply that research has progressed through the various modes and that Technical Action Research is a thing of the past. The notion of progression is certainly correct as those who work from non-positivist perspectives have come to use Action Research as a preferred approach. However, this has not resulted in the previous mode or modes being dropped. Rather, Action Researchers work from within the mode which best suits their philosophical stance.

Technical Action Research

In this mode of Action Research the judgement made by the researcher(s) results in ‘making’ action. It might be undertaken by one or more researchers who enter a collaborative Action Research project with an identified problem to solve and a specific intervention in mind to solve that problem and achieve more effective and efficient practice. In other words, the researcher wants to know if a particular intervention can be usefully applied to in a practical setting. At this point, his or her task is to gain the interest of the members of the organisation concerned, and persuade them to help with the implementation of the planned intervention.

Alternatively, as Grundy suggests the project might be initiated by a particular person, or persons, who, because of their greater experience or qualification, would be seen as ‘experts’. For example, a senior person, or persons, on a ward or unit, might decide to adopt a different approach to nursing on that ward or unit and, having decided that it would be helpful to use an Action Research approach, persuade other members of staff to go along with them. In this mode, the initiative for change would be unlikely to be initiated by someone lower down the hierarchy — they would lack the authority.

The essence of this mode is that the ‘idea’ by which the outcome of the change will be measured, pre-exists in the mind of the person initiating the research. The purpose is to implement a preconceived plan rather than to incorporate, into the Action Research process, the identification of the most suitable form for change.

At this point, according to Grundy, some of the practitioners involved may reject the ‘idea’ and refuse to co-operate. If most of the group refuse then the research is unlikely to go ahead although the change could still be imposed on the group. Another scenario is that the group agree to work towards the goal but without any real commitment to it. Providing the group did work towards the goal, a lack of commitment on their part would not be a problem since the success of the project is judged by the outcome, that is, the actual change in behaviour that has taken place. Providing that the practitioners implemented the change, whether or not they liked working in this way would be irrelevant, at least in the short term, to the perceived success of the project.

The third possibility would be that the practitioners adopt the ‘idea’ as their own, in which case the Action Research would move into another mode of Action Research.

Holter and Schwartz-Barcott suggest that the Technical approach can lead to an efficient and immediate change in practice with the change being, ‘…like an injection that gives an immediate effect.’ However, they go on to argue that the long term effectiveness of the intervention is likely to be limited because previous structures and practices begin to reappear and, in consequence, the enthusiasm of the members of the organisation decreases.

Grundy argues that change can be significant but provides another explanation that seems to provide further insights into why Technical Action Research may so easily fail. She argues that, since it is not essential that the practitioners involved should be committed to the ‘idea’, there is the possibility of them playing the ‘Action Research game’ where they act and deliberate in an authentic way whilst the project is underway but once the ‘game’ is over they are no longer obliged to play by its rules. This possibility gives weight to Grundy’s claim that in Technical Action Research, the participants are regarded as the instruments of change rather than the agents of change.

Technical Action Research is the original mode of Action Research. Associated with positivism and neopositivism it was in use before the other research perspectives were well developed.

This form of Action Research was not chosen for the larger study because there is no ‘idea’ in a researcher’s mind that he or she wishes to test out. Furthermore, the stakeholders in this study do not regard the participants as instruments of change but as agents of change.

Practical Action Research

Holter and Schwartz-Barcott label their second mode as the Mutual Collaborative Approach. They describe it as one in which researcher(s) and practitioners come together to identify potential problems, together with their underlying causes, and possible interventions that can be carried out through Action Research.

By doing so, researchers and practitioners, together, arrive at a new and deeper understanding of the nature of the problem and its causes. This diagnosis leads to theory which is specific to the situation under study and which, therefore, is usually known as ‘local theory’. So, in this mode practical judgement results not in ‘making’ action but in ‘doing’ action (praxis)

Because of this new understanding, changes that are made in the Action Research project are usually longer lasting since practitioners are not simply caught up in enthusiasm generated by the change itself.

However, Holter and Schwartz-Barcott argue that, because the change may be tied to a group of practitioners rather than the wider organisation, the intervention may be short lived if those practitioners move on, or if a considerable number of new people enter the system.

Grundy’s second mode, Practical Action Research, involves, in common with Holter and Schwartz-Barcott’s Mutual Collaborative Approach, an initial examination of the problem rather than the implementation of an already chosen solution. And, no doubt, this initial examination would lead to the development of ‘local theory’. Furthermore, with their insistence on the shared meanings of the participants it can be assumed that this mode is underpinned by an interpretive and/or social constructionist perspective.

What separates Practical Action Research and the Mutual Collaborative Approach of Holter and Schwartz-Barcott is Grundy’s argument that Practical Action Research is informed by values and by an accumulated store of wisdom. There is more here than the identification of possible solutions to problems, once they, and their underlying causes, have been identified. The solutions are not technical ones, rather they reflect the shared values of all those involved in the project. It is not a case of choosing from any possible solutions that might solve the problem. It is the choice of a solution which, given the experience and wisdom gathered over time by the practitioners, ‘feels’ right and ‘good’ in the circumstances. What is usually meant here is ‘good’ for the clients of the practitioners.

Grundy suggests that, because of the part played by values and accumulated wisdom, the group would be well advised to share the process of change with a facilitator who would help them in the process of self reflection and reasoning. Furthermore, when carried out in this way, Grundy believes that change can frequently be successful and lasting.

This form of Action Research was not chosen for the larger study partly because practitioners in this situation do move on and a considerable number of new people do enter the system and partly because of structural concerns which are discussed under ‘Emancipatory Action Research’.

Emancipatory Action Research

Holter and Schwartz-Barcott refer to their third mode of Action Research as the Enhancement Approach. They claim that, as well as developing local theory to explain and resolve problems identified by researchers and practitioners together, the researcher(s) raise questions about the underlying assumptions and values which inform practice in this setting. By doing so, the researcher(s) assist the practitioners to reflect critically upon their practice by raising these assumptions and values to their collective consciousness. In this way, the practitioners are now able to identify the differences between what they believe to be the values underlying their practice and the unwritten rules, norms and conflicts that are really governing their practice. In consequence, the local theory developed through discussion more clearly reflects what is happening in the situation.

The focus here is upon personal, cultural and organisational norms and raising these to the collective consciousness of the group allows any negative, unconscious forces to be dissipated. In consequence, meaningful change can be achieved and sustained. According to Holter and Schwartz-Barcott, the Enhancement Approach tends to lead to lasting change.

Grundy’s final mode — emancipatory Action Research — differs substantively from Holter and Schwartz-Barcott’s Enhancement Approach. It does have much in common with that approach — for example, both make it clear that there is an emphasis on the surfacing of the underlying value system, including norms and conflicts which may be at the core of the identified problems.

However, whilst the explanation could lie in their choice of language, Holter and Schwartz-Barcott’s Enhancement Approach does seem to be a rather watered down version of what Grundy and many others call Emancipatory Action Research.

This is an approach that is used when organisational change is not amenable to improvement through the operation of group reflection and action. Such a situation occurs where institutional restrictions impinge upon professional practice in such a way that, no matter how determined are the group to bring about change, they are powerless to do so because of the strength of the system within which they work. That system may be administrative, managerial or even theoretical.

When this occurs action must be outwardly directed, towards the system as well as focused upon individual and group practice. But to do this there is a need to be critical and the reflection stage of Emancipatory Action Research is characterised as taking a Critical Theory perspective — most commonly that of Habermas.

This form of Action Research was chosen for the study because it resonates with the concerns for change that were discussed in the introduction to this report. It also focuses on structures as well as the values of individuals and groups and is the most likely mode of Action Research to be effective where structures impede change and where stakeholders are likely to ‘move on’.

Where there is likely to be movement of stakeholders over the lifetime of a project it becomes even more important that structures are challenged and, where necessary, changed in such a way that those changes will survive the departure and renewal of stakeholders.

2.4. Evaluation, the research approach for the Experienced Nurse Rotation Scheme

The term evaluation is a generic term for a number of different approaches to evaluation. Guba & Lincoln (1989) refer to what they call four generations of evaluation. The focus of their work is the American education system but it is equally applicable to education systems in general and to any age of students. They are four generations or approaches, as they are referred to here, in that they developed one after the other but they are all still in use. Consequently, it is important to briefly examine each of them and to explain why one particular approach was chosen here.

The first generation is measurement of what the students have mastered during the course of their studies. Today, in a traditional first or postgraduate degree this would be achieved through the submission of course work and by written examination.

The second generation discussed by Guba and Lincoln (1989) came into being when it was realised that the targeting of students was not sufficient to ensure quality of learning. What was also needed was to examine whether or not what they learned was what those who designed the programme of learning had intended them to learn. Thus we have programme evaluation that is achieved by defining learning outcomes and examining the students against them. Guba and Lincoln (1989) call this descriptive evaluation because what characterises it is a ‘description of patterns of strengths and weaknesses with respect to certain stated objectives.’ (p 28).

Objectives can be of two kinds, outcome objectives and process objectives. The objectives in Work-Based Learning can be only partially decided on in advance. Learning outcomes and objectives are identified in the three-way learning agreement negotiated between the learner, the organisation and the University during the Programme Planning Module (see Appendix 2). The setting up and running of a project is a process objective. The learning achieved and the success of that project can only be conjectured initially but will appear retrospectively. A Work-Based Learning Programme is, essentially, focused upon process rather than outcome objectives. There will be outcomes but they cannot be only partially decided upon in advance.

As Susman and Evered (1978) argue it is the processes rather than the outcomes which are the focus of Action Research, and how such processes lead to greater knowledge about change in organisations. The outcomes resulting cannot therefore be predicted, as they are unique to the individual setting:

“The Action Researcher also recognizes that the objectives, the problem, and the method of the research must be generated from the process itself, and that the consequences of selected actions cannot be fully known ahead of time.” (Susman & Evered 1978 p 590)

Gradually, it was realised that a purely descriptive approach was also insufficient. There is no great value in students performing well in relation to certain objectives if those objectives are unsuitable. Thus the third generation of evaluation became judgemental. The objectives were open to scrutiny by those who were deemed expert in the field. Today, in Britain, quality control demands on universities require the implementation of periodic programme evaluation in the form of a summative evaluation of student learning outcomes against specific objectives decided on in advance by the institution concerned in collaboration with those who are experts in the field. For the reasons discussed in relation to second generation evaluation, this approach to evaluation is also unsatisfactory for this study. However, as is the case with earlier generations this does not mean that evaluation should not take place. As Boud et al (2001) argue, the educational institution involved must assess learning outcomes ‘…with respect to a framework of standards and levels.’ Such a framework is somewhat different to the objectives discussed in the three approaches mentioned above but is still open to external scrutiny.

Guba & Lincoln (1989) suggest a fourth generation of evaluation which, unlike the other three, does not establish its parameters and boundaries in advance of carrying out the research but rather decides on the information to be collected through listening to the claims, concerns and issues identified by the stakeholders, those who might in some way be put at risk by the research.

It is this model of evaluation that has been used here. And in relation to both the larger Action Research study and to this evaluation of the Experienced Nurse scheme, the people who are at risk are many. In this segment of the Action Research study the practitioners themselves are at risk because their futures are dependent on the success of the programme; the tutors who are operating the Experienced Nurse programme are at risk because their good name depends on an open and honest evaluation; and, on behalf of the Trusts, the senior advisor for research who led the scheme is at risk.

When we turn to the larger, Action Research study those at risk are all the stakeholders involved. Clearly the people mentioned above are involved but so are the two Trusts involved in the various schemes that are part of the Action Research study. Their reputation is at risk in relation to future employees and funders as is the reputation of the university in relation to future students and purchasers of education. Also at risk are the Education Consortia (now the Workforce and Education Confederation) who funded the various schemes and the researchers whose credibility will be at risk if their methodology fails to identify, as the schemes progress, the extent and quality of the changes being made.

The perspective taken in the evaluation of the Experienced Nurse scheme is that whilst some knowledge is socially constructed it almost always built upon a considerable basis of taken for granted knowledge. Since it is part of the larger, Action Research, study it does involve a critical perspective but only to the extent that the methods of data collection used aided the practitioners in their reflections and in that, with the permission of the practitioners, any problems discussed at interviews were fed back to those running the scheme in the Trusts so that changes could be made. This is in keeping with an Action Research approach.

Guba and Lincoln (1989) do not arbitrarily suggest this fourth approach. They see each generation of evaluation as being a progression first from no evaluation through evaluation of student to evaluation of curricula and then to evaluation of the worth of that curricula but, from the point of view of researchers carrying out an evaluation, they criticise all three generations on other grounds which, collectively, lead to the fourth.

Their first criticism relates to a tendency to what they call ‘managerialism’. Put briefly their concern is that those who are commissioning the research tend to have too great a say in what questions will be asked. Guba & Lincoln (1989) feel strongly that, whilst they should have some say, the important people are those stakeholders who are actively involved in the learning process (see 2.5 below). If critical Action Research is carried out correctly this is not an issue.

A second criticism is that although the existence of values is implicit in the very term ‘evaluation’ this is not acknowledged in most evaluation research because it is assumed that good research is value free. Guba and Lincoln (1989) and many other researchers including ourselves, challenge this, and point out that if it isn’t value free then the ‘facts’ discovered by researchers are influenced by the value system they bring to the evaluation and ‘then every act of evaluation becomes a political act’ (p35). Again, this is not an issue if critical Action Research is carried out correctly.

They also argue that all three generations are overly committed to the positivist paradigm (in this research the equivalent term ‘perspective’ is used) to guide its methodological work. The perspective Guba and Lincoln take was discussed above in 2.2.2

In summary, in the Experienced Nurse Rotation Scheme, what Guba & Lincoln (1989) call fourth generation analysis has been used with participants being interviewed. The perspective underlying the study was that of interpretive hermeneutics. Each of these influenced the choice of data collection and/or data analysis methods.

The Experienced Nurse Study is one spiral of a pre-existing Action Research project i.e. the ‘D/E Rotation Scheme’. In each spiral, the Action Research partnership includes evaluation undertaken by both an ‘insider’ member (Rotation Scheme Coordinator/principal investigator) of the NHS community and ‘outsider’ research experts (Titchen and Binnie 1999). Since there are multiple approaches to evaluation, there has been and will be different methods used e.g. gathering opinions through questionnaires and focus groups with the D/E Rotation Scheme members.

2.5 Data Collection and Analysis for the Experienced Nurse Rotation Scheme

Introduction

As part of the context of the Experienced Nurse Rotation Scheme, a brief comment is appropriate about the concept of time in relation to the development of the project. One way of looking at time is from the setting up of the project prior to people going to the university in May 2001 — a preamble of a year or more between getting the idea together, writing the paper, getting the money from the consortium, putting out the first advertisement, ironing out the difficulties in order to operationally develop it in the two Trusts and then getting people actually signed up with the university and attending seminars. Another way is to consider only the period from May 1st 2001 until the data collection was completed plus the period after that. Both periods are covered in this report and it is left to the reader to decide which concept of time is more useful.

2.5.1 Trustworthiness

As discussed in 2.2.1 researchers working from other perspectives reject the positivist methods of determining validity and reliability. In this study various ways were used to establish it’s trustworthiness including meeting participants in advance of interviews to help to establish rapport; involving them in identifying the topic areas to be discussed; providing participants with copies of their scripts; and inviting them to read and critique the report prior to publication.

However, the main way of ensuring the trustworthiness of this study is that of providing readers with a decision trail. A reflective diary was kept to note the experiences of data collection and the decisions made at each part of the study. These have been included here in the form of ‘thick description’ in order to turn the decision trail into an audit trail for the reader. This accounts for the care taken over explaining the overall methodology used and the methods used for data collection and analysis. The reader must decide for him/herself if the study is Trustworthy rather than depending on ‘methods’.

2.5.2 Participants

The participants in this study included the representatives of key stakeholder groups. There were the ten Experienced Nurses from two NHS Trusts in NW London, two senior lecturers from the educational institution involved, and the Senior Nurse Advisor for research for the two Trusts.

The Experienced Nurses were the practitioners undertaking the programme described in 1.1. As well as holding the role of students, these practitioners hold a number of roles within the community namely NHS service users; tax payers and funders of the NHS; employees of the NHS; and holders of leadership roles that contribute to the modernisation and leadership of the NHS. As experienced professionals they also hold the role of advocates and lobbyists for the individual users of the NHS whilst at the same time addressing the public health needs of a civilised society.

Positivism favours the random selection of subjects in order to also randomise any unknown intervening variables. Purposive selection, on the other hand, looks for people who have experience of the topic under consideration and it tends to look upon those taking part in the research as participants rather than subjects. In this research all members of the cohort were willing to take part and, since there were only ten practitioners, all were included. As mentioned in 1.1, two tutors and the Senior Nurse Advisor for research for the Trusts were also participants although their contributions have been used mainly to check if they shared similar expectations for the programme to those of the practitioners.

2.5.3 Gaining Access and ethical issues

The practitioners were approached as a group when they were attending a programme seminar. Some time was spent discussing the purpose of the research and eliciting their views on topic areas to be considered. It was explained that each interview would be taped but that only the researcher and the participant would have access to the tape and to the subsequent transcript. Confidentiality was also assured, as was anonymity to the extent that this is possible with such a small group whose members are well known in the Trusts. It was pointed out that they could refuse to participate without any possibility of it affecting the outcome of their studies or affecting their position at work and that they could choose to remove themselves from the research at any time and have their data destroyed. Telephone numbers and e-mail addresses were exchanged to facilitate the planning of interview times and places. The tutors and the Senior Nurse Advisor for research who led the scheme for the Trusts concerned were given similar reassurances.

2.5.4 Topics

The topics finally decided upon for the practitioners were as follows:

One other topic area was introduced later. This was:

The tutors and the senior nurse research advisor were asked to talk about the aims, form and process of the scheme as they saw it.

2.5.5 Data Collection — unstructured interviews

As was discussed in 2.5.3, the topic areas to be discussed in the interviews were defined in advance but in keeping with an interpretive hermeneutical approach the interviews were unstructured and participant led, with the researcher intervening only to ask questions which led on from participant’s comments.

The practitioners chose the place to be interviewed and all initially chose their place of work and agreed to find a room where we would be neither disturbed nor overheard. One interview eventually took place in the Trust’s headquarters because the location suited the activities of the participant on the day of the interview. The tutors chose to be interviewed in offices at their place of work and the Senior Nurse Advisor for research chose to be interviewed in the interviewer’s home. Since these were all busy people, fitting in the interviews was quite difficult, indeed, on four occasions planned interviews had to be rescheduled. Consequently the interviews were spread over a number of weeks and the practitioner’s were at different stages of their RAL module (1.4.1).

Whilst the recording equipment was being set up and tested, the participants completed informed consent forms and were reminded about the purpose of the research and about the ethical issues discussed under ‘gaining access’. A written reminder of the topic areas was then placed in front of them. They were assured that the order for discussion was not important and in the event they mostly moved back and forth amongst the topics with the interviewer asking probing questions where relevant. The practitioner interviews lasted from just under one hour in two cases to over one and a half hours in two others. Most practitioner interviews lasted just over one hour and all the interviews came to an end when the participants felt that they had fully covered the topics.

The interviews with the tutors and the Senior Nurse Advisor for research were somewhat briefer (but see 4.1) because the topic area was far more limited.

2.5.6 Data Analysis

A transcription was made of each interview and the tapes and transcriptions were kept under lock and key. No other person but the interviewer had access either to these or to the computer files holding the data. Participants were sent copies of their transcripts.

It was at the point of transcription that pseudonyms were given to the students to further their anonymity and confidentiality. For the same purpose, some of the women were given men’s names and some of the men were given women’s names.

Because the topic areas were quite specific the responses of the practitioners could be relatively easily grouped under those headings. This was done and then the responses in each topic area were analysed to identify commonalities and differences in views amongst the group. These were listed and comments of practitioners were chosen to represent these views.

The responses of the tutors and of the Senior Nurse Advisor for research were used to check for agreements or differences between them and the students.