Mechanism design for total quality management: Using the bootstrap ...

1 . The effect of EFQM self-assessments on action research performance: Lessons on how to improve the efficiency and effectiveness of research on New Public Management Petter Øgland, Department of Informatics, University of Oslo NEON, 25th-27th November 2014 

2 . Plan for presentation • Motivation (3 minutes) – Problem: Action research is important but risky design for studying NPM – Possible solution: Use TQM for controlling and improving action research • EFQM model and double-loop learning (7 minutes) – Hypothesis 1: EFQM improves EFFICIENCY of action research – Hypothesis 2: EFQM improves EFFECTIVENESS of action research – Hypothesis 3: EFQM improves LEARNING about improving action research • Testing the hypotheses in a NTAX action research study (7 minutes) – The NTAX action research is slow and costly – EFQM succeeds in diagnosing and suggesting treatments – Discussion: Controlling and improving AR within AR traditions based on interpretivism, post-positivism and critical theory • Conclusions (3 minutes) – Critical theory allows EFQM to align with action research – Action researchers should use NPM logic for improving AR performance 

3 . New Public Management (NPM): Managerial logic vs. practitioner logic How can practitioner logic be understood in the context of an economic/managerial need to improve cost-efficiency? 

4 . Action research is a way of building knowledge by designing and evaluating social interventions (Clark, 1972) But it is a research design with high risk for failure (Simonsen, 2007) 

5 . What if we applied the logic of NPM on action research? 1. Define the production 2. Improve the production process: process: Producing change and Reducing costs and cycle time in AR scientific knowledge process by use of TQM methods 

6 .Using the European Foundation for Quality Management (EFQM) model 

7 .Theoretical framework (Argyris & Schön, 1978) 1. EFFICIENCY of action research (strategy) 2. EFFECTIVENESS of action research (goals) 3. LEARNING HOW TO DESIGN Single-loop and double-loop learning 

8 . Use TQM-model (EFQM) to evaluate consequence of action strategy Hypothesis 1: EFQM improves EFFICIENCY of action research (strategy) 

9 . Use TQM-model (EFQM) to reflect on governing variables Hypothesis 2: EFQM improves EFFECTIVENESS of action research (goals) 

10 . Use TQM-model (EFQM) to improve AR performance Hypothesis 3: EFQM improves LEARNING HOW TO DESIGN Single-loop and double-loop learning 

11 .Case: Trying to improve action research at NTAX by use of EFQM Diagnosis: NPM strategy of outsourcing IT services, managing through contracts, and controlling through ISO 9001 audits. No audit process. Treatment plan: Design audit process through use of action research approach. NTAX wants AR to be externally Operational hypothesis: funded. Applying EFQM self- assessments will result in Execution and evaluation: improved efficiency, Eleven months of trying to get effectiveness and learning on funded, but no success so far. how to do Action Research. Learning: NTAX research has to focus on 

12 . Using the EFQM model to diagnose the action research project Diagnosis AR: AR strategy of proposing to design audit process through action research fails because NTAX will not allow AR without external funding. 

13 .Using the EFQM model to plan treatments for action research project Treatment AR: EFQM model identifies improving customer satisfaction (NTAX) and improving partnership management (UiO) as the two most important issues. 

14 .Using the EFQM model to plan treatments for action research project Treatment AR: EFQM model identifies improving customer satisfaction (NTAX) and improving partnership management (UiO) as the two most important issues. Sub-criteria 4a, 4b, 4c, 4d, 4e 

15 . Using the EFQM model to plan treatments for action research project Treatment AR: EFQM model identifies improving customer satisfaction (NTAX) and improving partnership management (UiO) as the two most important issues. Sub-criteria 1c, 2a, 2c, 3b, 3c, 4a, 5b, 5c, 5d, 5e 

16 . Using the EFQM model to evaluate the action research project Evaluation AR: Partnership with UiO improved by collaborating on grant application, but no measurable effect on customer (NTAX) because of poorly calibrated measurement methods. 

17 . Using the EFQM model to reflect on the action research project Validity and reliability: When comparing with EFQM results with earlier experiments, the observed change from 2013 to 2014 is not significant. Learning AR: Use feedback from rejection for writing new grant application. 

18 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

19 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

20 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

21 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

22 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

23 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant 

24 .Analysis: Did the use of the EFQM model improve AR performance? Analysis of AR project Use of EFQM model • Diagnosis AR performance • EFQM self-assessment • Plan for improving AR • Use EFQM theory about performance causes and effects • Execute plan and record • New EFQM assessment of what happens enablers • Evaluate new AR • New EFQM assessment of performance level results • Reflect on causes for • Observed change was not change in AR performance statistically significant H1, H2 & H3: Confirmed by qualitative data 

25 .Discussion: Related research - interpretive AR approach Improving AR through ethics and moral standards (Boog et al, 2007) 

26 .Discussion: Related research - interpretive AR approach Improving AR through ethics and moral standards (Boog et al, 2007) EFQM is concerned with DIAGNOSING 

27 .Discussion: Related research - interpretive AR approach Improving AR through ethics and moral standards (Boog et al, 2007) EFQM is concerned with DIAGNOSING RICH DESCRIPTIONS are of limited use if they do not result in better treatments 

28 .Discussion: Related research - interpretive AR approach Improving AR through ethics and moral standards (Boog et al, 2007) EFQM is concerned with DIAGNOSING RICH DESCRIPTIONS are of limited use if they do not result in better treatments EFQM suggests that TREATMENTS should be evaluated on objective terms 

29 .Discussion: Related research - interpretive AR approach Improving AR through ethics and moral standards (Boog et al, 2007) EFQM is concerned with DIAGNOSING, but not as a goal in itself RICH DESCRIPTIONS are of limited use if they do not result in better treatments EFQM suggests that TREATMENTS should be evaluated on objective terms