Full Text of The Disscussion

Quality assurance, performance management and impact analysis - Team Task Two, second time-bound online discussion
Shown immediately below is the original stimulus material that was posted on the site to stimulate online discussion. Members of the group responded separately to each of the key questions raised and a summary of their responses is shown in subsequent sections.

"What relationship, if any, is there between quality assurance and quality assurance systems, and performance management and impact analysis? We know that there is no shortage of quality assurance systems and quality marks for most types of organisations to choose from, including those organisations delivering guidance. However, does much quality assurance place too great an emphasis upon policies and procedures and too little on directly assessing and enhancing the impact of services on end-users?

Take the matrix Standard as an example. For those not totally familiar with it, the matrix Standard is a national quality standard for organisations delivering information, advice and guidance services for learning and work and it is supported by two organisations - The Employment NTO and The Guidance Accreditation Board (GAB). The standard focuses on five areas of service delivery and five areas of management of the service.

Each of these ten elements is underpinned by a number of service delivery and management criteria, all of which make perfectly good sense. However, how do we know that compliance with all of the relevant criteria actually improves, or at least helps maintain, the service the organisation delivers to the end-user? It's undeniable that there are many perceived benefits of matrix. (Check out the favourable quotations from accredited organisations from the official matrix website www.matrix-quality-standard.com). However, is there any harder evidence available in terms of end-user benefits?

Another parallel example is the Investors in People standard (IIP). Here is a pretty typical statement of the benefits of IIP: " STATISTIC: There is a strong association between achieving the IIP standard and overall business effectiveness." (LSC 2003. Developing your workforce: Invest, Grow, Compete. Coventry: Learning and Skills) But does this "statistic" really mean that achieving IIP actually improves business effectiveness; or could it simply mean that those businesses that had the time, resources and motivation to go for a nationally recognised quality mark were already effective organisations, and that the award itself has added little value?

So, what relationship, if any, is there between quality assurance and quality assurance systems, and performance management and impact analysis?"

Does much quality assurance place too great an emphasis upon policies and procedures and too little on directly assessing and enhancing the impact of services on end-users?

1) A very interesting point this one...

2) I would answer 'yes'

3) I agree ...

4) Quality standards are insufficient

5) Given that so far we are unanimously in favour...

6) Well im going to buck the trend a little and say it could be no...

7) role for critical relection and review

8) QA does need to emphasise policies

9) A question about Career Mark

10) In career mark (Modular) modules...

11) Reply to question about QS being the best way to get practitioners to reflect on impact.

12) Confounding systems and practice

13) We should not fear impact analysis

14) we need a greater emphasis upon practice and the assessment and evaluation of practice.

15) Comment from a Quality manager

How do we know that compliance with all of the relevant criteria actually improves, or at least helps maintain, the service the organisation delivers to the end-user?

Is there any research being planned on the effect of Matrix on impact?
I'm not aware of any such planned research but I fully admit not being up to speed with that aspect of impact analysis. Help! Is there anyone out there who does know the answer?
In reply to the question, this is something we at the Guidance Council have been asking for even before matrix. What impact do quality systems have on the experience of the client? Some before and after evaluations would be very welcome.
I don't know BUT there is a tremendous amount of client feedback being collected all over the country by all sorts of guidance users in the name of quality - can we tap into this in some way?

Does achieving IIP improve business effectiveness?

STATISTIC: " There is a strong association between achieving the IIP standard and overall business effectiveness." . (LSC 2003. Developing your workforce: Invest, Grow, Compete. Coventry: Learning and Skills)

Does this statistic really mean that achieving IIP actually improves business effectiveness; or could it simply mean that those businesses that had the time, resources and motivation to go for a nationally recognised quality mark were already effective organisations, and that the award itself has added little value?

1) Can we have more evidence?

2) Just an observation about liP...

3) In response to request for more evidence

4) Here is some more evidence...

5) I agree with need for longitudinal studies

6) Organisations choosing not to re-accredit to IiP

What relationship, if any, is there between quality assurance and quality assurance systems...

What relationship, if any, is there between quality assurance and quality assurance systems, and performance management and impact analysis?

1) I agree with an earlier comment...

2) Interesting final paragraph...

3) An article by Gold and Villeneuve throws...

4) Challenges fpr resarch to provide an evidence base to inform policy and practice