Skip to main content


<?xml version="1.0"?>

<!DOCTYPE TEI.2 SYSTEM "base.dtd">




<title>Evaluating Student Experiences of Community-Based Medicine Using the Nominal Group Technique</title></titleStmt>

<publicationStmt><distributor>BASE and Oxford Text Archive</distributor>


<availability><p>The British Academic Spoken English (BASE) corpus was developed at the

Universities of Warwick and Reading, under the directorship of Hilary Nesi

(Centre for English Language Teacher Education, Warwick) and Paul Thompson

(Department of Applied Linguistics, Reading), with funding from BALEAP,

EURALEX, the British Academy and the Arts and Humanities Research Board. The

original recordings are held at the Universities of Warwick and Reading, and

at the Oxford Text Archive and may be consulted by bona fide researchers

upon written application to any of the holding bodies.

The BASE corpus is freely available to researchers who agree to the

following conditions:</p>

<p>1. The recordings and transcriptions should not be modified in any


<p>2. The recordings and transcriptions should be used for research purposes

only; they should not be reproduced in teaching materials</p>

<p>3. The recordings and transcriptions should not be reproduced in full for

a wider audience/readership, although researchers are free to quote short

passages of text (up to 200 running words from any given speech event)</p>

<p>4. The corpus developers should be informed of all presentations or

publications arising from analysis of the corpus</p><p>

Researchers should acknowledge their use of the corpus using the following

form of words:

The recordings and transcriptions used in this study come from the British

Academic Spoken English (BASE) corpus, which was developed at the

Universities of Warwick and Reading under the directorship of Hilary Nesi

(Warwick) and Paul Thompson (Reading). Corpus development was assisted by

funding from the Universities of Warwick and Reading, BALEAP, EURALEX, the

British Academy and the Arts and Humanities Research Board. </p></availability>




<recording dur="00:29:38" n="5713">


<respStmt><name>BASE team</name>



<langUsage><language id="en">English</language>



<person id="nm0295" role="main speaker" n="n" sex="m"><p>nm0295, main speaker, non-student, male</p></person>

<person id="nm0296" role="participant" n="n" sex="m"><p>nm0296, participant, non-student, male</p></person>

<person id="of0297" role="observer" n="o" sex="f"><p>of0297, observer, observer, female</p></person>

<person id="nf0298" role="participant" n="n" sex="f"><p>nf0298, participant, non-student, female</p></person>

<person id="nm0299" role="participant" n="n" sex="m"><p>nm0299, participant, non-student, male</p></person>

<person id="nm0300" role="participant" n="n" sex="m"><p>nm0300, participant, non-student, male</p></person>

<person id="nm0301" role="participant" n="n" sex="m"><p>nm0301, participant, non-student, male</p></person>

<person id="nm0302" role="participant" n="n" sex="m"><p>nm0302, participant, non-student, male</p></person>

<person id="nm0303" role="participant" n="n" sex="m"><p>nm0303, participant, non-student, male</p></person>

<person id="nf0304" role="participant" n="n" sex="f"><p>nf0304, participant, non-student, female</p></person>

<person id="nm0305" role="participant" n="n" sex="m"><p>nm0305, participant, non-student, male</p></person>

<person id="nm0306" role="participant" n="n" sex="m"><p>nm0306, participant, non-student, male</p></person>

<person id="nm0307" role="participant" n="n" sex="m"><p>nm0307, participant, non-student, male</p></person>

<person id="nm0308" role="participant" n="n" sex="m"><p>nm0308, participant, non-student, male</p></person>

<personGrp id="ss" role="audience" size="m"><p>ss, audience, medium group </p></personGrp>

<personGrp id="sl" role="all" size="m"><p>sl, all, medium group</p></personGrp>

<personGrp role="speakers" size="16"><p>number of speakers: 16</p></personGrp>





<item n="speechevent">Lecture</item>

<item n="acaddept">Primary Health Care and General Practice</item>

<item n="acaddiv">ls</item>

<item n="partlevel">Staff</item>

<item n="module">unknown</item>




<u who="nm0296"><kinesic desc="projector is on showing slide" iterated="n"/> thanks very much for coming to our symposium <pause dur="0.2"/> on educational <pause dur="0.5"/> research <pause dur="1.1"/> # <pause dur="0.3"/> as you probably know we were officially named Department of Undergraduate G-P Medical Education <pause dur="0.6"/> on the first of <pause dur="0.2"/> March <pause dur="0.3"/> this year when we have been administering <pause dur="0.4"/> the undergraduate G-P <pause dur="0.5"/> course for the <pause dur="0.3"/> <gap reason="name" extent="1 word"/> <pause dur="0.2"/> Medical School of the <gap reason="name" extent="1 word"/> <gap reason="name" extent="1 word"/> <pause dur="0.6"/> Medical Schools <pause dur="0.6"/> # <pause dur="2.0"/> so <pause dur="0.3"/> what we do here at the moment is to mirror <pause dur="0.7"/> exactly what happens at the undergraduate <pause dur="0.6"/> department <pause dur="0.3"/> at <gap reason="name" extent="1 word"/> University the undergraduate G-P department it's exactly the <pause dur="0.4"/> the same curriculum <pause dur="0.7"/> # <pause dur="0.2"/> inevitably <pause dur="0.4"/> our remit is to provide <pause dur="0.5"/> medical education <pause dur="0.4"/> within <pause dur="0.3"/> primary care <pause dur="0.4"/> and <pause dur="0.2"/> teaching students # <pause dur="0.2"/> competencies <pause dur="0.3"/> in the consultation <pause dur="0.9"/> but that also <pause dur="0.4"/> involves research <pause dur="0.4"/> into medical education <pause dur="0.6"/> and <pause dur="0.8"/> all the different people that are speaking today will be talking about a different aspect <pause dur="0.4"/> of <pause dur="0.3"/> undergraduate <pause dur="0.4"/> medical education that will be <pause dur="0.4"/> related to <pause dur="0.4"/> primary care <pause dur="0.7"/> but we're also joined by the <pause dur="1.1"/> # <pause dur="0.2"/> group of people who

are doing a module in effective teaching of the Master's degree <pause dur="0.4"/> in Medical Education who have joined us today <pause dur="0.3"/> of which i am also <pause dur="0.5"/> a student <pause dur="1.2"/> so about <pause dur="0.7"/> ten of our colleagues from that <pause dur="0.4"/> module have joined us <pause dur="0.6"/> today <pause dur="1.2"/> now <pause dur="0.3"/> i'm handing evaluation forms round i'd be very grateful if you would <pause dur="0.3"/> fill those evaluation forms out <pause dur="0.4"/> as the day goes on and various people are <pause dur="0.3"/> are talking they're the same evaluation forms that we're using <pause dur="0.3"/> for the Master's in <pause dur="0.4"/> in Medical Education so it will it will <pause dur="0.4"/> help us to <pause dur="0.2"/> to evaluate that <pause dur="0.6"/> # <pause dur="0.2"/> inevitably because it's an educational research <pause dur="0.5"/> # symposium <pause dur="0.3"/> it's also become an <trunc>educ</trunc> <pause dur="0.2"/> educational research project <pause dur="0.3"/> in itself <pause dur="0.4"/> and i'm going to ask <pause dur="0.4"/> <gap reason="name" extent="2 words"/> from the Centre for <pause dur="0.4"/> English Language <pause dur="0.5"/> Teachers' Education to explain to us <pause dur="0.7"/> what is happening <pause dur="0.6"/> <gap reason="name" extent="1 word"/> </u><gap reason="break in recording" extent="uncertain"/> <u who="of0297" trans="pause"> thanks a lot Dr <gap reason="name" extent="1 word"/> <pause dur="0.4"/> # as Dr <gap reason="name" extent="1 word"/> said my name's <gap reason="name" extent="2 words"/> and i come

from the Centre for English Language Teacher Education <pause dur="0.3"/> on main campus <pause dur="0.9"/> and # this is part of a research project <pause dur="0.4"/> into # <pause dur="0.5"/> the language of <pause dur="0.3"/> academic speech <pause dur="0.4"/> # what we're doing is putting together a database of a whole lot of lectures and seminars from a whole <pause dur="0.4"/> # a whole range of subject areas different subject areas around the university <pause dur="0.5"/> and it's # becoming <pause dur="0.8"/> part of a <pause dur="0.3"/> corpus <pause dur="0.3"/> that is going to be made available to researchers into language so <pause dur="0.4"/> # <pause dur="0.6"/> in fact part of the database has already been used in several papers on # <pause dur="0.3"/> academic vocabulary and syntax and the difference between <pause dur="0.4"/> written and spoken English # these sort of projects that are going on <pause dur="0.4"/> # part of <pause dur="0.3"/> # <pause dur="0.6"/> the clips from the <trunc>res</trunc> # recordings are also being used for another project which is called <pause dur="0.4"/> Essential Academic Skills in English <pause dur="0.3"/> and that's a C-D-ROM project that's aimed at <pause dur="0.6"/> <trunc>univers</trunc> # students who are coming to university from <pause dur="0.5"/> # <pause dur="0.2"/> foreign countries for the first time <pause dur="0.4"/> and what it aims to do is actually ease

them into <pause dur="0.3"/> the university experience give them exposure to the kind of language that they're going to be exposed to <pause dur="0.4"/> when they come to university <pause dur="0.4"/> so as to <pause dur="0.2"/> make the ride easier for them <pause dur="0.2"/> as it were <pause dur="0.5"/> so i just need to make sure that you're okay with being filmed for this <pause dur="0.4"/> is it is it all right </u><pause dur="0.7"/> <u who="nf0298" trans="pause"> yeah </u><u who="nm0299" trans="latching"> sure <vocal desc="laughter" iterated="y" n="ss" dur="1"/></u><u who="of0297" trans="latching"> brilliant thanks a lot <vocal desc="laughter" iterated="y" n="ss" dur="1"/></u><gap reason="break in recording" extent="uncertain"/> <u who="nm0296" trans="pause"> i should have introduced myself actually right at the beginning but i think <pause dur="0.2"/> you all <pause dur="0.2"/> know who i am <pause dur="0.5"/> # but i'm <gap reason="name" extent="2 words"/> i'm a general practitioner in <gap reason="name" extent="3 words"/> and a senior lecturer here <pause dur="0.6"/> and i was informed by one of my colleagues at the beginning <pause dur="0.3"/> the longer the title that you have in the university is usually <pause dur="0.4"/> # correlates with your <pause dur="0.2"/> lesser importance <pause dur="0.2"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> in the university so i've got this grand title of

being Director of G-P Undergraduate Medical Education <pause dur="0.5"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> # <pause dur="1.3"/> so that's a <trunc>l</trunc> # a little bit about myself <pause dur="0.4"/> with relation to the day itself all the <pause dur="0.4"/> # speakers will <pause dur="0.2"/> be allocated half an hour <pause dur="0.3"/> the idea is that they speak for fifteen minutes because hopefully it's going to generate a lot of discussion <pause dur="0.4"/> amongst the group so hopefully <trunc>feef</trunc> fifteen minutes discussion <pause dur="0.5"/> at eleven-thirty we'll break for coffee <pause dur="0.4"/> and then at twelve-forty-five we will make our way to the central <pause dur="0.4"/> campus to eat in what is referred to as the Eat restaurant <pause dur="0.4"/> in the Arts Centre <pause dur="0.5"/> where you'll have a very nice meal <pause dur="0.4"/> and # then we'll come back here <pause dur="0.4"/> again at two o'clock <pause dur="0.5"/> # i should also say that i've organized today with my colleague who's sat here <kinesic desc="indicates member of audience" iterated="n"/> <gap reason="name" extent="2 words"/> <pause dur="0.5"/> who is a G-P in <gap reason="name" extent="1 word"/> and a lecturer here <pause dur="0.4"/> in the department so he's been putting in a lot of work <pause dur="0.4"/> behind the scenes <pause dur="0.3"/> for today and is also going to be <pause dur="0.2"/> speaking a little bit later <pause dur="0.8"/> so we should introduce our <pause dur="0.4"/>

first speaker <pause dur="0.5"/> # Dr <gap reason="name" extent="2 words"/> <pause dur="0.5"/> who is from <gap reason="name" extent="1 word"/> University <pause dur="0.4"/> he is an academic G-P registrar but is shortly to become a lecturer <pause dur="0.4"/> in communication skills <pause dur="0.4"/> at <gap reason="name" extent="1 word"/> University he's also a general practitioner <pause dur="0.5"/> and he's going to talk about student centred evaluation using <pause dur="0.4"/> the nominal group technique <pause dur="1.4"/> thank you </u><pause dur="10.1"/> <u who="nm0295" trans="pause"> right <vocal desc="clears throat" iterated="n"/> thank you very much <pause dur="0.2"/> # it's a it's a pleasure to come here to speak to this group <pause dur="0.5"/> # as <pause dur="1.2"/> as i slowly sort of spread the word of what i've been doing # <pause dur="0.9"/> as the the title says # i submitted this for my # my M-S-C in primary care <pause dur="0.4"/> and yesterday i was # i was fortunate to receive the news that it has formally been accepted <pause dur="0.4"/> # subject to the final exam boards so probably as <pause dur="0.3"/> formally accepted as it possibly can be <pause dur="0.7"/> # <pause dur="1.3"/> whereabouts are the <pause dur="0.3"/> forward <pause dur="0.3"/> and back <pause dur="0.2"/> for the slides on the <pause dur="1.5"/> for the Powerpoint <pause dur="0.5"/> is there # something

i'm not seeing <pause dur="0.4"/> <event desc="opens drawer" iterated="n"/> aha </u><u who="nm0300" trans="latching"> just pull it </u><u who="nm0295" trans="overlap"> yes there is there's a hidden drawer <vocal desc="laughter" iterated="y" n="sl" dur="1"/><pause dur="0.6"/> <vocal desc="cough" iterated="n"/> <pause dur="0.3"/> marvellous right <pause dur="0.9"/> okay <pause dur="0.2"/> so <pause dur="0.4"/> # the the nominal group technique i'll come to describe a bit more but just to mention that it was something that perhaps i came upon entirely by accident it may well be something that <trunc>p</trunc> people here don't know an awful lot about <pause dur="0.2"/> but hopefully i'll change that as the course of this as we go along <pause dur="1.8"/><kinesic desc="changes slide" iterated="n"/> some of what i i wanted to talk about really is evaluation and the whole process of evaluation that <pause dur="0.4"/> # for me <pause dur="1.7"/> i i felt that evaluation wasn't always a terribly useful thing in the way that it's being done <pause dur="0.5"/> and <trunc>th</trunc> this just a just a way of summarizing what happens as part of any educational event <pause dur="0.3"/> that you know you want to collect data analyse interpret disseminate it make a decision <pause dur="0.3"/> and this informs then to change your practice <pause dur="0.8"/> # <pause dur="1.1"/>

what i found particularly interesting about evaluation was the the the aspects really of data collection <pause dur="1.0"/> # and with data collection and <trunc>e</trunc> evaluation <pause dur="0.8"/> there were two things that i always wanted to know about this is <pause dur="0.4"/> why we were <pause dur="0.2"/> were we collecting this information <pause dur="0.4"/> and who were we collecting this sort of information for <pause dur="0.5"/> and generally what i found was that it probably isn't actually for you <pause dur="0.5"/> that often evaluations are done because somebody has decided that something is important <pause dur="0.5"/> but it might not necessarily be what's actually important to you the the person that's experiencing <pause dur="0.6"/> the learning <pause dur="1.3"/> so i wanted to create something that was a student centred view <pause dur="0.4"/> # <pause dur="0.3"/> of an educational experience <pause dur="2.1"/> now <pause dur="0.8"/> evaluation is a is a difficult thing and <trunc>i</trunc> <pause dur="0.2"/> in different circles it's thought of in in sort of very high terms or very low terms <pause dur="0.4"/> some people <pause dur="0.2"/> and if i but to me this is

the at the top of the list is that people believe it's a purely administrative task <pause dur="0.3"/> it's done because it has to be done because somebody above you says <pause dur="0.4"/> evaluate this <pause dur="0.3"/> you tick the boxes you hand it in and the job is done <pause dur="1.4"/> there can be resistance within a faculty setting towards evaluation again believing that it's that you know it's a waste of time <pause dur="0.6"/> # <pause dur="0.7"/> often there's little opinion in what the students actually think about what's going on yes they tick the boxes they fill in the things <pause dur="0.4"/> but then these things just get cast to one side <pause dur="0.5"/> # <pause dur="0.4"/> if you're asking for self-reported data # i think i'm a wonderful lecturer <pause dur="0.3"/> isn't going to be particularly useful when it comes to evaluation <pause dur="0.6"/> and overall just apathy and disinterest in the whole process <pause dur="1.3"/> so i wanted to create something that was <trunc>t</trunc> <trunc>th</trunc> <trunc>w</trunc> that was as as student centred as i could possibly make it <pause dur="0.5"/> so i began by taking a focus group of <kinesic desc="changes slide" iterated="n"/> # final year medical students and took them through a semistructured <pause dur="0.5"/> # focus group

asking them broadly about what they were doing <pause dur="0.4"/> # within the the community based medicine course at Birmingham University <pause dur="2.2"/> then <pause dur="0.2"/> what i did is i used the information from this focus group <pause dur="0.7"/> to devise statements for the use with the nominal group technique <pause dur="0.5"/> now by a statement i mean it's sort of like a stimulus statement <pause dur="0.4"/> so just # <trunc>a</trunc> an example from outside of medicine is <pause dur="0.3"/> what do you think it's important to do before you go to work in the morning <pause dur="0.4"/> i mean it's a silly example but it just gives you an idea of what a stimulus statement is so that poses a question <pause dur="0.4"/> to which people may then generate lots of ideas <pause dur="1.7"/> <kinesic desc="changes slide" iterated="n"/> so what is the nominal group technique well the nominal group technique <pause dur="0.4"/> is a a a structured way of generating ideas <pause dur="0.3"/> and it consists of several different phases <pause dur="1.3"/> the first part is a silent phase so you <trunc>pr</trunc> you <trunc>y</trunc> you you <pause dur="0.2"/> provide <trunc>th</trunc> the the participants with a

statement <pause dur="0.4"/> and <pause dur="0.4"/> so for instance the one about what you do before you go to work <pause dur="0.3"/> then they work individually <pause dur="1.0"/> # in response to this so they work on paper <pause dur="0.4"/> and they write down a list of everything they can think of in response to that statement and it has to be individual no discussion occurs <pause dur="1.3"/> the second part is the round robin phase so <pause dur="0.7"/> each person <pause dur="0.2"/> in turn reads out the first thing on their list it gets transmitted # <trunc>trans</trunc> <pause dur="0.2"/> transmitted transcribed <pause dur="0.3"/> to a chart on the wall <pause dur="0.6"/> where <pause dur="0.5"/> that goes on in in a cycle process each person saying the next thing on their list until you have a grand list of absolutely everything that people have written down <pause dur="0.4"/> again <pause dur="0.2"/> there's no time for # <pause dur="0.5"/> for <trunc>s</trunc> speaking <pause dur="0.3"/> other than saying what's on your list you can't debate whether or not that's a stupid thing a good thing or a bad thing <pause dur="0.3"/> everything goes on the list <pause dur="1.9"/> this is the only time at which <trunc>speak</trunc> speaking is allowed and this is the clarification phase so you can get to understand what

other people mean by their statements <pause dur="0.6"/> it is also the only time at which anything can be deleted from the list <pause dur="0.4"/> so for instance <pause dur="0.2"/> <trunc>a</trunc> you know <trunc>i</trunc> identical duplications which do happen a lot during this technique <pause dur="0.4"/> but people can't say i don't agree with that being on the list take it off <trunc>b</trunc> if it goes on the list <pause dur="0.2"/> it stays on the list <pause dur="1.1"/> and then something that's quite crucial for the nominal group technique <pause dur="0.2"/> is the voting phase <pause dur="0.9"/> so for instance in response to a statement they may be presented with <pause dur="0.3"/> thirty statements <pause dur="0.2"/> you then say to them <pause dur="0.4"/> what are the seven most important things to you <pause dur="0.4"/> so you as the individual learner in this experience <pause dur="0.2"/> what is important to you <pause dur="0.6"/> the most important thing then getting seven votes the next thing six votes five votes four <trunc>fo</trunc> four votes and so on <pause dur="0.8"/> and then you collect in all the votes for all the statements <pause dur="0.5"/> and then you feed that information back <pause dur="0.3"/> to the participants <pause dur="0.3"/> so it's a way of generating a large amount of information <pause dur="0.3"/> but

then asking people to whittle it down to what really is important <pause dur="0.3"/> and then using the votes to calculate what the most important thing is <pause dur="0.3"/> to the group as a whole <pause dur="2.2"/> <kinesic desc="changes slide" iterated="n"/> now the nominal group technique has many advantages <pause dur="0.7"/> it's a structured way <pause dur="0.3"/> so it's not like filling in evaluation forms and then reading thousands of comments that have been written and trying to tabulate them in some way <pause dur="0.2"/> it's a structured way of doing this <pause dur="1.0"/> it's non-confrontational if you write it down you say it people can't shout you down for saying things that you don't necessarily agree with <pause dur="0.8"/> it disperses dominance you can't have one voice going on and on about one particular thing <pause dur="0.3"/> if you've said it once you can't say it again it's on the list it comes down to the voting <pause dur="0.3"/> as to whether or not that's important to everyone <pause dur="1.8"/> it's also very easy <pause dur="0.2"/> to produce data as we'll

come on to see how much data this sort of thing produces <pause dur="0.8"/> so what statements did i use <pause dur="0.9"/> well through the nominal <trunc>gr</trunc> through the the focus group we discovered various bits that were quite interesting for the students and things that provoked the most debates so we generated three statements and they're quite similar to some other pieces of research that have been done <kinesic desc="changes slide" iterated="n"/> like this <pause dur="0.8"/> so we asked them first of all what would you change <pause dur="0.3"/> about the community based medicine course in the final year <pause dur="1.2"/> then we asked them what did you enjoy <pause dur="0.6"/> about the community based medicine course in the final year <pause dur="0.7"/> and finally <pause dur="0.2"/> what does final year community based medicine teaching give you <pause dur="0.5"/> that teaching in other settings does not <pause dur="3.1"/><kinesic desc="changes slide" iterated="n"/> so <pause dur="0.2"/> <vocal desc="clears throat" iterated="n"/><pause dur="0.3"/> at the nominal group we recruited eighteen students to attend

and they were split on the day into two groups purely for logistical reasons and and handling large groups <pause dur="0.8"/> then they were combined <pause dur="0.3"/> towards the end of the process so that they all had their master lists to look at <pause dur="0.4"/> so in response to the three different statements <pause dur="0.2"/> they generated a hundred-and-twenty-four separate ideas <pause dur="0.8"/> and from that <pause dur="0.2"/> they voted the top seven in response to each statement so we had twenty-one statements <pause dur="0.3"/> that according to this group <pause dur="0.4"/> were the most important things overall <pause dur="0.5"/> about community based medicine at Birmingham <pause dur="0.9"/> so you may ask well what does eighteen students mean in comparison to the whole of the year it was about approximately two-hundred students <pause dur="0.6"/> so what we did is we then converted that into a questionnaire <pause dur="0.5"/> so the twenty-one statements we then asked the whole year do you agree or disagree <pause dur="0.3"/> with this <pause dur="0.2"/> as a as an important statement <pause dur="1.0"/><kinesic desc="changes slide" iterated="n"/> a hundred-and-eighty students # responded which is a response rate of about eighty per cent which we felt was quite

acceptable <pause dur="0.8"/> and overall out of the twenty-one statements that were made twenty of them obtained majority support <pause dur="1.2"/> # what you say is majority support is you know <trunc>s</trunc> is very much up to you <pause dur="0.5"/> the the cut-off for twenty <unclear>year</unclear> to twenty-one is at fifty per cent so over fifty per cent of the year agreed <pause dur="0.4"/> but as we'll come on to see that most people did have a very high level of agreement <pause dur="1.8"/> <kinesic desc="changes slide" iterated="n"/> so here we have <pause dur="0.2"/> # <pause dur="1.0"/> the layout of it so the three colours represent the three different statements <pause dur="0.4"/> with <pause dur="0.2"/> each statement number <pause dur="0.4"/> so these are the seven things <pause dur="0.2"/> in response to each stimulus statement these are the seven items along the bottom <pause dur="0.7"/> and you can see that on the whole if you draw a rough line across eighty per cent you still get most of the people <pause dur="0.5"/> agreeing over eighty per cent and then down here you've got these two <pause dur="0.5"/> statements which really didn't overall get very good # agreement within the whole year <pause dur="2.5"/> so the poorer scoring items what were they <pause dur="0.3"/> <kinesic desc="changes slide" iterated="n"/> well the first one was about general

knowledge on the ophthalmology and dermatology exam and i think this was a specific thing to this group that they were upset about the content of the exam when they had it <pause dur="0.5"/> but it seemed when it was presented to the whole year <pause dur="0.2"/> that overall that just wasn't agreed with <pause dur="1.5"/> there was a # # just a <trunc>s</trunc> a slight shift towards agreement with this statement just saying that you know we wanted more formal tutorials in general practice they wanted more set taught # tutorials on audit <pause dur="0.3"/> on screening on cardiovascular disease <pause dur="0.4"/> but again <trunc>y</trunc> <trunc>n</trunc> there was a high level of disagreement with that statement <pause dur="0.8"/><kinesic desc="changes slide" iterated="n"/> so i'll i'll just move on now to the the supported items and i won't bore you in great detail of absolutely every <trunc>th</trunc> single thing i just picked out the ones that i thought were the most exciting <pause dur="1.5"/> <kinesic desc="changes slide" iterated="n"/> so <pause dur="0.2"/> what would you change about community based medicine course well they wanted more

dermatology teaching there was no doubt about that and i think this harks back to even the days when i was a medical student the dermatology <pause dur="0.5"/> among students is poorly thought of and on the whole can be quite badly taught <pause dur="1.3"/> there were small numbers who felt that it was being done well and i think that represented small pockets of dermatology teaching within the practices that were being done very well <pause dur="0.3"/> but on the whole <pause dur="0.2"/> it wasn't being done very well at all <pause dur="2.3"/> <kinesic desc="changes slide" iterated="n"/> the course at Birmingham is split into an inner city practice and an <trunc>ou</trunc> sort of an outer city practice <pause dur="0.5"/> and the students were flitting between the two centres and they actually felt that it was very bitty and # disorganized <pause dur="0.3"/> and they'd much rather have <pause dur="0.3"/> core central blocks <pause dur="0.4"/> at one place rather than being spread out and moving all the time <pause dur="2.8"/> <kinesic desc="changes slide" iterated="n"/> and finally on this one they wanted feedback on their examination stations

they have a voices exam where they they rotate round different stations <pause dur="0.3"/> and unless they do particularly badly or there is something of particular concern the students don't get <pause dur="0.3"/> feedback about that station they were very keen to have that <pause dur="0.3"/> 'cause they do get that in some of the specialties <pause dur="1.4"/> <kinesic desc="changes slide" iterated="n"/> what did you enjoy about final year community based medicine <pause dur="0.2"/> well they enjoyed the one to one tuition <pause dur="0.3"/> which i thought was very very reassuring <pause dur="0.6"/> they found the communication skills role plays very useful <pause dur="0.3"/> # <trunc>th</trunc> they found the the words they used were both useful and informative so that that was good feedback for us <pause dur="0.9"/> and also that the voices exam was relaxed and fair <pause dur="0.2"/> which again for students to volunteer that <unclear>something</unclear> about an exam i think is a very positive step <pause dur="1.7"/> then the final part <pause dur="0.9"/> # <pause dur="0.7"/> <kinesic desc="changes slide" iterated="n"/> is what i call the Heineken question <pause dur="0.6"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> and <pause dur="0.2"/> this was <pause dur="0.3"/> what does final year community based

medicine give you <pause dur="0.2"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> that other settings do not so what's community based medicine reaching out to <pause dur="0.3"/> that other teaching settings <pause dur="0.2"/> simply aren't getting to <pause dur="0.4"/> so we'll just focus on this one a little bit more 'cause i think this is the most exciting of them <pause dur="1.2"/><kinesic desc="changes slide" iterated="n"/> first of all # they felt that this was the <trunc>f</trunc> you know their real opportunity to have clinical autonomy and responsibility more than in other settings <pause dur="0.8"/> holding your own consultations they they they seemed to feel that in hospital settings <pause dur="0.3"/> you don't get to hold your own consultations you go in you you take the whole medical history you present it you don't have anything to do with it <pause dur="0.4"/> within the primary care community based setting that was very different <pause dur="0.9"/> it was less focused on the rare they were fed up of seeing ridiculous <pause dur="0.4"/> # outside # things in hospital whereas they're actually seeing normal things happening in community based <pause dur="0.3"/> setting <pause dur="0.8"/> and also they

were being treated as a colleague rather than as a student so they were saying that they could develop this more professional relationship with those around them <pause dur="1.9"/> <kinesic desc="changes slide" iterated="n"/> they were able to to see and experience # continuity of care something they weren't getting in the secondary care setting <pause dur="0.6"/> they were improving their confidence in their patient skills <pause dur="0.5"/> and i think just an important thing for some of us here <pause dur="0.2"/> is that they were had more enthusiastic teachers and support from their mentors <pause dur="0.3"/> which again i think it's a great thing to hear coming out of out of this setting <pause dur="1.8"/><kinesic desc="changes slide" iterated="n"/> so overall as a technique <trunc>w</trunc> you know <trunc>wha</trunc> what do i think about this technique well i think it's i think it's a a <trunc>w</trunc> a wonderful technique <pause dur="0.5"/> # it's cheap i mean the the whole thing other than supplying <trunc>s</trunc> # food for the students 'cause it does take <pause dur="0.2"/> a few hours <pause dur="0.3"/> was was you know the the manpower of me <pause dur="0.5"/> # it was easy to run it was just simply setting the questions the students

went <trunc>a</trunc> went away made their own data <pause dur="0.3"/> minimum input from me <pause dur="0.6"/> and it so it was reasonably quick the students in their feedback to me did suggest that it did take quite a while which was three hours <pause dur="0.6"/> i mean i suppose in student terms three hours is a very long time <pause dur="1.0"/> # <pause dur="0.5"/> it's a student centred way they generated this data my only input was to create the three things <pause dur="0.2"/> that they could then <pause dur="0.3"/> spill out and talk about whatever they wanted to about <pause dur="0.7"/> and also they created a large amount of data and that's to say that <pause dur="0.4"/> i know the twenty-one statements are the most important they had the highest votes <pause dur="0.4"/> but they were <pause dur="0.3"/> the # you know a hundred or so other items <pause dur="0.2"/> that were still interesting that were coming out of this <pause dur="1.0"/> and as the study has has suggested <pause dur="0.5"/> the <trunc>s</trunc> the opinions of the students there and then were representative of the year as a whole <pause dur="0.4"/>

and that's just to say that's Birmingham and that was that final year <pause dur="0.3"/> i don't know if it would ever <pause dur="0.2"/> # sort of leak out in to be applicable to other settings <pause dur="1.5"/> <kinesic desc="changes slide" iterated="n"/> as far as community based medicine is concerned at Birmingham it's thought of highly by the students who have participated in the course <pause dur="0.7"/> it allows personal and professional growth <pause dur="0.4"/> and something that i that i felt was you know very nice was that it does bridge this gap it seems <pause dur="0.3"/> from being a student to becoming a doctor <pause dur="0.4"/> # and it seems that # you're allowed and able to do that in the primary care setting more so than other places <pause dur="1.3"/><kinesic desc="changes slide" iterated="n"/> so the future of this i think <trunc>i</trunc> it can go anywhere and everywhere for those people who wish to use it <pause dur="0.4"/> # we've already used it evaluating our firm one scheme in in the year four <pause dur="0.3"/> and we're using it soon to look at the communication needs for overseas doctors <pause dur="0.9"/><kinesic desc="changes slide" iterated="n"/> as to where it's

already been for me and <pause dur="0.4"/> it's already been disseminated partially within the university and it's going to be <pause dur="0.4"/> # sent off to the curriculum development and implementation group <pause dur="0.3"/> along with quality assurance as well <pause dur="0.3"/> it's been presented at the R-C-G-P research symposium in Birmingham <pause dur="0.6"/> and it's been accepted as a short communication for AMEE in Bern this year <pause dur="0.4"/> and it's also # sort of <pause dur="0.3"/> in the pipeline for publication </u><gap reason="break in recording" extent="uncertain"/> <u who="nm0301" trans="pause"> i've got one <pause dur="1.5"/> hi <pause dur="0.2"/><vocal desc="laughter" iterated="y" n="sl" dur="1"/> how are you <pause dur="1.0"/> # <pause dur="0.2"/> i was just <trunc>won</trunc> <pause dur="0.3"/> what went through my mind # as you as you were talking was it's a great way of collecting a majortiy view <pause dur="0.4"/> of <trunc>ar</trunc> of areas # <pause dur="0.3"/> of improvement things that need to be need to be dealt with <pause dur="0.6"/> i was <pause dur="0.2"/> i was wondering what happens to the other vast amount of data that gets lost <pause dur="0.4"/> during the deletion process because it seems that's where you might get <pause dur="0.2"/> slightly more interesting off the wall <pause dur="0.3"/> responses to the programme </u><pause dur="0.3"/> <u who="nm0295" trans="pause">

i mean absolutely i mean that's why i mentioned that the fact that you do get this great yield of data <pause dur="0.2"/> # <pause dur="0.4"/> and as yet we haven't done anything with the other data that's there <pause dur="0.5"/> # i'm not i don't have the numbers with me but the votes that were allocated to <pause dur="0.4"/> # these particular <pause dur="0.2"/> things that made it into the top twenty-one <pause dur="0.2"/> were head and shoulders way above absolutely everything else <pause dur="0.4"/> that the that the the highest one i think was scoring seventy to eighty votes <pause dur="0.4"/> and then suddenly you get down into the mid-twenties and it will drop off into just being three or four votes in total for all the other items <pause dur="0.8"/> # and it's interesting looking at the data that came out of there <pause dur="0.3"/> there was some <pause dur="0.3"/> # <trunc>r</trunc> really <trunc>o</trunc> odd stuff that came out which i which i felt was very good because it does give everyone a voice and i think one of the things said <pause dur="0.4"/> # tutors need to stop taking themselves so seriously <pause dur="0.5"/><vocal desc="laughter" iterated="y" n="ss" dur="1"/> # which is of course is not going to get something to the top of the

vote but it's something that some students did feel was quite important <pause dur="0.8"/> but <trunc>w</trunc> i mean you qualitatively you could look at the sorts of things then group it together in categories that other things <pause dur="0.2"/> other things people find useful <pause dur="0.4"/> but we've not done anything with it yet </u><pause dur="0.3"/> <u who="nm0302" trans="pause"> are you going to send an e-mail <shift feature="voice" new="laugh"/>round the department about tutors taking themselves <vocal desc="laughter" iterated="y" n="sl" dur="1"/> seriously <shift feature="voice" new="normal"/></u><pause dur="0.3"/> <u who="nm0295" trans="pause"> probably wouldn't be the best idea <vocal desc="laughter" iterated="y" n="sl" dur="1"/></u><pause dur="1.6"/> <u who="nm0303" trans="pause"> can you just clarify for me from the <pause dur="1.0"/> how does the focus group decide at three statements i'm a bit confused with the process </u><u who="nm0295" trans="latching"> okay i mean <trunc>i</trunc> it didn't directly decide the three statements it informed the three statements <pause dur="0.4"/> that # # <pause dur="0.4"/> i mean i i i had an idea as to what i thought might be important within

the or or <pause dur="0.3"/> or what what <pause dur="0.5"/> i i wanted to do something that would create an open enough question <pause dur="0.4"/> that would allow me to to find out what was out there what i thought was out there <pause dur="0.5"/> but at the same time not be so narrow and limited that it didn't give students the breadth and <trunc>off</trunc> of opportunity <pause dur="0.5"/> to you know to explore the the # what was going on <pause dur="0.5"/> so i one of the one of the reasons for doing it was to ensure that <pause dur="0.3"/> my questions wouldn't <pause dur="0.3"/> be too narrow <pause dur="0.4"/> and <pause dur="0.3"/> the data that came out of the focus group <pause dur="0.3"/> # on the whole in fact probably all of it <pause dur="0.2"/> was represented in one way or another within the nominal group <pause dur="0.5"/> i just wanted to make sure that there wasn't one specific area that we were going to completely miss by asking those questions </u><pause dur="0.2"/> <u who="nm0303" trans="pause"> so the three statements you decided the three statements </u><u who="nm0295" trans="overlap"> i <trunc>d</trunc> </u> <u who="nm0303" trans="overlap"> you asked about enjoyment change something <pause dur="0.2"/> right </u><u who="nm0295" trans="latching">

that's right <pause dur="0.6"/> i mean part of the reason behind that was <trunc>w</trunc> when i <pause dur="0.3"/> when i ran the focus group one of the <trunc>f</trunc> # one of the last things i asked <pause dur="0.3"/> was <trunc>w</trunc> what what would you change about # the the the C-B-M course <pause dur="0.3"/> and and and that just had them # running on and on and on and on and it was just such a <pause dur="0.3"/> a big area to be explored <pause dur="0.2"/> that so many ideas came out of it that i thought well <pause dur="0.3"/> i think that will be an important one to do <pause dur="0.4"/> because from the focus group i can't tell what the important things are here <pause dur="0.4"/> but in a voting <trunc>s</trunc> # setting <pause dur="0.2"/> then perhaps they will be able to narrow it down </u><pause dur="1.1"/> <u who="nm0303" trans="pause"> thank you </u><pause dur="1.6"/> <u who="nf0304" trans="pause"> i just want to ask <pause dur="0.3"/> # the three hour session that you had <pause dur="0.7"/> with the when you did the nominal group was that the whole year or was that <pause dur="0.2"/> a certain selected group </u><pause dur="0.2"/> <u who="nm0295" trans="pause"> that that was the eighteen students that agreed to participate </u><u who="nf0304" trans="latching"> i was going to say do you think you've got some responder bias from eighteen people who voluteered to come along </u><u who="nm0295" trans="latching">

i mean <pause dur="0.3"/> certainly yes <pause dur="0.2"/> because <trunc>th</trunc> <trunc>obv</trunc> <trunc>the</trunc> these are people who are motivated to come along and do this sort of thing <pause dur="0.6"/> and indeed they may well have generated data that to them was important and potentially biased but again it did achieve fairly good support from the year as a whole <pause dur="0.4"/> that's not to say that other things would all would would not have <pause dur="0.3"/> have obtained similar <trunc>r</trunc> <pause dur="0.2"/> # <pause dur="0.3"/> # overall agreement <pause dur="0.8"/> but i mean <trunc>i</trunc> <trunc>w</trunc> it was difficult to find any other way to to reach these people <pause dur="1.6"/> yep </u><u who="nm0305" trans="latching"> mine was on the same lines actually i was wondering how you chose your eighteen students and whether you think it would have made a difference if you had a different eighteen students </u><pause dur="0.3"/> <u who="nm0295" trans="pause"> # <pause dur="0.7"/> it would be really interesting to find out if it had if it would have actually have been different <pause dur="0.4"/> # <pause dur="0.3"/> # for the for the choosing we used e-mail to recruit them because <pause dur="0.2"/> all of the students at Birmingham <unclear>probably</unclear> i'm sure it's true of all universities have an e-mail address allocated

to them when they join the first year <pause dur="0.8"/> and # the response rate overall wasn't too bad to the initial e-mail the people who wanted <pause dur="0.3"/> to actually take part was much smaller <pause dur="0.8"/> # <pause dur="0.3"/> but i think the response rates to the the e-mails <pause dur="0.2"/> after badgering on e-mail was seventy-odd per cent <pause dur="0.6"/> so i mean we thought it was a reasonably good way of recruiting people because it was free <pause dur="0.7"/> you know there was no postage no printing </u><pause dur="2.4"/> <u who="nm0306" trans="pause"> # <pause dur="0.7"/> if you'd sat down and wrote a twenty-one item questionnaire with three headings yourself how <pause dur="0.3"/> different would it have been <pause dur="0.7"/> from what the students came up with </u><pause dur="2.2"/> <u who="nm0295" trans="pause"> that's a good question <pause dur="0.4"/> # <pause dur="5.2"/> i honestly don't know </u><pause dur="0.3"/> <u who="nm0306" trans="pause"> right </u><u who="nm0295" trans="overlap"> # <pause dur="0.4"/> i mean i i'm sure that that <pause dur="0.3"/> the faculty and students alike would create <pause dur="0.2"/> similar sorts of statements <pause dur="0.6"/> # <pause dur="0.2"/> in part but i i still think there are things that they brought up that that

have been of surprise to us <pause dur="0.5"/> i suppose we could have put them in <pause dur="0.4"/> # and be surprised by the results but i was i think <pause dur="0.3"/> part of it was a surprise that they came up with them in the first place </u><pause dur="1.0"/> <u who="nm0306" trans="pause"> i mean # the reason i ask is that we use the <pause dur="0.4"/> the traditional technique of writing a questionnaire from our <pause dur="0.3"/> obviously <pause dur="0.2"/> hopefully good knowledge of what our course entails and then <pause dur="0.4"/> offering free text comments to to harvest the idiosyncratic or different <pause dur="0.7"/> # <pause dur="0.2"/> views of students <pause dur="0.6"/> and we're reasonably comfortable with that it gives us reliable results i mean <pause dur="0.4"/> if if i was to <trunc>descrip</trunc> with a <pause dur="0.2"/> group of Clinical Methods students at Leicester i'm absolutely certain one of the items they would come up with is <pause dur="0.4"/> # <pause dur="0.7"/> student support to travel to practices <pause dur="0.2"/> yet <pause dur="0.2"/> i would never ask <pause dur="0.2"/> a question like travel to practices 'cause it's quite <trunc>with</trunc> without our control we have to use the practices we have <pause dur="0.5"/> so the students will generate items <pause dur="0.3"/> of <pause dur="0.3"/> and data about <pause dur="0.2"/> something that i can't change where of course </u><u who="nm0295" trans="overlap">

yeah <pause dur="0.2"/> <vocal desc="clears throat" iterated="n"/> </u><u who="nm0306" trans="overlap"> i wouldn't put that in my <pause dur="0.3"/> own questionnaire </u><pause dur="0.9"/> <u who="nm0295" trans="pause"> i i suppose our rationale behind it was <pause dur="0.3"/> because # <pause dur="0.5"/> you can't do anything about it we were still keen to know whether or not the students felt it was important <pause dur="0.3"/> and <trunc>i</trunc> and indeed they did come up with that that they wanted to have <pause dur="0.4"/> i think they wanted to have closer practices and they wanted to have better # # monetary reimbursements <pause dur="0.4"/> but they got voted out <pause dur="0.4"/> <trunc>th</trunc> there were things that were more important to the student that came above it </u><pause dur="1.8"/> <u who="nm0307" trans="pause"> were there actually any incentives for the students to turn up and do your focus group </u><pause dur="0.4"/> <u who="nm0295" trans="pause"> # the the first focus group # <pause dur="0.3"/> there were no incentives for that <pause dur="0.2"/> # but for the the nominal group the only incentive was food </u><pause dur="0.4"/> <u who="nm0307" trans="pause"> right <pause dur="0.5"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> 'cause it does <vocal desc="laughter" iterated="y" n="nm0295" dur="1"/>

make a difference to what group of students will <pause dur="0.6"/> turn up </u><u who="nm0295" trans="overlap"> yeah </u><u who="nm0307" trans="overlap"> 'cause i always went when i was a student <vocal desc="laughter" iterated="y" n="nm0295" dur="1"/> if there was something like ten pound in an envelope or something like there was just # out of interest <pause dur="0.3"/> sort of <trunc>whi</trunc> which bias group you get the okay </u><u who="nm0295" trans="overlap"> yeah <pause dur="0.2"/> i mean it was it was quite strange because i i did evaluation with the group at the end and i said you know <trunc>w</trunc> what did you enjoy what didn't you enjoy about this technique <pause dur="0.4"/> and even though i'd spent a fortune on food hot pizzas <pause dur="0.3"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> and i'd <trunc>s</trunc> supplied them all evening <pause dur="0.3"/> several people commented we would like to have had more food <pause dur="0.4"/> <vocal desc="laughter" iterated="y" n="ss" dur="2"/> # several said <pause dur="0.3"/> that they would like to have beer <pause dur="0.4"/> <vocal desc="laughter" iterated="y" n="ss" dur="1"/> and have i i can imagine the responses i would have had if i'd given them beer <vocal desc="laughter" iterated="y" n="sl" dur="1"/> </u><u who="nm0307" trans="overlap">

different probably </u><pause dur="0.2"/> <u who="nm0295" trans="pause"> yes <vocal desc="laughter" iterated="y" n="sl" dur="1"/></u><pause dur="1.2"/> <u who="nm0308" trans="pause"> do you think with the # <pause dur="0.6"/> computer now and Internet is it possible to do a similar thing <pause dur="0.4"/> but <pause dur="0.4"/> through voting in the Internet </u><pause dur="0.4"/> <u who="nm0295" trans="pause"> <trunc>the</trunc> there is </u><u who="nm0308" trans="overlap"> then you might get more people involved </u><pause dur="0.2"/> <u who="nm0295" trans="pause"> # i mean that # a sort of an allied technique to the nominal group technique is the Delphi technique <pause dur="0.4"/> which is <trunc>aim</trunc> <trunc>th</trunc> <trunc>th</trunc> this <trunc>th</trunc> this is different 'cause this is face to face <pause dur="0.4"/> whereas the Delphi technique involves circulating a questionnaire <pause dur="0.4"/> # <trunc>ei</trunc> either by mail or electronically <pause dur="0.3"/> and then collating the results formulating a new questionnaire sending it out and refining and and recycling <pause dur="0.5"/> it's something that's that's potentially in the pipeline for me because <pause dur="0.2"/> i i mean i i agree with you i think with the the the the growth of electronic use <pause dur="0.3"/> then it would be a very <trunc>g</trunc> easy and a

very cheap way of doing that sort of research </u><pause dur="5.1"/> <u who="nm0296" trans="pause"> <gap reason="name" extent="1 word"/> thanks very much and congratulations on your <pause dur="0.2"/> Master's degree </u><pause dur="0.8"/> <u who="nm0295" trans="pause"> thank you </u><pause dur="5.8"/> <kinesic desc="applause" iterated="y" n="ss" dur="5"/> <u who="nm0296" trans="pause"> it's very exciting what the academic <pause dur="0.2"/> training schemes in general practice <pause dur="0.4"/> have <pause dur="0.3"/> # or the individuals like <gap reason="name" extent="1 word"/> who've been involved have have generated as as research projects <pause dur="0.3"/> for those of you who don't know there are academic training schemes in general practice based here at <gap reason="name" extent="1 word"/> <pause dur="0.4"/> at <gap reason="name" extent="1 word"/> and <gap reason="name" extent="1 word"/> and i suspect there's one at <pause dur="0.2"/> <gap reason="name" extent="1 word"/> as well i'm not sure <pause dur="0.6"/> # </u><u who="nm0301" trans="overlap"> # there isn't any more </u><pause dur="0.3"/> <u who="nm0296" trans="pause"> <trunc>th</trunc> there isn't </u><u who="nm0301" trans="latching"> no </u><pause dur="0.4"/> <u who="nm0296" trans="pause"> but but within the <gap reason="name" extent="2 words"/> there are <pause dur="0.3"/> there are the three <pause dur="0.4"/> and there's an

opportunity if you're a G-P registrar to go on and do an extra six or twelve months working in <pause dur="0.3"/> at a university based at # <pause dur="0.3"/> a primary care <pause dur="0.2"/> central general practice centre <pause dur="0.4"/> and being supervised to undertake such research and it's really great that <gap reason="name" extent="1 word"/> <pause dur="0.5"/> is going on from strength to strength and is now <pause dur="0.3"/> taking his skills forward now as a <pause dur="0.4"/> as a lecturer in communication skills at <gap reason="name" extent="1 word"/> <pause dur="0.9"/> # <pause dur="0.3"/> we've created <pause dur="0.2"/> long breaks today <pause dur="0.3"/> # particularly at coffee time and lunchtime because <pause dur="0.2"/> it's a wonderful opportunity to share ideas <pause dur="0.4"/> and experiences and hopefully the talks will generate a lot of discussion <pause dur="0.5"/> and <pause dur="0.2"/> one of the really good things about today is that we've brought together colleagues not just from <gap reason="name" extent="1 word"/> <pause dur="0.5"/> but # also <pause dur="0.3"/> <gap reason="name" extent="1 word"/> and and <gap reason="name" extent="1 word"/> and we're you know obviously very grateful that you <pause dur="0.5"/> that you have come today