Summary of 2014-2015 meetings

Date

Book (and chapters)/Journal

Presented/ led by

Oct - Dec 2014

Collett, D. (2003). Modelling survival data in medical research (2nd ed). Chapman & Hall/CRC.

The book by David Collett begins with a basic introduction to survival analysis and a description of four studies involving time-to-event data. These studies are then used to illustrate the techniques presented in the following chapters and thereby making it easy for an applied researcher to understand the methods and apply them in their daily work. Following the general introduction Collett describes all standard survival analysis models in great clarity.

Technical details are provided but only to the amount required by researchers wanting to apply the methods in their daily work. Following the description of standard methods in chapters 1-6 Collett introduces more advance techniques in chapters 7-11 (chapter 10 being an exception). It was felt that these later chapters were more of an introduction to the covered topics and statisticians who want to use analyses described here would probably need to do additional reading. This is aggravated by the fact that these techniques were the subject of ongoing research when the book was written 10 years ago whilst the techniques of chapters 1 to 6 were already well established.

In summary we felt that the book is a great reference for survival analysis and therefore a welcome addition to any medical statistician’s shelf. We felt that chapters 3 and 4 are particularly useful to statistician and clinicians as the author described with clarity and detail about fitting and checking of models. Statisticians interested in the theoretical background and proofs of survival analysis techniques might prefer to look for an alternative text.

14 Oct

Ch 2. Some non-parametric procedures

Ch 3. Modelling survival data

Ch 4. Model checking in the Cox regression model

12 Nov

Ch 5. Parametric proportional hazard models

Ch 6. Accelerated failure time and other parametric models

Ch 7. Model checking in parametric models

25 Nov

Ch 8. Time-dependent variables

Ch 9. Interval-censored survival data

Ch 10. Sample size requirements for a survival study

20 Jan 2015

Meeting was based upon articles in the Special Issue of Patient-reported Outcomes in Statistical Methods in Medical Research, October 2014; 23 (5).

Firstly, we discussed the editorial by Kammerman and Grosser, which gives an overview of issues arising from the use of patient reported outcome measures. Many of these issues were familiar to us as everyday issues in designing and analysing clinical studies, but some were new (e.g. issues arising from psychometric models) or had further issues specific to using outcome measures (e.g. missing data). We discussed the need to define concepts and terms clearly to avoid confusion, as differences in terminology (e.g. “PROs” vs “PROMS”) and definitions of concepts are common between papers.

We then moved on to discuss the article Interpretation of patient-reported outcomes by Cappelleri and Bushmakin. This paper discusses methods from three schools of interpreting scores and changes in scores of PROs: 1) Anchor based methods require the definition of a measure of criterion related to the instrument. A strength of these methods is that a choice of clinically meaningful anchors allows clinically meaningful information to be embedded into the instrument. 2) Distribution based methods are based purely on statistical theory. One method - responder analysis - is a key technique as it is required by the US FDA to gain approval for any PROs used in medical licensing studies. However, a limitation of all distribution based methods is that they cannot include clinical information. 3) Mediation analysis is relatively new as a tool for analysing outcome methods, but provides a method for capturing indirect effects of clinical interventions.

This special issue raises many important issues in the use of patient reported outcomes in clinical studies and provides a good general introduction to the field.

- Helen Parsons

18 Feb 2015

This meeting has been postponed and will be rescheduled in June

Mar - May 2015

Pepe, M. (2004). The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press

The book was written at a time where further development of statistical methods in diagnostic research was needed and where diagnostic methods were behind those used for therapeutic and preventative research (which I would say was still the case). From the outset the author states that the book ‘should serve as both a reference for the statistician applying these techniques in practice and as a survey of the current state of statistical research for the academic interested in developing new methodology in this field’. Therefore, I would say that the book provides a snapshot of the statistical techniques used for diagnostic research before the more recent development of methodologies in this field.

The acceleration in the design and methodology of diagnostic studies over the last few years has been facilitated through the Cochrane Diagnostic Test Accuracy Group who are in the process of publishing a handbook for systematic reviews and meta-analysis. In addition, the development of the QUADAS-2 quality assessment tool and the STARD (checklist for reporting studies of diagnostic accuracy) have helped to improve the quality of diagnostic research. See remit of the Cochrane Diagnostic Group http://srdta.cochrane.org/

Perhaps related to this, the more recently established Cochrane Prognosis Methods Group have crossovers with the remit of diagnostic studies and are in the process of developing further methodological tools. These include the TRIPOD statement which is a set of reporting guidelines for both diagnostic and prognostic prediction models and the PROBAST tool which identifies the risk of bias for risk prediction studies.
http://prognosismethods.cochrane.org/scope-our-work

Personally I found the book quite hard to digest due to the conflicting terminology used and feel the book is definitely in need of an update due to the expansive development in the field. However, I would also say that a lot of papers/texts just consider a dichotomous outcome but this book considers a range of scenarios and methods to analyse data on binary, ordinal and continuous test data which is a key strength of the book. The book also offers practical solutions to the common problems faced in diagnostic research such as imperfect reference standards, incomplete data and sources of bias and devotes much attention to the infamous ROC curve and analysing different thresholds. Finally, there are very few texts which go into detail about statistical techniques like this one – there is a book called ’Statistical Methods in Diagnostic Medicine’ by Xiao-Hua Zhou (2011) which considers similar topics but in further detail.

The Evidence Base of Clinical Diagnosis – Theory and Methods of Diagnostic Research by Knottnerus and Buntinx 2009 (BMJ book) considers more modern methodologies and gives practical guidance for carrying out diagnostic studies. I have attached a pdf of this book.

- Jenny Cooper

31 Mar

Ch 2. Measures of accuracy for binary tests

Ch 4. The receiver operating characteristic curve

Ch 5. Estimating the ROC curve

21 Apr

Ch 3. Comparing binary tests and regression analysis

Ch 6. Covariate effects on continuous and ordinal tests

26 May

Ch 7. Incomplete data and imperfect reference tests

Ch 8. Study design and hypothesis testing

Ch 9. More topics and conclusions

16 Jun 2015

Meeting was based upon two articles:

Meade, AW and Craig, SB. (2012). Identifying careless responses in survey data. Psychological Methods, 17:437-455.

Huang, JL, Curran, PG, Keeney, J, Poposki, EM and DeShon, RP. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27:99-114.

These two articles are the only few literatures with a comprehensive review of insufficiency effort responding (IER). They also reported empirical studies designed and conducted by them to detect IER and to minimize the phenomenon. The term IER was coined by Huang et al. as an umbrella term to depict the occurrence whereby participants are unmotivated to complete a questionnaire/survey as instructed. It was estimated that 10-15% of responses are considered as IER, a similar percentage as the occurrence of missing data.

Some issues and more questions arise from discussing these two articles:

• What are the possible actions when IER is detected? Should the data be ignored and/or imputed? The authors have suggested in inserting an item in each of the questionnaire to detect IER. However, if IER only occurs in a single page, are the suggested actions applied to responses only from that page or all responses from that individual?
• Investigators may a priori expect or hypothesize the direction of certain responses but how can we differentiate an otherwise responses as genuine or as a result of IER?
• Participants in a clinical trial will have different attitudes in responding to questionnaire as they are willingly taking part in a trial and so will be more motivated than, say, marketing survey participants. However, trial participants are inundated with an ever increasing number of questionnaires and this may cause respondent fatigue which leads to the question if the preventive and analysis methods described in these two articles are transferable to clinical settings. If yes, can these be done such that the reliability and validity of these established questionnaires are not compromised?

- Siew Wan Hee

14 July 2015

Meeting was based upon three articles:

Hedayat et al. (2015). Minimum clinically important difference in medical studies. Biometrics, 71:33-41.

Copay et al. (2007). Understanding the minimum clinically important difference: A review of concepts and methods. The Spine Journal, 7:541-546.

Dodick et al. (2015). Assessing clinically meaningful treatment effects in controlled trials: Cchronic migraine as an example. The Journal of Pain, 16:164-175.

The minimum clinically important difference (MCID) is considered important and a hot topic by many clinicians at the moment, yet very little material is published on the theoretical (statistical) justification of this concept. The discussion during the meeting focuessd on the paper by Copay et al which provides and overview over several approaches to determine MCID and on the paper by Hedayat et al which attempts to establish a theoretical foundation for MCID. The third paper was regarded as less useful and only mentioned briefly during the meeting.

Copay et al provide a general definition of MCID as the threshold value for a change that would be considered
meaningful and worthwhile by the patient such that he/she would consider repeating the intervention if it were
his/her choice to make again. Meeting attendees noted that this definition is not what they assumed the MCID to be (expectation was essentially that MCID is the same as, or similar to, the clinically significant difference used when designing studies) and that the definition is artificial in that patients wouldn’t repeat an intervention in most medical areas. Copay et al then describe anchor-based and distribution-based approaches to elicit MCID in their paper. They did acknowledge that all approaches have limitations. The Hedayat et al definition of MCID is somewhat different describing it as a large margin classification problem. They propose using machine learning techniques to estimate the MCID threshold value. Particular discussion points during the meeting were:

• The Copay et al methods are essentially looking at the proportion of patients who manage to get above the MCID threshold. This was considered inefficient because lots of information is omited and if lots of patients were just below the (somewhat arbitrary) threshold the new treatment would in many cases still be considered useful
  
• MCID uses simple questions like "do you feel better/same/worse than 3 months ago". The reason why researchers use complicated PROs is because such simple questions do not provide reliable answers.
• Attendees thought that MCID methods might make sense when you are conducting the first trial ever in an area. Otherwise it would usually be better to compare against other treatments (usual care) in the application area.

Overall meeting attendees came to the conclusion that MCID is a problematic technique which probably doesn’t answer the questions it attempts to answer. Attendees would recommend not including it in future studies (grant applications), however, given the current interest in this by clinicians in some fields this might prove challenging.

- Thomas Hamborg