ISPOR updates its recommendations for eCOA comparability – what do you need to know?

RWS were thrilled to note the latest updates to the ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) guidance, which were put into effect recently (May, 2023). This brings a much needed update to a valuable evidence piece used frequently in the eCOA space.

 

Despite a multitude of important industry publications on this topic being produced over the last ten years, we’ve found ourselves continuously referencing the foundational recommendations from 2009 when discussing comparability between modes of COA administration. In particular, to the evidence table below (Fig 1), which has been ubiquitous in eCOA Best Practice publications of the last decade.

 

Although deeply respected, the industry as a whole has functioned with the knowledge that changing “circle” to “select” in PRO instructions is quite unlikely to impact data collection properties in recent years, and the level of evidence cited here was in need of an update.

The updated recommendations touch on a range of pressing topics, such as eCOA administration for pediatric and elderly populations, VAS scale length, scrolling, and BYOD. They also provide an excellent and usable list of “key works to date” and encourage the continued research and expansion of supporting literature.

 

Here are three key takeaways from our team:

 

This seems logical, and aligns with Muehlhausen et al’s 2018 synthesis 53 unpublished cognitive interview and usability test studies, along with other publications included in the key works. It will also cover the vast majority of electronically administered COAs, and eliminate the ambiguity surrounding many former “minor” and “moderate” levels of modification.

 

2. Usability Testing is not required for most minor or moderate levels of modification. 

This is a relief to many stakeholders since the 2009 levels of recommended evidence were increasingly unfeasible, and no longer felt appropriate to the changes. Examples of modifications that do not require additional evidence are: instructional text updates, format change from paper to single item format, and change in item presentation.

 

That said, Usability Testing is still a critical tool for gathering user feedback on an eCOA app. When working with novel response scale types, and with patients who suffer from certain accessibility issues, Usability Testing may still be an asset to the eCOA providers and sponsors. 

 

The 2009 guidance became difficult to apply in practical settings, since the technology and global equivalence evidence had outgrown the guidance. This is always a risk and a testament to the efficacy of the guidance in spotting these areas of concern, and to subsequent publications for proving them true. In the updated guidelines, we’re provided with a decision tree, and a call to continue this research and to produce bodies of comparability evidence.