Skip to Content
Healthcare Professionals

Guidance for Reporting Design Changes to AI

Criteria for “substantial” changes

Product changes should be considered substantial if the change may affect:

  • The conformity with the essential requirements and/or
  • The indications and/or contraindications and/or warnings determined by the manufacturer to be appropriate to ensure the clinical performance of the device.

When determining whether or not a particular product change is “substantial” following considerations should be made (the list is not exhaustive):

  • Changes of the intended purpose and/or the performance specification of the device?
  • Are new hazards introduced which have not previously been addressed?
  • Are risks associated with existing hazards affected?
  • Does the change alter the details on intended use given in the design/type approval dossier submitted to the MHRA?
  • Does the change trigger a need to alter the indications or contraindications for use or warnings necessary to ensure safety and efficacy for the intended use of the device?
  • Does the change mean that the device will have different end users or be used in a different manner?
  • Does the change mean that the clinical data/performance evaluation data for the original device is not sufficient to assure conformity of the changed device with the required characteristics and performance?
  • Is the change a direct result of actions taken related to concerns arising from Post Market Surveillance including incidents/recalls/complaints?

Changes to software

The following would be considered substantial changes:           

  • an alteration in software that modifies an algorithm impacting the diagnosis
  • a software change that impacts the way data is read or interpreted by the user, such that the diagnosis of the patient may be altered when compared to the previous version of the software.
  • a software change that replaces previously required user input to a closed loop decision.
  • addition of a new feature to the software that may change the diagnosis, or the therapy delivered to the patient.
  • a software change that incorporates a significant change to the operating system on which the software runs. If the software is modified to correct an error (for example, a change in algorithm), for which there is a safety risk to the patient if the error is not corrected, this software change may require an evaluation and approval by the MHRA. 

HRA

The kinds of things to consider are whether the changes indicate that the purpose of the review has now expanded into research e.g. because generalisable findings are being sought from the analysis of the data post-changes (i.e. falling outside of the original intended use of the AI app algorithm).

If the reason for altering the algorithm (e.g. sensitivity) goes beyond the clinical into e.g. generating a hypothesis, that would indicate that the changes are straying into research activity that should be notified on IRAS where NHS data are being used (and require REC if that data is identifiable of the patients).

This HRA toolkit, designed for students, but useable by all as a simple navigable PowerPoint to download, is a great roadmap to use to determine when and what type of approvals may be required from an HRA perspective: Student research toolkit - Health Research Authority.

Back to top