METRIQX
Work with Metriqx
Free tool

Check your own evidence.

Twelve questions, four for each layer of the framework. It takes about five minutes and tells you which layer your evidence is thinnest at. Everything is scored in your browser: no data leaves this page, and nothing is stored.

03Integration validityDoes it hold where it is used?02Derived output validityDoes the metric built on it hold?01Measurement validityDoes the signal reflect the thing?Each layer is only as good as the one beneath it.
Layer 01Measurement validity

Does the raw signal reflect the physiological quantity the device claims to capture.

  1. Can you name, in one sentence per output, the physiological quantity each sensor output is meant to represent?

  2. Has the raw output been compared against a recognised reference standard, rather than against another consumer device?

  3. Was that comparison run in the population and conditions the product is actually used in (skin tone, age, body composition, motion, environment)?

  4. Do you have the error, bias and limits of agreement of the raw measurement as numbers you could put in a table?

Layer 02Derived output validity

Do the metrics computed from that signal actually hold.

  1. Is the algorithm that turns the signal into your headline metric documented well enough that someone outside the team could reproduce it?

  2. Has the derived metric been validated against an accepted criterion measure, rather than shown to correlate with itself over time?

  3. Do you know how the metric behaves at the extremes, and in the people who differ most from your development sample?

  4. When the algorithm changes between releases, can you say what that change did to the metric's validity?

Layer 03Integration validity

Does it work inside real clinical or performance practice, with the people who rely on it.

  1. Is it written down, specifically, which decision each output is meant to inform, and who makes it?

  2. Has anyone watched an intended user read the output under realistic time pressure, rather than in a demo?

  3. Do you know what happens when the output is wrong: what the user does next, and who absorbs the consequence?

  4. Is a misleading measurement logged as a hazard in your risk file, or in an equivalent record if you sit outside healthcare?

0 of 12 answered. Unanswered questions count as no.

Where the framework comes from

Layers 01 and 02 are the V3 framework (verification and analytical validation), the field's standard reference for whether a biometric monitoring technology is fit for purpose. We did not invent it and we use it because it is right. Layer 03 is our extension: V3 asks whether a metric tracks a meaningful clinical state, and we push that further into whether it survives the decision and the workflow it is built for, in high-performance settings as well as clinical ones.

Goldsack et al. (2020), Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digital Medicine 3:55. doi:10.1038/s41746-020-0260-4 ↗

Found a layer you cannot answer for?

That is the useful result. Bring it to a call and we will tell you what closing it would actually take.

Book a call ↗See the engagements