click to enlarge scorecard
In November 2013, The Agency for Healthcare Research and Quality (AHRQ), a division of the
US Department of Health & Human Services, released the online Patient Education Materials
Assessment Tool (PEMAT). The Health Mirror team has been using these criteria to standardize
our evaluation of videos prior to allowing in our library.
“The Patient Education Materials Assessment Tool (PEMAT) is a systematic method to
evaluate and compare the understandability and actionability of patient
Understandability: Patient education materials are understandable when
consumers of diverse backgrounds and varying levels of health literacy can
process and explain key messages.
Actionability: Patient education materials are actionable when consumers of
diverse backgrounds and varying levels of health literacy can identify what they
can do based on the information presented.”
As an example, our team score for the heart disease awareness video
by GoRedforWomen.org was 89% for understandability and 100% for actionability (see
figure 1). This positively reinforced our subjective good impression. The narrators implore patients
to realize that "1 in 3 women" die from heart disease, and to know their family history, blood
pressure, and cholesterol numbers. The audio is clear, the script has visual text cues, and
patients are given direct actions to take. Certainly, there is a subjective component to the
assessment, but we find that with regular use and more than one reviewer to check for bias,
there is no more than 10% variation in scoring.
Our goal is to allow videos with a usability score of over 70%. Videos can miss that mark by using
medical terms without explanation, or failing to add useful visual cues, an introduction or a
summary. Some videos are allowed despite failing to meet our ideal PEMAT score because they
are accurate, carry evidence based information and fill an educational need that is not met
otherwise. We also allow a percentage of videos that have no actionability because their sole
purpose is improving health literacy, for example “Understanding Aortic Aneurysm.”
Health Mirror has found the AHRQ/PEMAT to be useful in standardizing the evaluation of videos
in a systematic and objective fashion.