Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Aug 5:9:941036.
doi: 10.3389/fvets.2022.941036. eCollection 2022.

Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies

Dora Praczko  1 , Amanda K Tinkle  1 , Crystal R Arkenberg  1 , Robyn L McClelland  2 , Kate E Creevy  1 , M Katherine Tolbert  1 , Brian G Barnett  1 , Lucy Chou  1 , Jeremy Evans  1 , Kellyn E McNulty  1 ; Dog Aging Project Consortium; Jonathan M Levine  1
Affiliations

Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies

Dora Praczko et al. Front Vet Sci. .

Abstract

Here we describe the development and evaluation of a survey instrument to assess the research suitability of veterinary electronic medical records (EMRs) through the conduct of two studies as part of the Dog Aging Project (DAP). In study 1, four reviewers used the instrument to score a total of 218 records in an overlapping matrix of pairs to assess inter-rater agreement with respect to appropriate format (qualification), identification match (verification), and record quality. Based upon the moderate inter-rater agreement with respect to verification and the relatively large number of records that were incorrectly rejected the instrument was modified and more specific instructions were provided. In study 2, a modified instrument was again completed by four reviewers to score 100 different EMRs. The survey scores were compared to a gold standard of board-certified specialist review to determine receiver operating curve statistics. The refined survey had substantial inter-rater agreement across most qualification and verification questions. The cut-off value identified had a sensitivity of 95 and 96% (by reviewer 1 and reviewer 2, respectively) and a specificity of 82% and 91% (by reviewer 1 and reviewer 2, respectively) to predict gold standard acceptance or rejection of the record. Using just qualification and verification questions within the instrument (as opposed to full scoring) minimally impacted sensitivity and specificity and resulted in substantial time savings in the review process.

Keywords: clinical trial; electronic medical record; inter-rater agreement; point score; verification.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Scatterplot of 100 medical record scores between generated by 4 individuals grouped into reviewer 1 and reviewer 2 categories in study 2. Unverified records are assigned a zero score.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Bartlett PC, Van Buren JW, Neterer M, Zhou C. Disease surveillance and referral bias in the veterinary medical database. Prev Vet Med. (2010) 94:264–71. 10.1016/j.prevetmed.2010.01.007 - DOI - PubMed
    1. Paynter AN, Dunbar MD, Creevy KE, Ruple A. Veterinary big data: when data goes to the dogs. Animals. (2021) 11:1872. 10.3390/ani11071872 - DOI - PMC - PubMed
    1. McGreevy P, Thomson P, Dhand NK, Raubenheimer D, Masters S, Mansfield CS, et al. . VetCompass Australia: a national big data collection system for veterinary science. Animals. (2017) 7:74. 10.3390/ani7100074 - DOI - PMC - PubMed
    1. Hur B, Hardefeldt LY, Verspoor K, Baldwin T, Gilkerson JR. Using natural language processing and VetCompass to understand antimicrobial usage patterns in Australia. Aust Vet J. (2019) 97:298–300. 10.1111/avj.12836 - DOI - PubMed
    1. Jones-Diette JS, Dean RS, Cobb M, Brennan ML. Validation of text-mining and content analysis techniques using data collected from veterinary practice management software systems in the UK. Prev Vet Med. (2019) 167:61–7. 10.1016/j.prevetmed.2019.02.015 - DOI - PubMed

LinkOut - more resources