Skip to content. | Skip to navigation

Personal tools

DebugIT

Daniel Schober et al, (submitted).


Supplemental Material

for DebugIT SWAT4LS paper – D. Schober

The section numbering on this page recapitulates the paper section numbering for eased referencing. Sections without numbering are not referenced in the paper, but are supplemental as well.



2.1 Deriving Requirements for an EU wide resistance surveillance system

The U.S. Center for Disease Control and Prevention (CDC) defines ‘surveillance’ as ‘the ongoing systematic collection, analysis, and interpretation of health data essential to the planning, implementation, and evaluation of public health practice, closely integrated with the timely dissemination of these data to those who need to know’ [1]. This definition however is quite broad, resulting in wide interpretation spectra, and ultimately in the difficulty to classify and compare surveillance systems. For example, ‘ongoing’ can mean ‘daily’ or ‘once every 5 years’. The expression ‘systematic collection’ can refer to automatic or manual data gathering, or to the use of an agreed informal terminology or formal data exchange standard. The term ‘tool’ can refer to informal guidelines which have to be enforced and checked manually, to simple form-based semiformal approaches (e.g. CDC Active Bacterial Core surveillance, CDC-ABCs1), or to fully automated software pipelines. Consequently, we find the existing surveillance efforts difficult to compare. Nevertheless, there have been attempts to review resistance monitoring and surveillance programs for the veterinary [2], as well as for the human medical field [3]. An own review of the literature on this topic highlighted the following drawbacks of existing surveillance systems:

The Paul Ehrlich Gesellschaft (PEG) surveillance effort [4] delivers resistance data from <30 clinical sites in a 3 year interval for German speaking countries, so its time resolution is low and it is locally restricted, not covering the EU.

The European Antimicrobial Resistance Surveillance System (EARS-NET) [5] is allowing for European resistance comparisons, but is limited to specific clinical isolates, i.e. blood culture samples and gathers data on a yearly basis.

  • The German Network for Antimicrobial Resistance Surveillance (GENARS) [6] allows for a better timely resolution, but is considering only six German clinics.

  • The German Krankenhaus-Infektions-Surveillance-System (KISS) [7] looks only at intensive care units and at a reduced set of particular pathogens.

  • The SARI system [8] is again locally restricted to Germany and has a limited low resolution time frame.

To summarize, the most prevalent factors currently hampering EU wide resistance surveillance are

a) Limited access to local data-sets, although international cooperation is required [9]. The problem of data security issues and anonymisation may be a contributing factor.

b) Limited data coverage and granularity, resulting in differences in surveyed data and incomparable data sets.

c) Limited timely availability and coarse time resolution, although real-time surveillance is necessary [10].

d) Limited data dissemination and re-use, due to lack of standardization of clinical data syntax and semantics.

e) Limited automatisation, where data has to be gathered and compared manually. This is a consequence of the above limitations.

3.1 Terminology based instance conversion

As Fig 1 shows, we apply two formalization steps, from database values (I) in the Clinical Information System (CIS) to DDO instances (II), and from DDO instances to DCO/OO instances [III]:

Fig. 1. Instance mappings. The two-step formalization and normalization of a local CIS data entry for the case of a ‘bacterial pneumonia’ diagnosis is shown. Step 1 (left) exploits D2R mappings and term normalization, while step 2 (right) exploits terminology to DCO mappings via SKOS, or DDOtoDCO N3 mapping rules. On step 1, a SKOS mapping from ICD-10 to the DDO class diagnosis is achieved. In step 2, this DDO Instance is converted into a DCO instance by exploiting a consecutive SKOS mapping from ICD10 to SnomedCT, and from SnomedCT to DCO. The color and numbering scheme is applied in the exemplified N3 mapping section below.

Exemplarily, the transition from the ICD10 code “J15.9” in the local CDR to the global “DCO:BacterialPneumonia” class is shown in N3 notation:

1) DDO Instance:

a ddo:Diagnosis; skos:notation "J15.9"^^:icd10DT.

A data value in the DB e.g. icd10 code 'J15.9' is converted by D2R into an 'icd10' datatyped literal (via skos:notation).

2) ICD10-to-SnomedCT mapping file

skos:notation "J15.9"^^clisko:icd10DT] skos:exactMatch [skos:notation "53084003"^^clisko:sct20080731DT.

3) DCO Class (mapped to SnomedCT)

dco:BacterialPneumoniaProcess skos:exactMatch

skos:notation "53084003"^^clisko:sct20080731DT.

4) DCO Instance

a dco:Diagnosis; biotop:encodes :dco:BacterialPneumoniaProcess.


4.2 Requirements for setting up the DebugIT SIP at a local site

The advantage of our approach is that it is scalable. New sites can be integrated into the DebugIT query pipeline with considerable low effort: We will briefly illustrate what a new site needs to do to participate in the DebugIT surveillance network. To connect a new local CIS to DebugIT for data mediation, a lexical normalization of the local database values has to be carried out, e.g. via provided Morphosaurus webservices. An RDF server featuring a SPARQL endpoint needs to be set up. In case the local source is a relational database, D2R can be used to sparqlize the local database and a database-to-RDF conversion mapping table needs then to be specified to publish the data for SPARQL/semantic web access. Anonymisation has to be applied at this step. A local DDO that matches the local sites RDF CIS schema needs to be created from a provided DDO template. Eventually the DDO-to-DO N3 mapping rules need to be expanded to reflect new data to be integrated into the vCDR.

4.2.1. Generating an exemplary CASQ query

The Debug IT semantic integration platform has been evaluated against the set of competency questions and responded to all competency questions successfully. Here, we provide an exemplary view on the semantic integration process for a selected competency question and corresponding KART user query template. All intermediate formal artefacts are shown and explained.

To generate a new ‘customized’ CASQ query the following steps need to be carried out in which the query is successively formalized:

Step 1: The clinician states his clinical analysis question (CASQ) in natural language, e.g. “What percentage of Escherichia coli cases, cultured from urine samples, is resistant to the combination of trimethoprim/sulfametoxazol (TMP/SMX) or trimethoprim in the period 2006-2010 ?

Step 2: A clinical researcher formalizes the CASQ using the DOs (DCO & OO). This step is optional and serves to narrow down unintended interpretations through logically more rigid ontological formalization.

Step 3: Local data miners formalize the corresponding DSSQ for each targeted CDR using the local DDOs.

The corresponding local DSSQ, using the site-specific DDO expressions, looks as follows:

CONSTRUCT {
?percentage
quex:percentageOf ?total;
quex:percentageThat ?part;
quex:hasValue ?percentageValue; quex:hasUnit units:percent.
?total rdfs:subClassOf cao:EColi,[
a owl:Restriction; owl:onProperty cao:culturedFrom; owl:someValuesFrom [
rdfs:subClassOf dco:UrineSample;
a owl:Restriction; owl:onProperty biotop:outcomeOf; owl:someValuesFrom [
rdfs:subClassOf dco:UrineSampleCollection;
a owl:Restriction; owl:onProperty event:during; owl:hasValue [
dco:hasStartDateTime"2006-01-01T00:00:00" ^^xsd:dataTime;
dco:hasEndDateTime"2010-12-31T23:59:59" ^^xsd:dataTime]]]].
?part rdfs:subClassOf ?total, [
a owl:Restriction; owl:onProperty cao:resistantTo; owl:someValuesFrom [
owl:unionOf (dco:Trimethoprim dco:SulfamethoxazoleAndTrimethoprim)]]}
WHERE { same as in construct clause }

Step 4: A data miner aggregates the DSSQ result graph using the Euler Eye reasoner on the local-to-global rule sets. He then performs the clinical analysis via the CASQ and adds the results to the KART knowledge repository. The resulting globally integrated, site independent CASQ, using the DOs, looks as follows:

CONSTRUCT {
?percentage
quex:percentageOf ?total;
quex:percentageThat ?part;
quex:hasValue ?percentageValue; quex:hasUnit units:percent.
?total rdfs:subClassOf cao:EColi,[
a owl:Restriction; owl:onProperty cao:culturedFrom; owl:someValuesFrom [
rdfs:subClassOf dco:UrineSample;
a owl:Restriction; owl:onProperty biotop:outcomeOf; owl:someValuesFrom [
rdfs:subClassOf dco:UrineSampleCollection;
a owl:Restriction; owl:onProperty event:during; owl:hasValue [
dco:hasStartDateTime"2006-01-01T00:00:00" ^^xsd:dataTime;
dco:hasEndDateTime"2010-12-31T23:59:59" ^^xsd:dataTime]]]].
?part rdfs:subClassOf ?total, [
a owl:Restriction; owl:onProperty cao:resistantTo; owl:someValuesFrom [
owl:unionOf (dco:Trimethoprim dco:SulfamethoxazoleAndTrimethoprim)]]}
WHERE { same as in construct clause }


Step5:
External semantic web tools can now be applied for secondary data usage on the formalized integrated data, e.g. the ARTEMIS monitor or the Pan-European resistance monitoring dashboard, displaying selected query results as freely configurable diagrams.

Step 6: Clinical researcher and clinicians can then validate the CASQ result. If the result is not satisfactory a new cycle starts for answering an adapted version of the original CASQ.

5.2 Validating Results

The application of ‘competency questions’ (CQs [11]) approved by a clinical advisory board ensured the ontologies ‘fitness for purpose’. DCOs ‘content correctness’ has been ensured via the application of consistency checks by coherent and description logic reasoning.

Clinical validation is ensured by a) a clinical advisory board (CAB) that steers the overall goals and selection of clinical important research questions to be answered, and b) by microbiologists from the application sites that verify the usability of the tools in terms of access pertinence, GUI weaknesses and result presentation. The microbiologist also verifies result correctness and importance of those results with regard to their clinical impact.

An exemplary result comparison to existing surveillance efforts, - the German PEG and the French InVS surveillance studies, has shown evidence for good alignment of the detected resistance trends [20]: An initial pre-evaluation on the provided results was carried out, by comparing the results of three frequent queries between the DebugIT ARTEMIS tool and local as well as external monitoring efforts. To analyze result reliability compared with the original local data, the DebugIT result3 was compared to results obtained for an equal query on a selected local microbiology database at HEGP. The overlaid result graphs during a period from 2001 to 2007 show sufficient resemblance to support the applied mappings and the resulting outcome quality of the DebugIT monitoring tool (Fig. 2).

Fig. 2. Comparison of DebugIT vs. HEGP resistance trends for a period between 2001 and 2007.

To compare the DebugIT results with an external (collated and integrated) monitoring result, we selected the Paul Ehrlich Society (PEG) Monitoring Program, which gathers data manually. For a query on Staphylococcus aureus resistance towards Oxacillin, the DebugIT ARTEMIS monitor showed a 12.5 % resistance in UKLFR data. In comparison, the PEG monitoring showed a 20 % resistance. Discussions with the UKLRF Microbiology department revealed that the DebugIT value is locally more correct, as resistance values actually vary considerably within Germany.



5.2 Impact assessment for surveillance approaches

The impact of IT systems such as DebugIT on patient outcomes is inherently difficult to measure. Although impact assessments for more general health information systems are available [17] and confirmed2, clinical impact assessments of antibiotics surveillance systems are scarce [18]. Whereas for the first, relevant impact measures, like changes in clinical processes, are well understood, the impact measures for surveillance systems are much more indirect and multifactorial. This is due to the multitude of ways in which surveillance data is consumed and processed to guide workflow redesigns. In addition, in the mentioned review only 3 out of 23 included information systems deal with anti-infective drug support by means of decision support and these, again, measure easy-to-detect procedural changes rather than patient outcomes. Only one systematic review [35] tackling IT systems for drug monitoring was found in Medline, but none of its seven included studies tackled antibiotics monitoring. Within general drug monitoring impact assessment only resistance monitoring displays the unique characteristic of being rather complex. This is due to itself being a self-reflexive, higher-order analytical system.

Another paper [19] reviews therapeutic monitoring of antibiotics in the US, but was not yet included in review [35]. Although it exclusively limited itself to one single antibiotic (Vancomycin), some issues can be generalized and learned from this effort, e.g. ensuring that the recommendations with strong evidence - such as monitoring parameters as well as timing and frequency of monitoring- are included into a future DebugIT-based antibiotics monitoring system.

Ethical issues and data privacy

The integration of the clinical data is compliant with the EU statement on data protection in Electronic Health Records (EHR), retaining the data in situ. As data integration is done on the fly, it is avoided to store data outside the clinical setting, alleviating data security issues. Patient de-identification, pseudo-identifier and longitudinal linkage management is implemented at the CIS-CDR transition stage. The European Data Protection Directive (officially Directive 95/46/EC) on the protection of individuals has been accounted for. The local ethical boards on each data-delivering site approved the conditions of the usage of local data.

Next Steps and outlook

As a next step, the set of DebugIT queryable hospital data should be compared to the European standard protocol for surveillance of health-care-associated infections [21]. Once aligned in coverage, the standardization aspect of the DebugIT data integration effort could contribute to the ECDCs HELICS Surveillance program [15], i.e. by underpinning data as required in the Healthcare-associated Infections Surveillance Network (HAI-Net)4, but also to allow for a more detailed/granular statistical data analysis, e.g. as carried out in the Burden Study [16].

Although DebugIT should still be seen as a prototype, initial validation has shown that incorporating DebugIT tools into CIS can be advantageous, i.e. serving case- and population-based clinical data analysis as well as real-time hospital monitoring applications, especially surveilling antibiotics resistance. By increasing domain granularity and time resolution, we hope to detect seasonal variations in antibiotics usage for particular diseases, e.g. antibiotics against cold/respiratory tract infection which are prescribed more often in winter, indicating bad prescription practice as medical evidence here is poor [13]. Mutation mechanisms, lateral and horizontal resistance gene-transfer have been neglected as well. The content of the knowledge base needs to be expanded, e.g. studies suggest that agricultural misuse of antibiotics and the increase of antibiotic-resistant human infections are causally linked: Preventively given sub-therapeutic administration of antibiotics to live-stock animals fosters dissemination of antibiotic-resistant bacteria and their intrusion into the food chain [2].

References

  1. Thacker S, Berkelman R. Public health surveillance in the United States. Epidemiology Review 1988; 10: 164–190.

  2. Silley P, de Jong A, Simjee S, Thomas V, Harmonisation of resistance monitoring programmes in veterinary medicine: an urgent need in the EU?. Int J Antimicrob Agents. 2011 Jun;37(6):504-12. Epub 2011 Feb 3.

  3. Giskea, CG, Cornagliab, G, et al., Supranational surveillance of antimicrobial resistance: The legacy of the last decade and proposals for the future, ESCMID Study Group on Antimicrobial Resistance Surveillance (ESGARS), doi:10.1016/j.drup.2010.08.002

  4. PEG-Studie-2007, http://media.econtext.de/v1/stream/16-337/c9f96a352b9085300b185c09105d9c71/1363860367/16/337.econtext , last accessed 01.03.2014

  5. European Antimicrobial Resistance Surveillance System (EARSS), http://www.hpsc.ie/hpsc/A-Z/MicrobiologyAntimicrobialResistance/EuropeanAntimicrobialResistanceSurveillanceSystemEARSS/, last accessed 13.03.2014

  6. GENARS - German Network for Antimicrobial Resistance Surveillance, http://www.genars.de, last accessed 27.03.2014

  7. ITS-KISS, http://www.nrz-hygiene.de/surveillance/kiss/its-kiss/, last accessed 27.03.2014

  8. SARI Germany, http://sari.ipse-freiburg.de/pages/home.php, last accessed 06.01.2012 (now dead, look at http://www.p-e-g.org/archiv_tmp/bad_honnef_symposium_2006/Meyer.pdf instead)

  9. Hulscher ME, van der Meer JW, Grol RP, Antibiotic use: how to improve it? Int J Med Microbiol. 2010 Aug;300(6):351-6.

  10. Rempel OR, Laupland KB., Surveillance for antimicrobial resistant organisms: potential sources and magnitude of bias. Epidemiol Infect. Review, 2009 Dec;137(12):1665-73. Epub 2009 Jun 4PMID: 19493372

  11. Gruninger M, Fox MS. The role of competency questions in enterprise engineering. Proceedings of the IFIP WG5. 1994. S. 212–21. http://www.eil.utoronto.ca/enterprise-modelling/papers/benchIFIP94.pdf

  12. Blobel B, Kalra D, Koehn M, Lunn K, Pharow P, Ruotsalainen P, et al. The role of ontologies for sustainable, semantically interoperable and trustworthy EHR solutions. MIE. 2009. Sarajevo, S. 953–7.

  13. Coenen S, Ferech M, Haaijer-Ruskamp FM, Butler CC, Vander Stichele RH, Verheij TJ, Monnet DL, Little P, Goossens H; European Surveillance of Antimicrobial Consumption (ESAC): quality indicators for outpatient antibiotic use in Europe. ESAC Project Group. Qual Saf Health Care. 2007 Dec;16(6):440-5.PMID: 18055888

  14. Roe MT, Pillai SD. Poult, Monitoring and identifying antibiotic resistance mechanisms in bacteria. Sci. 2003 Apr;82(4):622-6. Review. PMID: 12710483

  15. Hospitals in Europe Link for Infection Control through Surveillance (HELICS) and Improving Patient Safety in Europe (IPSE). http://www.ecdc.europa.eu/en/activities/surveillance/hai/documents/0811_ipse_technical_implementation_report.pdf, last accessed 13.03.2014

  16. Lambert ML, Suetens C, Savey A, Palomar M, Hiesmayr M, Morales I, Agodi A, Frank U, Mertens K, Schumacher M, Wolkewitz M, Clinical outcomes of health-care-associated infections and antimicrobial resistance in patients admitted to European intensive-care units: a cohort study, The Lancet Infectious Diseases, Volume 11, Issue 1, p.30-38, January 2011, doi:10.1016/S1473-3099(10)70258-9

  17. Jamal A, McKenzie K, Clark M, The impact of health information technology on the quality of medical and health care: a systematic review. HIM J. 2009;38(3):26-37. Review.PMID: 19875852

  18. Hayward GL, Parnes AJ, Simon SR, Using health information technology to improve drug monitoring: a systematic review. Pharmacoepidemiol Drug Saf. 2009 Dec;18(12):1232-7. Review. PMID: 19725020

  19. Rybak MJ, Lomaestro BM, Rotschafer JC, Moellering RC Jr, Craig WA, Billeter M, Dalovisio JR, Levine DP., Therapeutic monitoring of vancomycin in adults summary of consensus recommendations from the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, and the Society of Infectious Diseases Pharmacists. Pharmacotherapy. 2009 Nov;29(11):1275-9. Review. PMID: 19873687

  20. Daniel C. et al., Comparing the DebugIT dashboards to national surveillance systems, BMC Proceedings 2011, 5(Suppl 6):O1 http://www.biomedcentral.com/1753-6561/5/S6/O1

21. Suetens C, Savey A, Labeeuw J, Morales I. The ICU-HELICS programme: Towards European surveillance of hospital-acquired infections in intensive care units. Euro Surveill.
2002;7(9):pii=359. Available online: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=359

1 http://www.cdc.gov/abcs/index.html, last accessed 03.03.14

2 I.e. Computerized Provider Order-Entry or Decision Support via analysis of adherence to evidence-based guidelines

3 for a query on the resistance of E. Coli towards the antibiotics Trimethoprime and Cefixime