HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Maruszewski, B. Articles by Kansy, A. Search for Related Content PubMed PubMed Citation Articles by Maruszewski, B. Articles by Kansy, A. Related Collections Congenital - acyanotic Congenital - cyanotic

Eur J Cardiothorac Surg 2005;28:400-404
© 2005 Elsevier Science NL

Original articles

An attempt at data verification in the EACTS Congenital Database

^a The Children's Memorial Health Institute, Warsaw, Poland
^b Denver Children's Hospital, University of Colorado, Denver, CO, USA
^c Southampton General Hospital, Southampton, UK
^d The Heart Hospital, Westmoreland Street, London, UK

Received 24 November 2004; received in revised form 15 February 2005; accepted 30 March 2005.

^* Corresponding author. Tel.: +48 22 815 73 46; fax: +48 22 815 73 50. (Email: bmar{at}ecdb.pl.pl).

Abstract

Objective: The multi-national and multi-institutional collection of data on outcomes in Congenital Heart Surgery (CHS) provides the possibility of analysis of results of treatment and may result in further improvement of the quality of care. The verification of data as far as the completeness and accuracy is necessary to give confidence to all sides—the patients, centers and regulatory authorities. The source data verification (SDV) although difficult, appears possible even in such a large-scale database. Methods: Out of 5.274 patients and 5.612 procedures data of 2003, collected in the database, 1.703 (32.3%) and 1.895 (33.8%), respectively, have been verified at five sites on following fields: IPPV time, date of birth, date of admission, date of surgery, date of discharge/mortality, body weight, case category, CPB time, AoX time, Circulatory arrest time. SDV was performed at five sites by two database officers using the sources of information different to the local copy of the database (patients' files, operation notes, perfusion charts, OR Books). Verification was performed between June 1st and July 31st 2004. Statistical analysis was performed using R-project software, ver. 2.0.0. and Welch's t-test for comparison of continuous variables. P-value >0.05 was used as statistically significant difference between groups. Results: Pre- and post-verification mortalities in all groups showed no significant differences although seven deaths out of 68 (10.27%) were missed. None of the other verified fields showed significant differences after verification. Conclusions: Source Data Verification showed no statistically significant differences between verified and non-verified data on 30 days mortality, LOS, age, body weight, CPB time, AoX and Circulatory arrest time. IPPV time was not available in 58.6% procedures.

Key Words: Congenital heart surgery • Quality of care • Outcome analysis • Database • Data verification • Data validation

1. Introduction

The continuous evaluation of the quality of care became one of the highest priorities in modern surgical practice [1]. Especially, in the field of Congenital Heart Surgery (CHS), where adverse outcomes can be frequent due to the wide spectrum of congenital cardiac conditions and severity of pathologies, as well as high proportion of palliations, it is important to know the exact information on mortalities and morbidities [2,3]. Even more important is the possibility of the comparison of outcomes between surgeons and institutions on large, international, multi-institutional scale [4–8].

In this spirit in early 1990s, a group of congenital; heart surgeons of the European Congenital Heart Surgeons Foundation (Association since 2003, ECHSA) decided to create the European Congenital Heart Defects Database [9]. This project was supervised by M. Elliott and based at the Great Ormond Street Hospital for Sick Children, London, UK. The data on 16.000 procedures had been collected until 1998, when this database was moved to Warsaw, Poland under responsibility of B. Maruszewski of the Children's Memorial Health Institute [9,10]. Since 2000, the European Association for Cardio-Thoracic Surgery (EACTS) has accepted this project and took substantially the financial responsibility for its further development together with ECHSA. At the same time, the simultaneous efforts were made at the Society of Thoracic Surgeons (STS) where the STS National Congenital Heart Surgery Database had been developed under the leadership of C. Mavroudis of the Children's Memorial Hospital, Chicago, USA [4–7,11,12].

These initial efforts provided the impetus for further cooperation between Congenital Heart Surgeons from both continents and initiated the International Congenital Heart Surgery Nomenclature and Database Project [11,12,13,15,16].

Its major aims were to establish a common International Nomenclature for CHS and the Minimum Data Set that have been used and updated continuously since year 2000 in both databases [11,13].

Until September 2004, the data on 23.735 CHS procedures have been collected, analyzed and reported online at the EACTS Congenital Database (www.eactscongenitaldb.org) (Table 1 ).

The meaning of ‘data validation’ should be understood as testing internal data consistency, e.g. the date of operation should not be earlier than the date of birth.

Data validation is possible at data entry (surgeon, software) and in the database before data analysis. The EACTS Congenital Database developers have provided the internal database validation mechanism that automatically excludes non-accurate data according to agreed criteria (www.eactscongenitaldb.org).

The next key step towards improvement of the data quality is the efficient data verification [17]. According to the Good Clinical Practice guidelines defined and recommended by World Health Organization (WHO), all database contents should be verified by comparison with the original records at the site of their origin—Source Data Verification (SDV) [17,18]. SDV is an evaluation of the conformity of database contents with the source data.

SDV, although very difficult on international, multi-institutional scale due to its costs, time consuming and lingual problems, appears to be possible and remains essential. For all reasons described above the EACTS Congenital Database management decided to make an attempt at data verification and check on the potential discrepancies between verified and non-verified data.

2. Material

Five European CHS Centers volunteered to participate in the project and undergo 2003 data verification at their sites. The verification was conducted between 1st of June and 31st of July 2004 and concerned the following fields:

• 30 days mortality

• LOS

• IPPV

and additionally:

• date of birth

• date of admission

• date of surgery

• date of discharge/mortality

• body weight

• Case category

• CPB time

• AoX time

• Circulatory arrest time

Out of 5.274 patients and 5.612 procedures data of the year 2003 collected in the database 1.703 (32.3%) and 1.895 (33.8%) have been verified, respectively (Table 2 ).

SDV has been conducted by EACTS Congenital Database staff (two officers).

The data verification process was organized in the following steps:

• 100% of patients data of 2003 have been supplied to the Database before site visit.

• The data underwent standard internal validation procedure.

• Person locally responsible for the verification indicated the sources of information at the site, different to the local copy of the database or related to the database which were as follows:

– patients' files

– hospital computerized system

– operation notes

– perfusion charts

– OR books

• The database staff visited the site and checked on the accuracy of 100% of 2003 records for the fields undergoing verification. The central database printouts have been compared with the local sources.

• The statistical analysis was performed on both sets of data (verified, non-verified) for comparison (Table 3 ).

Results were expressed as mean values. All analyses were performed with appropriate software (R-project, ver. 2.0.0). Comparisons of continuous variables were performed by Welch's t-test. P-value greater than 0.05 means that difference between groups is not statistically significant.

5. Results

Two patients, out of 1.705 after verification, have been missed.

There have been no statistically significant differences between pre- and post-verification mortalities in all verified groups of patients, nevertheless seven deaths were missed out of 68 deaths found after verification (10.27%) (Tables 4–7 ).

Though only two patients were not reported out of 1.705 (after verification), there have been bidirectional shifts between age groups due to the correction in dates of birth and/or of operation (Table 8 ).

6. Conclusions

Source Data Verification showed no statistically significant differences between verified and non-verified data on 30 days mortality, LOS, age, body weight, CPB time, AoX and Circulatory arrest time IPPV time was not available in 58.6% procedures.

7. Discussion

We believe that this study is the first one to present the verification process on the big, multi-institutional and international data registry of the outcomes of the CHS.

It is obvious that 30 days mortality measure carries important limitation as far as the assessment of even early survival [8]. Undoubtedly, modern intensive care technology extends critical patients survival time what does not necessarily mean increase in long-term survival and improvement of the quality of life. Nevertheless this has been the only objective measure available 6 months after study time frames (2003). In future works, 30 days mortality measure should be avoided and probably survival analysis should concentrate more on mid-term outcome (6 months or 1 year). This is hard to believe that such a data can be obtained completely and accurately, especially in international big scale registry.

Another limitation of the study is due to arbitrary selection of verified Centers. Relatively optimistic results of this first attempt at data verification, performed on the Centers being ‘good data submitters’, may not reflect the overall quality of data in the whole data collection. The only solution to decrease the negative impact of arbitrary choice of Centers will be using ‘by chance’ mechanism for the next 2005 attempt or include the Centers with relatively short experience in database works.

The methodology employed in this project was very demanding, since 100% of 2003 records for nine fields have been checked, which finally verified seven investigated parameters.

The verification process according Good Clinical Practice for the clinical trials, especially for the pharmaceutical products should cover 100% of data. This rule is very rarely applied to any verification on the big data collection, although some institutions have used it to verify their clinical data (e.g. CHOP, Pennsylvania). There are important differences between various verification protocols used for medical data sets [15]. The verified samples vary from as small as 10 up to 100%. It shows that an International Committee of experts is needed to define common Data Verification methodology and to apply it in future works on outcome analysis in CHS. Such a group has been recently established between the EACTS and STS. This is the work in progress and some feedback from this Committee is expected before the end of 2005.

Although this study is only an attempt at data verification within big, international, multi-centric database, it has proven the scientific value of such a data collection. The verification process will be continued and further works will focus on improvement of the quality of data. This should result in better understanding of the specific risk factors and areas of weakness in the quality of care in CHS.

Appendix A. Conference discussion

Dr J. Monro (Southampton, United Kingdom): I do congratulate you on all this work and the remarkable database that you've collected for all these years. The results of the verification in these five units are amazing. With the British experience in mind, it is incredible that they only missed two patients and those seven deaths. I wonder if this is because you chose the five best, most organized centers, who therefore did have these very accurate data. When you've looked into the others, I think you may find that it's not quite so good. But as you've said before, everything has got to be 100%, both centers and the data they're collecting.

Dr Maruszewski : And of course, you are right. We have taken the centers that offered us cooperation. And I have to use the opportunity to thank my colleagues from the five centers for their very hard work, because this is a lot of very hard work. In the future, we want to make the decision about the centers that we choose by chance, so machine will choose the centers and then we will see.

Mr V. Tsang (London, United Kingdom): The work you have done is very admirable, but can I talk about how to tackle a big problem in general terms? For example, your database is huge. And in psychological terms, when you have a huge problem people feel overwhelmed as to what to do next. I think the way to solve that sort of problem is to break the big problem into smaller blocks. The progress made in each of the smaller blocks, when added together, would become major progress.

Dr Maruszewski : Those of you who saw the outcome analysis that we were showing on various occasions, including yesterday at the business meeting, probably understand that is exactly what we are trying, we are attempting to do. So take part of the data and clear them and make sure they are very fine, and then to conclude on one question that you want to answer. That's what we are doing with neonates, Norwoods, influencing the overall outcomes.

Dr B. Williams (Toronto, Ontario, Canada): I have tremendous respect for what you've done with this database and the enormous amount of work that you put into it.

I'd like to comment on your findings that the time to extubation was much more difficult to find. That doesn't surprise me because it's data that's out of our domain. It's kept by other people in other areas. And my bias would be that we should not try to capture that information in a surgical database. Surely the intensivists have their own database and would it not be far more logical to create a link between your surgical registry and the database in the other areas of the hospital, specifically the intensive care unit.

If you carry this analogy one step further and look at the genetics, I think our data about genetics is going to be extremely poor and we will need to link to the genetics department to find out which patients have genetic disorders.

Dr Maruszewski : As you know, we are meeting beginning of October, CHSS and our association, in Montreal. One of the tasks is to make sure that both associations realize that IPPV time is a weak measure. And I think we should not carry on promoting this as the quality of care measure, because, first of all, we don't have this measure, we don't have the data, I agree.

Dr G. Ziemer (Tuebingen, Germany): Regarding ICU-data we have to keep in mind that ICU care is organized so differently across Europe and even within certain countries, that uniform data collection is not practical at the moment, although the idea I think is a great one.

Dr W. Mrowczynski (Poznan, Poland): Although the database is voluntary, the data verification may require agreement of the authorities governing, for example, cardiac surgery department. Don't you think it can be an obstacle in data verification?

Dr Maruszewski : It hasn't been for those who keep sending data for years; and this problem has been sorted out at the European union level and the same in States. The parents of the patient who is admitted to the hospital, on the admission accept that the data are used for the medical record and for the medical database. It may be different in various states and, for example, I'm not aware of that in details, but so far we are able, in the legal manner, to collect this data.

Dr F. Lacour-Gayet (Denver, Colorado, USA): I would like to congratulate you and your group in Warsaw for the incredible amount of work that produced for the last 10 years in finally building what is probably the best congenital heart surgery database in the world.

Let me give you some information from the STS database. We have also decided to verify data at the STS. It's a bit more complicated and there are some authorizations that are needed, but we are proceeding. We have decided on a protocol that is based essentially on comparing the data of the hospital registry to those of the department and check that there is no death missing. Ultimately, there is a need for a common protocol to verify data.

Dr Maruszewski : Before I finish, I would like to say just a few words. I would like to give my best thanks and a lot of respect to our Association for the support that has always been given to the database even in a very premature state when the outcome was not sure.

Then I would like to thank Bill Williams for his stimulation and motivation and criticism that we received some time ago and that motivated us to work.

I would like to thank all the colleagues in the European countries for putting the data in and the effort made on verification.

At the end, I would like to thank Dr Tobota from Warsaw who is running, in fact, this database. He is doing 90% of the work and I am just presenting. So thank you, Dr Tobota, very much.

Acknowledgments

We would like to acknowledge doctors Duccio Di Carlo, Mark Hazekamp, Victor Hraska, Giovanni Stellin, and all staff members of five verified hospitals for the support and understanding during demanding process of data verification.

Footnotes

Presented at the joint 18th Annual Meeting of the European Association for Cardio-thoracic Surgery and the 12th Annual Meeting of the European Society of Thoracic Surgeons, Leipzig, Germany, September 12–15, 2004.

References

1. Daenen W, Lacour-Gayet F, Aberg T, Comas JV, Daebritz SH, DiDonato R, Hamilton JR, Lindberg H, Maruszewski B, Monro J. EACTS congenital heart disease committee. Optimal structure of a congenital heart surgery department in Europe. Eur J Cardiothorac Surg 2003;24(3):343-351.[Free Full Text]
2. Williams WG, McCrindle BW. Practical experience with databases for congenital heart disease: a registry versus an academic database. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2002;5(1):132.[CrossRef][Medline]
3. Mavroudis C (Chairman) and Congenital Database Subcommittee: Backer CL, Bove E, Burke RP, Cameron D, Drinkwater DC, Edwards FH, Grover FL, Hammon JW Jr, Jacobs JP, Kron IL, Mayer JE, Myers JL, Ring WS, Siewers RD, Szarnicki RJ, Watson DC Jr. Data Analyses of the Society of Thoracic Surgeons National Congenital Cardiac Surgery Database, 1994–1997. Summit Medical, Minnetonka, MN; September 1998..
4. Mavroudis C, Gevitz M, Rings WS, McIntosh CL, Schwartz M. The society of thoracic surgeons national congenital heart surgery database: analysis of the first harvest. Ann Thorac Surg 1994-1997;68(2):601-624.
5. Jacobs JP, Mavroudis C, Jacobs ML, Lacour-Gayet FG, Tchervenkov CI, Gaynor JW, Clarke DR, Spray TL, Maruszewski B, Stellin G, Elliott MJ, Dokholyan RS, Peterson ED. Lessons learned from the data analysis of the second harvest (1998–2001) of the society of thoracic surgeons (STS) congenital heart surgery database. Eur J Cardiothorac Surg 2004;26(1):18-37.[Abstract/Free Full Text]
6. Jacobs JP, Jacobs ML, Mavroudis C, Lacour-Gayet FG. Executive summary: the society of thoracic surgeons congenital heart surgery database—second harvest—(1998–2001) beta site test. The Society of Thoracic Surgeons (STS) and Duke Clinical Research Institute (DCRI), Duke University Medical Center, Durham, North Carolina, United States; Fall 2002 Harvest..
7. Jacobs JP, Jacobs ML, Mavroudis C, Lacour-Gayet FG. Executive summary: the society of thoracic surgeons congenital heart surgery database—third harvest—(1998–2002). The Society of Thoracic Surgeons (STS) and Duke Clinical Research Institute (DCRI), Duke University Medical Center, Durham, North Carolina, United States; Spring 2003 Harvest..
8. Stark J, Gallivan S, Lovegrove J, Hamilton JRL, Monro JL, Pollock JCS, Watterson KG. Mortality rates after surgery for congenital heart defects in children and surgeon's performance. Lancet 2000;355(9208):948.[Medline]
9. Maruszewski B, Tobota Z. The European congenital heart defects surgery database experience: pediatric cardiothoracic surgical registry of the European association for cardio-thoracic surgery. Seminars in thoracic and cardiovascular surgery. Pediatr Cardiac Surg Annu 2002;5(1):143-147.
10. Jacobs JP. Software development, nomenclature schemes and mapping strategies and for an international pediatric cardiac surgery database system. Seminars in yhoracic and cardiovascular surgery. Pediatr Cardiac Surg Annu 2002;5(1):153-162.
11. Lacour-Gayet F, Maruszewski B, Mavroudis C, Jacobs JP, Elliot MJ. Presentation of the international nomenclature for congenital heart surgery. The long way from nomenclature to collection of validated data at the EACTS. Eur J Cardiothorac Surg 2000;18(128):35.
12. Mavroudis C, Jacobs JP. Congenital heart disease outcome analysis: methodology and rationale. J Thorac Cardiovasc Surg 2002;123(1):6-7.[Free Full Text]
13. Gaynor JW, Jacobs JP, Jacobs ML, Elliot MJ, Lacour-Gayet F, Tchervenkov CI, Maruszewski B, Mavroudis C. International congenital heart surgery nomenclature and database project. Congenital heart surgery committees of the society of thoracic surgeons. European association for cardio-thoracic surgery. Congenital heart surgery nomenclature and database project: update and proposed data harvest. Ann Thorac Surg 2002;73(3):101.
14. Maruszewski B, Lacour-Gayet F, Elliott MJ, Gaynor JW, Jacobs JP, Jacobs ML, Tchervenkov CI, Kurosawa H, Mavroudis C. International congenital heart surgery nomenclature and database project. Congenital heart surgery committees of the society of thoracic surgeons, European association for cardio-thoracic surgery. Congenital heart surgery nomenclature and database project: update and proposed data harvest. Eur J Cardiothorac Surg 2002;21(1).
15. Gibbs JL, Monro JL, Cunningham D, Rickards A. Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1. Br Med J 2004;328:611.[Abstract/Free Full Text]
16. Mavroudis C, Jacobs JP, editors. Congenital heart surgery nomenclature and data base project. Ann Thorac Surg. 2000; 69: S1–372..
17. Khosla R, Verma DD, Kapur A, Khosla S. Efficient source data verification. Indian Journal of Pharmacology 2000;32:180-186.
18. WHO guidelines for Good Clinical Practice for Trials on Pharmaceutical Products; 1994..

This article has been cited by other articles:

				J. P. Jacobs, R. J. Cerfolio, and R. M. Sade The Ethics of Transparency: Publication of Cardiothoracic Surgical Outcomes in the Lay Press. Ann. Thorac. Surg., March 1, 2009; 87(3): 679 - 686. [Full Text] [PDF]

				S. M. O'Brien, J. P. Jacobs, D. R. Clarke, B. Maruszewski, M. L. Jacobs, H. L. Walters III, C. I. Tchervenkov, K. F. Welke, Z. Tobota, G. Stellin, et al. Accuracy of the Aristotle Basic Complexity Score for Classifying the Mortality and Morbidity Potential of Congenital Heart Surgery Operations Ann. Thorac. Surg., December 1, 2007; 84(6): 2027 - 2037. [Abstract] [Full Text] [PDF]

				J. P. Jacobs, M. L. Jacobs, C. Mavroudis, B. Maruszewski, C. I. Tchervenkov, F. G. Lacour-Gayet, D. R. Clarke, T. Yeh Jr, H. L. Walters III, H. Kurosawa, et al. What is Operative Morbidity? Defining Complications in a Surgical Registry Database Ann. Thorac. Surg., October 1, 2007; 84(4): 1416 - 1421. [Full Text] [PDF]

				P. E. Falcoz, M. Conti, L. Brouchet, S. Chocron, M. Puyraveau, M. Mercier, J. P. Etievent, and M. Dahan The Thoracic Surgery Scoring System (Thoracoscore): Risk model for in-hospital death in 15,183 patients requiring thoracic surgery J. Thorac. Cardiovasc. Surg., February 1, 2007; 133(2): 325 - 332. [Abstract] [Full Text] [PDF]

				V. L. Vida, H. Berggren, W. J. Brawn, W. Daenen, D. Di Carlo, R. Di Donato, H. L. Lindberg, A. F. Corno, J. Fragata, M. J. Elliott, et al. Risk of Surgery for Congenital Heart Disease in the Adult: A Multicentered European Study Ann. Thorac. Surg., January 1, 2007; 83(1): 161 - 168. [Abstract] [Full Text] [PDF]

				M. Neuhauser, N. Lehmann, M. Nonnemacher, and J. Stausberg An attempt at data verification in the EACTS Congenital Database: data before and after verification differ significantly. Eur. J. Cardiothorac. Surg., October 1, 2006; 30(4): 691 - 691. [Full Text] [PDF]

				J. P. Jacobs, C. Mavroudis, M. L. Jacobs, B. Maruszewski, C. I. Tchervenkov, F. G. Lacour-Gayet, D. R. Clarke, T. Yeh Jr, H. L. Walters III, H. Kurosawa, et al. What is Operative Mortality? Defining Death in a Surgical Registry Database: A Report of the STS Congenital Database Taskforce and the Joint EACTS-STS Congenital Database Committee Ann. Thorac. Surg., May 1, 2006; 81(5): 1937 - 1941. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Maruszewski, B. Articles by Kansy, A. Search for Related Content PubMed PubMed Citation Articles by Maruszewski, B. Articles by Kansy, A. Related Collections Congenital - acyanotic Congenital - cyanotic