August 24, 2000

Hew,

At the Teratology Society meeting I promised to send you my thoughts on the MARTA/MTA Historical Control Data project. Here they are.

First of all, let me say that I have changed my views since we last spoke. After playing some more with the search functions of the system I realised that the database is a lot more useful than I thought. Your demonstration didn't really do it justice (because of the obvious technical difficulties in doing the presentation to such a large group by remote connection). Also, like many others present, I may have forgotten what Teratology is all about! The major function of the database is of course the compilation of fetal abnormality data, which it does better than ever, thanks to the standardised terminology.

Before getting down to specifics, we should reflect on how background control data (or "hysterical data", as I like to call them) are used in the interpretation of results from regulatory toxicology studies. In practice, the incidence data and numerical litter data are evaluated very differently. We usehistorical incidence data to help assess whether a given occurrence in a treated group is consistent with a chance event; the parameter may be group-based (eg. 3abortions in a given group) or litter-based (eg. 3 fetuses from 2 litters with spina bifida). In this case, we are trying to compensate for the inadequacy of the sample size in our concurrent control group for assessing low-frequency events. This is the strong point of the database (see below). When assessing
numerical litter data, historical values are most often used to check for abnormal responses in the concurrent control group. Results from treated groups are usually only compared with historical data when the concurrent controls are considered abnormal. Also, the confidence attributed to historical parameters depends largely on their variability in comparison with that of the concurrent control. The pertinence of historical data is greatest when faced with highly variable concurrent control data. Very heterogeneous historical data (eg. maternal body weight and food consumption) are of little interest.

All that we need from the database for the purposes of assessing the significance of low frequency events (such as malformations) is a good estimate of the background incidence in the population. The database answers this question in seconds, with the added advantage of calculations on a litter-, fetus- or group-basis. With this information, we can estimate the probability of a given scenario by assuming a Poisson distribution. The same is true for higher frequency events, for which a binomial distribution may be assumed.

In order to make use of historical data in assessing numerical litter values, estimates of both the central tendency and dispersion are needed. All that we can calculate from the database at present is the mean and range. We cannot even assess the variability of the data in order to evaluate their reliability. The median would be better as a measure of central tendency than the mean, since the underlying distribution may not normally distributed. Better still, a comparison of the mean and the median would allow some estimation of the symmetry of the distribution. For me, the range is next to useless, since by definition it is most influenced by the extreme values. In a way, we are using the most abnormal members of the population as our standard. Many statisticians would actually consider excluding these extreme values as outliers. I am not sure how this could be improved in the database without greatly complicating the data input. Interquartile or percentile ranges might be one solution. Otherwise, you could require the users to enter the SD for each parameter and then estimate the population variance from that (there goes my argument about non-normal distributions!); but that would probably be too much effort for everyone concerned.

I agreed at the time with the comments from the floor during your demo that it is a shame not to make use of data from preliminary studies, but now I realise that the inclusion of under-sized groups would invalidate some of your parameters by disproportionately influencing the calculations (such as max % affected fetuses per group).

Therefore, overall, I think that the database is a very worthy initiative, which I certainly intend to use a reference in my work. Unfortunately, our laboratory is not in a position to enter data. First, as we have discussed, our management is paranoid about data confidentiality (for no good reason). Second, almost all of our data are derived using unique strains from a French breeder (namely the OFA rat and INRA rabbit) that are practically not used elsewhere, so our data are little interest to other labs. In addition, our data acquisition system (Grosse) already compiles historical data without having to re-enter the values.

I hope that all of this is of interest to you. Please don't hesitate to contact me if you have any questions.

Paul Barrow