Uncertainty and Weighted Averages (Statistics)

flapjack

Hi all, it's been a while since I've run one of these and I just want to get some verification that I'm approaching this the right way before I finalize and move on.

I have measurements of a quantity (specifically, a quantity in a beaker); three independent trials with 5 measurements each. Thus far, I've computed the individual mean and standard error for each trial (giving me three estimates of mean ± SE). I want to propagate the error of this through a system, so I'm going to want the overall best estimate of the measurement for this beaker quantity. In order to do that, what I've done at the moment is taken the weighted average using each mean and each uncertainty (which itself requires a small error propagation for the final uncertainty term) to calculate the overall "best estimate" of the beaker quantity. Given that I have three independent trials this seemed to make the most logical sense (rather than, say, taking the grand mean of all 15 replicates across the three trials and calculating the SE from that). Could I bother someone to verify that this is probably the best/most precise method for doing this? Any input is much appreciated. Feel free to ask if I've left any pertinent information out.

flapjack

Addendum: upon further thought, given that these are three independent trials, I believe I could also take the average of the three means and combine the uncertainty terms in quadrature (square root of the sum of squared uncertainties) if I'm not mistaken; cutting out the weighted portion of the above. My fear is that this would be less accurate overall, however. Any advice is appreciated.

mumbly_pie

With respect to the addendum: Standard deviations add this way for independent samples from a single distribution with known mean. Doing things this way will tend to slightly over-estimate the variance, relative to pooling all three of your trials together and estimating a single grand mean.

More generally: why not aggregate your trials? If they really are independent and measuring the same thing, this will tend to give better answers for most methods than arbitrarily dividing into three groups. As I mentioned above, this will give a smaller variance, which is generally what you want.

I don't understand what you mean by `propagating through a system'. Does this mean that you have some direct measurements (e.g. beaker volume) and would like to apply a function to the measurements (e.g. amount of time it takes for some Rube-Goldberg machine to run when the beaker is poured into a funnel)? If so, there are a few ways to deal with it. If you read a standard undergrad textbook, they will tell you to use things like `the delta method', otherwise known as taking a Taylor expansion of the function. Since you only have 15 data points, it is probably easiest to just apply your function to those points directly. There are a few cases where this won't do as well (e.g. if your function is very spiky AND you known a great deal about your function), so write again if you have some reason for suspecting this. If you're worried about these situations, and computing the function is easy, a first remedy might be to try to generate many (e.g. 10,000) data points by adding little bits of fuzz to your data and see if the resulting distribution is obviously screwy relative to that of the original points.

I hope this helps a little!

flapjack

Thanks for the response! I've asked for clarification and will hopefully get a response soon (I wasn't the one who actually ran the initial experiments). The way the data sheet is constructed implies that the three trials are independent but the five replicates per trial are not; this doesn't make a whole lot of sense to me in the scope of the experiment so it very well could be that each of the fifteen is an independent measurement and the three "trials" were simply arbitrary divisions (perhaps run across three separate days). If that's the case, as you said, aggregating the data will be the way to go.

Regarding propagation: I'm actually referring more generally to a propagation of error. That is, this measured beaker quantity is then used in subsequent calculations, and I'm propagating the uncertainty (based upon these empirical measurements) throughout said system using the best estimate of the quantity (its mean) and its uncertainty that I can calculate.

Hopefully I hit upon your points; please let me know if I neglected anything! And thanks again for the response; it's very much appreciated.

mumbly_pie

P1: Interesting. If the 5 replicates within each trial are dependant, it seems like you need to make some further assumptions. Taking means within a dependant sample may or may not be a good way to get at the real mean of a process, depending on the details of the dependence. Similarly, variances might not add this nicely. You do need *some* model to talk about these sorts of summary statistics.

P2: I meant exactly the same thing, sorry for being vague. I'm glad this is the case. So, again, it seems reasonable to do the calculations for each measurement separately if you're just interested in the final values, and then calculate summary statistics at the end - in other words, only compute the mean and variance of the thing you actually want. This can get substantially better answers than e.g. the Delta method. The fuzzing I mentioned was just a way to check against some possibly hidden numerical instability in doing this. As is often the case in statistics, there is a whole theory of how to propagate error through functions... but it gets pretty mathy pretty quickly.