Are there any G-theory aficionados out there?
I am using G-theory to calculate reliability coefficients for a nested
study.
I have calculated the variances of each term in my study design using
General Linear Model in SPSS but am unsure of the formulas for
calculating the G-coefficients.
I have laid out my reasoning below rather explicitly in the hopes that
someone can follow the logic and provide their two-cents worth!
The following is a simplified study design. There are 4 facets:
raters, patients, groups, and time.
3 Raters
5 Patients are nested in each of 2 Groups P(Grp)
2 Times
I am interested in how well I can differentiate between groups (e.g.
the ratings should help identify which group is 'doing better').
Therefore the facet of differentiation is Group. I don't care to
differentiate over time, therefore it is fixed facet. Patients are
nested so I think it would be a fixed facet for now ??
The basic formula for calculating a G-coefficient is:
The Numerator contains:
-your facet of differentiation PLUS all the interactions between your
facet of differentiation and your fixed facets.
The Denominator contains:
-your numerator PLUS all the interactions between your facet of
generalization PLUS the main effects of any random facets PLUS the
error term.
Using G-theory, I believe that the following will yield a G-
coefficient that is the Classical Test Theory-equivalent of inter-
rater reliability: V is variance.
G = V(grp) + V(grp*time)
----------------------------------------------------------------------------------------------------------------------------------------------
V(grp) + V(grp*time) + V(grp*rater) + V(patient(grp)*rater) +
V(rater*time) + V(rater*time*grp) + V(err)
where V(err) is V(rater*time*patient(grp))
My dilemmas are:
1. Should patients be in the formula for the G-coefficient?
2. Should variance of patient nested in group be contained in the
numerator?
Thanks in advance for your thoughts!
Heather
