Power analysis for mixed models

[Update: An updated version of pamm has been submitted to CRAN. See below for his author comments.]

This is a quick note that may be useful for some people. I was interested in knowing how many years of monitoring we need to detect a trend. This is a long term monitoring project, so we already have 7 years of data to play with. For a simple design, you can use the pwr library in R to answer your question, but for nested designs (i.e. random factors) things get hairy. In this and this paper they suggest building your own simulation and both has quite complex supplementary material with R code. I didn’t spent enough time to make sense of them. I also found thanks to @frod_san two packages that do it for you. The first, PAMM,  is broken (lme4 keep evolving, while the package didn’t, so even the example they use don’t work). The second (SimR) is not published yet, but is amazingly simple. All its code is in github and they are fast at fixing any bug you may detect (they fixed a small bug I found in no time). You can find a gist with an example of my question and how I calculate power here: https://gist.github.com/ibartomeus/e8eab8a32b57423341fb


8 thoughts on “Power analysis for mixed models

  1. Pingback: Power analysis for mixed models | Mubashir Qasim

  2. Pingback: Distilled News | Data Analytics & R

  3. Thanks for the Kain et al. reference I didn’t have.
    To (possibly) return the favor, 2 big (but woefully under-cited) papers on GLMM for temporal trends are:
    VanLeeuwen, D. M., Murray, L. W., & Urquhart, N. S. (1996). A mixed model with both fixed and random trend components across time. Journal of Agricultural, Biological, and Environmental Statistics, 435-453.
    Piepho, H. P., & Ogutu, J. O. (2002). A simple mixed model for trend analysis in wildlife populations. Journal of agricultural, biological, and environmental statistics, 7(3), 350-360.

    Trend McDonald, Bryan Manley, & Ryan Nielson have an unpublished survey available:
    McDonald, T. L., Manly, B. F., & Nielson, R. M. (2009). Review of environmental monitoring methods: trend detection. West Inc. Draft.
    Unfortunately, all 3 versions (plus Trent’s archived copy) don’t rend the equations properly in modern Adobe: you may need winXP & acrobat 6 or so.

    Alas, they, like the 2 behavioral papers you cite, don’t consider lack of independence in consecutive revisits (e.g., AR1 or various ecological drivers that persist longer than the revisit interval). Once the random effects are no longer i.i.d., you really have to use simulations, and you have to think in terms of process-models encapsulating your ecology to decide on a form for the non-independence. If you want to see my simulation, email me.

    In your gist, line 43 is one model for trends with multiple sites, but consider the separate slopes model too:
    m <- lmer(abundance ~ time + (1|site), data)
    m2 <- lmer(abundance ~ time + (1+time|site), data)
    The interpretations are subtly different: the first is “what is the trend in the population mean”, while the second is “what is the population mean of site-specific trends”. However, the second model generalizes to among-site differences. [If you’re doing behavior, my “sites” are your “individuals” or subjects.]

  4. I happen to be the (poor) maintainer of the PAMM package. Since I rarely us the package myself I totally forgot to update it. In addition I never recevied any email about the package being broken or obsolete. Sendin me an email (email address on Cran with the package) to let me know of the problem and I would have happily fixed the package. I have actually working on an updated version that should be released soon.

    • Thanks for commenting. You are right. I could have emailed you. Just try your example on SSF() and you will see the error. I just used a try catch for bypassing the lme4 models that fail. Great news you are planning an update, I am looking forward to that.

  5. Just to let you know that a fixed and updated version of pamm has been submitted to CRAN. Should be on it. Happy to help or implement any new functionalities. So far pamm is still offering function that are not covered by simr. At some point I might just integrate my work with simr so one package could do it all, but not in the near future I think.

Leave a Reply to Tom Philippi Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s