Claridge-Chang Lab
  • About
  • People
  • Papers
  • Join
  • Blog
  • Contact

Reporting mean differences

9/25/2016

0 Comments

 
Why do I need to do this?
A lot of biology involves (1) doing an experiment where you make an intervention and (2) measuring what effect the intervention has.

If you are living in the Dark Age of P, you assume your intervention has had absolutely no effect, and then calculate the probability of seeing your data (or more extreme data), under that assumption of zero effect. Since you likely did the experiment because you thought there would be an effect, it seems super weird and slightly depressing to then go assuming zero effect when you start analyzing your data.

​If you use estimation, a far more sensible method, you want to estimate the size of the effect of your intervention. The most straightforward effect size is the difference between the control mean and the intervention mean ('mean difference'), along with its confidence interval.

I've got confidence interval error bars, is that enough?
It's a good start to have CI error bars on your observed data plots. However, you also need to be able to say things like:
     "when we made the intervention to flies their behavior X
​     increased by +54% [95CI +35, +78], P = 0.01."

You need (1) know what your delta variable is and (2) know how to get these numbers.


What statistics are essential?
The three essential statistics are:
  • Mean difference (your 'delta variable')
  • Confidence interval of the mean difference
  • Sample sizes (N)
Figure out how to calculate these numbers, and work them into your writing, especially in the figure legend, and also in the main text when you want to highlight an effect. It's simply not enough to say "behavior X increased," or worse, "behavior X increased significantly" without putting an effect size number on that statement, either immediately in the Results section or in the relevant Figure legend. If you have this you can import it into a Google Spreadsheet and share with your co-authors, one sheet per Figure or Panel.

What statistics are nice to have?

  • Hedges' g
  • P
It's nice to have Hedges' g, because this effect size is in units of standard deviations, which is sort of a universal currency. Most scientists will know what it means for an intervention to produce a 1 SD shift in the mean, and have an intuition that this is a large effect. Report P for pro forma purposes only and state "no significance tests were conducted, P was reported pro forma." This will satisfy reviewers wanting to see P while avoiding significance testing.

What text style should I follow?
The text format we are using for mean difference with its confidence interval is:

     -1.5 [95CI -1.2, -1.8]
     or
      +1.5 [95CI +1.2, +1.8] 
Note the use of the +/- signs to denote that this is a measure of the change in the variable, not a measure of the variable itself. I prefer '95CI' instead of '95%CI' as I find the latter to be cluttered.
 
Often it is useful to write something like this.
     ∆weight = +1.5 µg [95CI +1.2, +1.8] 
to remind the reader of the change variable (∆weight) and the units (µg). The confidence interval bounds are contained within square brackets [], and this generally follows the mean difference closely in text. It is then followed by the supporting statistics. For example:

     ∆VO2 = -1.5 µl/fly/min [95CI -1.2, -1.9], N = 65, 62
     or
     ∆VO2 = -1.5 µl/fly/min [95CI -1.2, -1.9], g = 0.56, P = 0.01, N = 65, 62

The g statistic was invented by Larry Hedges, so Hedges' g uses a possessive apostrophe after the 's.' The 'g' is italicized. P should be italicized and capitalized by default. 

What graphical style should I follow in charts?
Instead of the little stars, you can put the effect size right next to your difference marker. In this case, simplicity is a virtue. You can write '∆ = -1.5' next to the first marker, then just the numbers without the '∆ =' for the rest of the markers in that Figure.
0 Comments

Some notes on google docs

9/15/2016

0 Comments

 
When using Google Docs, there are a few tricks. 
  • Use Paperpile for references.
  • Follow Practical Typography where possible.
  • Margins larger than 1 inch. Narrower text columns are easier to read.
  • Never use Arial, use Helvetica Neue instead if Arial is required.
  • If Arial not required, use Source Serif Pro and Source Sans Pro.
  • Use Serif for the body text, Sans for headers and figure legends.
  • Use the Text Styles menu to style each paragraph by typeface, margins, type weight, and paragraph spacing.
  • Main text paragraphs should have no space before, and a space afterwards.
  • Paragraph headers should have no spaces before or after.
  • Text Styles allows you to auto-generate a Table of  Contents. 
  • Use 200-300 dpi PNG files for figures.
  • When discussing a project, use Basecamp. When discussing the details of a manuscript, use the Docs Comment function. I admit there is a lot of overlap between the two.
  • Try to avoid using email to discuss either.
0 Comments

    Author

    ACCLab

    Archives

    March 2025
    February 2025
    January 2024
    July 2023
    April 2023
    March 2023
    September 2022
    March 2022
    February 2022
    October 2021
    September 2021
    September 2019
    March 2019
    June 2018
    May 2018
    April 2018
    March 2018
    August 2017
    January 2017
    December 2016
    October 2016
    September 2016
    August 2016
    July 2016
    May 2016
    August 2015
    July 2015
    June 2015
    May 2015
    January 2015
    December 2014
    March 2014

    Categories

    All

    RSS Feed