Completed on 20 Aug 2014
Login to endorse this review.
Endorsed by 1 person.
This is an excellent paper that introduces several new innovations in modelling and computation. It's reasonably well written but still the material is substantial and technical, my postdoc and I had difficulty understanding it even though we are reasonably familiar with the general field.
Some suggestions for improving the paper:
P4 LOCO seems a crude way to deal with proximal contamination, particularly for the large chromosomes a major chunk of the genome is being neglected. It seems worth a comment about further improvements available by leaving out smaller regions in the vicinity of a tested SNP.
P5 “candidate causal SNPs” is a confusing expression, it must be possible to come up with a better name. I think the point is that they have larger effect sizes, although strictly they don't, when 50% heritability is spread over 1,250 SNPs these on average have higher h^2 than 2% spread over 60 SNPs. So the reality is confusing as well as the name.
P6 Some version of FaST-LMM is the nearest thing we have to state of art, so you need to include a comparison on a terrain where it does well in order to assess the validity of your approximations (or perhaps you could compare with Limix? I'm not familiar with that, so just a suggestion, it may not be appropriate). It's great that you can handle much bigger problems than FaST-LMM but that's not an excuse for failing to compare with the standard method (I'm not calling for anything very extensive, but something)
P8 “BOLT-LMM increases power to detect associations for WGHS phenotypes” I find slightly grating the series of boasts in section headings, I am from an old school that regards too much boasting about your method as going against the ideal of academic restraint. One reason is that there are often caveats to the claims in the title. Here, you don't prove greater power for association, you find a stronger signal (apparently large but not always significant) at SNPs already reported from these data. It's evidence for the method but not conclusive.
P9 The justification for introducing the term “pseudo-h^2” is weak – it adds an extra level of confusion to an already complex paper (and the last thing we need is yet another opportunity to misuse the term “pseudo”). h^2_g is heritability tagged by the SNPs, it will vary according to the sample. I think it's hopeless to try to define something that is only the heritability due to “local” LD because it is undefinable. You could say that h^2_g for your sample includes some signal from pop struct etc. and so might be lower in other samples – not ideal but I don't think we can do better.
P2 and P10 “a challenge of applying Bayesian methods in the GWAS setting is that Bayesian statistics are not readily interpretable in the customary hypothesis testing framework.” This is a silly claim: nobody really understands what p-values mean, they are popular because of habit not reason, and we should be collectively ashamed of that not encourage it. The same p-value can mean weak evidence in a low-powered study and very strong evidence in a well powered study. Interpretability is not a problem for a Bayesian analysis: it gives you the probability of the outcome you are interested in, what can be more interpretable than that? (Although personally I'm not a big fan of hypothesis testing/Bayes factors, I would do everything as an estimation problem and report posterior distributions for effect sizes). I'm happy for you to use a frequentist method as a practical short-cut but if you could do a Bayesian analysis at similar computational cost then it would clearly be even better and you should say so.
P15 “the GRM does not accurately reflect the covariance matrix of the distribution of unassociated (i.e., null) SNPs.” I don't understand this. Admittedly there's quite a bit I don't fully understand in a 79 page paper! But this seems puzzling because it seems to me intuitive that you must only need the covariances of the causal SNPs, why does it matter to get covariances right if they are not causal?
P17 A key idea is the mixture of two normals prior for SNP effect sizes – this seems very similar in spirit to the many generalisations of the usual gaussian prior in the Bayes alphabet of methods developed by animal breeders – you should spell out the connections a little more on P17. Also presumably there is no requirement to stay with two components in the mixture, wouldn't some infinite mixture of Gaussians be even better? Is it even necessary to make the Gaussian assumption if you are only using fitted values? Finally why not put in covariates as fixed effects in (18), this seems more natural than what the authors propose, and I think might be superior.
One further point that might be useful to clarify is that SNPs all get the same prior, which is a weighted mixture of two Gaussians. In other words they are not individually allocated to one or other prior.
The increase in prediction performance in Fig 3b is impressive. If I understand correctly, BOLT-inf should be exactly equivalent to BLUP, if so wouldn't it be better to say that? And since BLUP is standard for prediction why even show PCA here since it is clearly inferior.
P4 “ranging up to N = 3,750 to 480,000 ”
P10 “to than ”
P11 “all of the above are N-vectors” Not strictly true, x and beta are not – probably better to omit.
P17 “σβ2,1 ≫ σβ2,2 small,”