Completed on 13 Feb 2017 by Krzysztof Jacek Gorgolewski .
Login to endorse this review.
Authors present and appealing methodological improvement on the Adaptive Segmentation (AS) method. The main improvement is alleviating the need to set input parameters (bandwidths sequence). Those parameters are fitted from the data in an optimal way.
Even though the paper has the potential to be a meaningful contribution to the field it lacks thorough comparison with the state of the art. The following steps to improve the situation should be considered:
- The selection of pattern used in the simulation seems to be motivated by the nature of fMRI data which is good, but at the same time, it does not highlight the specific issues that FAST is solving. Have a look at the simulations included in Polzehl et al. 2010 showing how smoothing across neighboring positive and negative activation areas can cancel the effect out. It would be beneficial to construct simulations that highlight the specific situation in which FAST overcomes the limitations of AS.
- Neuroimaging is strongly leaning towards permutation based testing methods due to the reduced number of assumptions. I would recommend adding cluster and voxel based permutation based inferences to your analysis. Please mind that permutation based testing is not the same as finding cluster cut offs via simulations.
- I would also recommend adding threshold-free cluster enhancement (Smith and Nichols 2009) to the set of compared methods. It is also a multiscale method that has been successfully used in many studies. This method also works best in comparison with permutation tests.
- It would be good to assess the rate of false positive findings in your comparison. This could be done by applying a random boxcar model to resting state data and evaluating how many spurious activations you find (see Eklund et al. 2012).
- Speaking of false positive and false negative voxels. It seems that the evaluation of your method in the context of the state of the art presented in Figure 4 is very sensitive to the threshold (alpha level) chosen for each method. I would suspect that AS and CT would perform better if a different alpha level was chosen. To measure the ability to detect signal more accurately I would recommend varying the alpha level to create a receiver operator curve (based on false positive and false negative voxels rather than Jaccard overlap) and calculate the area under it.
- In the figure, you use the TP11 acronym to denote the adaptive segmentation algorithm, but in the rest of the paper, you use AS. It would be good to normalize this.