Completed on 23 Jun 2016 by Justyna Hobot. Sourced from http://biorxiv.org/content/early/2016/06/09/058032.
Login to endorse this review.
Author response is in blue.
"@anilkseth: TMS to prefrontal (or parietal) cortex does NOT impair visual metacognition, new @sacklercentreled by @DanielBor https://t.co/LnHE3DRtL5."
Dear Authors, how would you rate your awareness that the quoted sentence is just a catchy overstatement? I allow myself to post some comments on the paper, I hope this might be helpful.
1. "An advantage of TMS, besides its non-invasive nature, is that TMS-induced changes are limited to short time periods so that plasticity is unlikely to affect performance."
Didn’t you apply TMS in order to induce the plasticity-like changes that affect cognitive performance?
2. "First, continuous theta burst TMS (cTBS) was used instead of repetitive TMS."
Continuous Theta Burst Stimulation (cTBS) is an example of repetitive TMS. Repetitive TMS simply means it has a precise temporal pattern of pulses, and cTBS has the precise temporal pattern of pulses (see e.g. Bergmann 2016 or Oberman 2011).
Bergmann, T. O., Karabanov, A., Hartwigsen, G., Thielscher, A., & Siebner, H. R. (n.d.). Combining non-invasive transcranial brain stimulation with neuroimaging and electrophysiology: Current approaches and future perspectives. NeuroImage. http://doi.org/10.1016/j.neuro...
Oberman, L., Edwards, D., Eldaief, M., & Pascual-Leone, A. (2011). Safety of Theta Burst Transcranial Magnetic Stimulation: A systematic review of the literature. Journal of Clinical Neurophysiology, 28(1), 67–74. http://doi.org/10.1097/WNP.0b0...
3. "This technique involves a very rapid sequence of TMS pulses, typically for 40 s, and is thought to suppress cortical excitability for up to 20 minutes (ref. 19)"
"thought to suppress cortical excitability" – the 40 s cTBS may suppress M1 excitability, as long as it is applied correctly and the basal state of the brain allows such changes to occur, but e.g. the change of current direction can reverse inhibition to facilitation (see e.g. Jacobs 2012), and the short version of cTBS (like the one used by you) may actually increase M1 excitability, if there is no prior voluntary motor activation (see e.g. Gentler 2008).
"for up to 20 minutes" – you referred to Huang 2005, where the motor cortical excitability after the 40 s of cTBS was suppressed for 60 min. The after-effects lasting up to 20 minutes were also reported, but after 20 s (not 40 s) of the cTBS. Therefore, there is no need to confuse the reader by writing: "TMS pulses, typically for 40 s, and is thought to suppress cortical excitability for up to 20 minutes".
Jacobs, M. F., Zapallow, C. M., Tsang, P., Lee, K. G. H., Asmussen, M. J., & Nelson, A. J. (2012). Current direction specificity of continuous θ-burst stimulation in modulating human motor cortex excitability when applied to somatosensory cortex. Neuroreport, 23(16), 927–931. http://doi.org/10.1097/WNR.0b0...
Gentner, R., Wankerl, K., Reinsberger, C., Zeller, D., & Classen, J. (2008). Depression of human corticospinal excitability induced by magnetic theta-burst stimulation: evidence of rapid polarity-reversing metaplasticity. Cerebral Cortex (New York, N.Y.: 1991), 18(9), 2046–2053. http://doi.org/10.1093/cercor/...
Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P., & Rothwell, J. C. (2005). Theta burst stimulation of the human motor cortex. Neuron, 45(2), 201–206. http://doi.org/10.1016/j.neuro...
4. "In this way, TMS administration can be entirely separated from the behavioural task, and therefore will not distract the participants from it."
It may be worth to note that what happens just after applying cTBS may reverse its after-effects (see e.g. Huang 2008), which means the first minutes of performing the post-TBS block may influence the effects observed on the following part. Did you try to check, how consistent the task performance was, by comparing the first 150 trials with the second half of the block?
Huang, Y.-Z., Rothwell, J. C., Edwards, M. J., & Chen, R.-S. (2008). Effect of physiological activity on an NMDA-dependent form of cortical plasticity in human. Cerebral Cortex (New York, N.Y.: 1991), 18(3), 563–570. http://doi.org/10.1093/cercor/...
5. "In addition, a small (n=7) patient lesion study showed that the anterior prefrontal cortex (i.e. a region neighbouring the DLPFC) selectively impaired perceptual metacognition, though not memory-based metacognition, compared with patients who had temporal lobe lesions (27)."
You may check Del Cul 2009 paper, which also indicated the involvement of aPFC in perceptual metacognition, and the study was conducted on a bigger group of patients (n=15) than the one you refer to. Moreover, McCurdy 2013 showed that variation in visual metacognitive efficiency in his study was correlated with volume of frontal polar regions, while the variation in memory metacognitive efficiency with volume of the precuneus. However, I wonder, how this should support the use of DLPFC, instead of aPFC? Only because it is a neighbouring region?
Cul, A. D., Dehaene, S., Reyes, P., Bravo, E., & Slachevsky, A. (2009). Causal role of prefrontal cortex in the threshold for access to consciousness. Brain, 132(9), 2531–2540. http://doi.org/10.1093/brain/a...
McCurdy, L. Y., Maniscalco, B., Metcalfe, J., Liu, K. Y., Lange, F. P. de, & Lau, H. (2013). Anatomical Coupling between Distinct Metacognitive Systems for Memory and Visual Perception. The Journal of Neuroscience, 33(5), 1897–1906. http://doi.org/10.1523/JNEUROS...
6. "In experiment 1 we therefore sought to replicate the Rounis study, as well as extend it to the posterior parietal cortex, since this region in neuroimaging studies is very commonly co-activated with DLPFC".
What do you mean when saying "this region"? PPC is an area, big enough to be consisted of subregions that have a different cytoarchitectonics, a different pattern of structural connectivity, and the activity of these subregions may correlate in a different way with the activity in different subregions of DLPFC (e.g. Leech 2011). The same of course applies to DLPFC (see e.g. Optiz 2016 for comparison of distinct DLPFC stimulation zones with respect to functional networks).
Leech, R., Kamourieh, S., Beckmann, C. F., & Sharp, D. J. (2011). Fractionating the Default Mode Network: Distinct Contributions of the Ventral and Dorsal Posterior Cingulate Cortex to Cognitive Control. The Journal of Neuroscience, 31(9), 3217–3224. http://doi.org/10.1523/JNEUROS...
Opitz, A., Fox, M. D., Craddock, R. C., Colcombe, S., & Milham, M. P. (2016). An integrated framework for targeting functional networks via transcranial magnetic stimulation. NeuroImage, 127, 86–96. http://doi.org/10.1016/j.neuro...
7. "Furthermore, we attempted to enhance the original Rounis design, by including an active TMS control (vertex), rather than sham stimulation."
Is there any reason to assume that by applying 2 times the same protocol to the same site (600 pulses to the vertex) you control for the effects of applying the same protocol to two different sites (300 pulses to each site)?
8. "We were concerned that managing the relative frequency of subjective ratings of "clear" and "unclear" labels across an experiment may have placed additional working memory demands on participants, since they would need to keep a rough recent tally of each rating in order to balance them out. In addition, these labels were difficult to interpret psychologically on account of their relative nature. We therefore opted instead for the labels "[completely] random [guess]" and "[at least some] confidence." Using confidence instead of clarity labels is a common practice, consistent with other recent metacognition studies (24, 25)."
What do you think about a possibility that by replacing the introspective report with a different kind of metacognitive report you investigated a different phenomenon/underlying processes than Rounis 2010 did (see e.g. Overgaard and Sandberg 2012)? In the papers of Fleming you refer to, metacognitive assessment always follows the behavioural response, which means it relies on processes such as e.g. error monitoring (see e.g. Young and Summerfield 2012), and in your paradigm the behavioural response is combined with the metacognitive rating, so it may be difficult to conceive it as a metacognitive measure of the confidence in choice ("Most notably, confidence in choice was used instead of visibility to determine metacognitive judgement.").
Overgaard, M., & Sandberg, K. (2012). Kinds of access: different methods for report reveal different kinds of metacognitive access. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1594), 1287–1296. http://doi.org/10.1098/rstb.20...
Yeung, N., & Summerfield, C. (2012). Metacognition in human decision-making: confidence and error monitoring. Phil. Trans. R. Soc. B, 367(1594), 1310–1321. http://doi.org/10.1098/rstb.20...
9. "The AMT was defined as the lowest intensity that elicited at least 3 consecutive twitches, stimulated over the motor hot spot, while the participant was maintaining a voluntary contralateral finger-thumb contraction."
There is no consistency in the literature in what is understood as AMT, the main differences are present in: the amount of pulses required, the amplitude of MEP required, the level of muscular contraction. By looking at this paper the reader cannot know what method was used, even if it was the same as Rounis 2010 it still says nothing, as she does not provide this information either.
10. "cTBS was delivered with the handle pointing posteriorly and the coil placed tangentially to the scalp"
What was the current direction used? If you did not change the current direction to the reversed (AP-PA in the brain), then the current flow (PA-AP) was the opposite to the optimal (AP-PA), that presumably resulted in higher motor thresholds compared to ones that are obtained by using the optimal method.
11. "The standard cTBS pattern used, as with the Rounis 2010 study, was a burst of three pulses at 50 Hz given in 200 ms intervals, repeated for 300 pulses (or 100 bursts) for 20 s."
It may be good to mention the pulses (if they) were biphasic. Also "given in 200 ms intervals" may confuse the reader, because she may not be sure whether the inter trial interval (the time period between the last pulse in the first train to the first pulse in the next train) was 160 ms (as it should be) or 200 ms.
12. You have performed a lot of stimulations, have you forgotten that PPC was stimulated as well? There is no information in the paper on how PPC was determined; neither about the region of interest (within PPC) nor about the method used to target this region. Also, you may want to change PPN to PPC on the charts.
13. Surprisingly, there are quite big differences in metacognitive sensitivity in the pre-TBS blocks of the experiment 1, which makes it impossible to compare the effects resulting from stimulation to the different sites. Even more surprisingly, you do not address this issue in the discussion.
14. "In this way, we could rigorously explore the within subject likelihood of both a metacognitive impairment (or enhancement) following DLPFC cTBS and no metacognitive change following vertex cTBS, with a potential single subject replication of this pattern."
Doesn’t the lack of counterbalancing across the simulation sites indicate this was not a "rigorous exploration" (e.g. an influence of the behavioural learning)?
15. "The remaining 17 participants are summarised in table 5. Ten of these participants had no meta d’ changes on the first DLPFC session, and thus were not asked to return for subsequent sessions."
Does it mean that if you got the intended effect (by rejection of >50% of the participants), you would conclude that cTBS influences metacognitive sensitivity? I assume that you would not, therefore it may be difficult to follow the idea behind the rejection of participants who do not confirm the expectations of the researchers.
16. "Of the remaining 7 participants, 3 showed the expected impairment, while 4 showed a clear metacognitive enhancement following DLPFC cTBS. 6 of these 7 participants also showed a clear metacognitive change for the vertex control session, and thus were not asked to return for the 3rd session (2nd DLPFC)."
Still quite difficult to follow. The possibility of obtaining some significant effects caused by stimulation to the control site, in my opinion, represents the goal of the active control stimulation (performed in order to evaluate whether the potential significant effect of stimulation is site-specific). Also it probably shouldn’t be surprising to observe some effects in your control condition, as the vertex stimulation may influence the activity in DMN (e.g. Jung 2016).
Jung, J., Bungert, A., Bowtell, R., & Jackson, S. R. (2016). Vertex Stimulation as a Control Site for Transcranial Magnetic Stimulation: A Concurrent TMS/fMRI Study. Brain Stimulation, 9(1), 58–64. http://doi.org/10.1016/j.brs.2...
17. "We have therefore not only failed to replicate the Rounis result, but provided evidence from our own experiments that on this paradigm there is no modulatory effect of theta-burst TMS to DLPFC on metacognition."
This evidence is not a scientific evidence, this explanation is as likely as the one that you did't apply the stimulation protocol properly (e.g. because it may work only when the current flow is perpendicular to the stimulated structure). The generalisations such as "no modulatory effect of theta-burst TMS" may not be accurate, especially in the case when one uses only the short version of one type of TBS protocols (300 pulses of cTBS), or "DLPFC" – this is just the general term, that is related to multiple subregions, and the stimulation in your study was (probably) applied just to one of them.
18. "First, it may well be that cTBS of cortex, at the medically safe stimulation thresholds commonly employed (80% of active motor threshold) is just not intense enough to induce a subtle cognitive effect, such as a reduction in metacognitive sensitivity."
Is there any way to verify this explanation? For example, by providing the reader with the information about the average MSO, the current direction used, the method used to determine AMT?
19. "To our knowledge, only one published paper to date, besides that of Rounis and colleagues, has demonstrated the general efficacy of DLPFC cTBS in modulating cognitive performance (38)."
What about, e.g.: cTBS applied to the left DLPFC impairs MCST performance (Ko 2008); DLPFC stimulation changes subjective evaluation of percepts, i.e. metacogniton (Chiang 2014); cTBS over the left DLPFC decreases medium load working memory performance (Schicktanz 2015). Moreover, Rahnev 2016 reported that both: cTBS applied to right aPFC and cTBS applied to right DLPFC affected metacognition. Is there any reason to ignore the results that are not consistent with the view presented in the discussion?
Ko, J. H., Monchi, O., Ptito, A., Bloomfield, P., Houle, S., & Strafella, A. P. (2008). Theta burst stimulation-induced inhibition of dorsolateral prefrontal cortex reveals hemispheric asymmetry in striatal dopamine release during a set-shifting task – a TMS–[11C]raclopride PET study. European Journal of Neuroscience, 28(10), 2147–2155. http://doi.org/10.1111/j.146
Schicktanz, N., Fastenrath, M., Milnik, A., Spalek, K., Auschra, B., Nyffeler, T., … Schwegler, K. (2015). Continuous Theta Burst Stimulation over the Left Dorsolateral Prefrontal Cortex Decreases Medium Load Working Memory Performance in Healthy Humans. PLoS ONE, 10(3). http://doi.org/10.1371/journal...
Chiang, T.-C., Lu, R.-B., Hsieh, S., Chang, Y.-H., & Yang, Y.-K. (2014). Stimulation in the Dorsolateral Prefrontal Cortex Changes Subjective Evaluation of Percepts. PLOS ONE, 9(9), e106943. http://doi.org/10.1371/journal...
Rahnev, D., Nee, D. E., Riddle, J., Larson, A. S., & D’Esposito, M. (n.d.). Causal evidence for frontal cortex organization for perceptual decision making.
20. "Following a 1 minute interval, this was repeated at a different site for a further 20s (or again on the vertex in the control condition), determined by which group the participant was assigned to. The five groups were: i) bilateral DLPFC, ii) bilateral PPC, iii) left DLPFC and PPC, iv) right DLPFC and PPC, and v) VERTEX (control)."
Did you counterbalance the starting sites of the stimulation?
21. "However, the fact that we did not observe metacognitive impairment reliably in any subject in experiment two speaks against interpreting our null results simply in terms of missing the DLPFC during cTBS."
Does it? Following this way of reasoning one may conclude you missed the DLPFC in the first experiment, as you observed the effect just for some of the participants.
22. "... our results nevertheless indicate that the cTBS approach is not sensitive enough to establish a causal link between DLPFC and metacognitive processes."
Can it stem from the fact you used a short version of the protocol (300 pulses), and a probability the conventional cTBS (600 pulses) is excitatory in the first half and switches to inhibition only after the full length protocol (see e.g. Gamboa 2010), so application of 300 cTBS pulses may result either in no change or in small inhibitory/excitatory effects? Or, can it rather result from a possibility that the site within DLPFC you were targeting may have nothing to do with metacognitive processes?
Gamboa, O. L., Antal, A., Moliadze, V., & Paulus, W. (2010). Simply longer is not better: reversal of theta burst after-effect with prolonged stimulation. Experimental Brain Research, 204(2), 181–187. http://doi.org/10.1007/s00221-...
Thanks Justyna for such a comprehensive comment on our preprint. And on emailing me with further helpful comments relating to these. Based on those further comments and our own thoughts, here are the responses to each issue (those involving changes to the revised manuscript, already uploaded, start with asterisks):
1) *This was in the context of lesion studies, where we were discussing permanent long-term plasticity (over months) following the brain damage, which obviously isn’t the case with TMS. We will clarify the text to reflect this point, given it seems not to have been clear earlier.
2) *We were under the impression that, historically at least, “repetitive TMS” referred to the more conventional single frequency pulses, which cTBS is not a version of. We will clarify the text to write “conventional repetitive TMS” to emphasise this point.
3) *Thanks for pointing this out, and we will use your email suggestion to change the text to: “the protocol used in our study is thought to suppress cortical excitability for up to 20 minutes”.
4) *We did look at the first 100 versus the last 100 trials. In both cases there were no significant results, and the results looked similar the full trials. We will now include this in the revised text.
5) *We already refer to the Del Cul 2009 paper in our manuscript, but that wasn’t about metacognitive impairment, but a decline in subjective confidence. As for the MCcurdy, this wasn’t a patient paper, but a VBM paper, so not relevant to this point. However, it’s clearly relevant to the rest of the manuscript, so we will now include it.
6) This quote is linked to 5 references, which make clear what region we are referring to. The fractionation of this region is a controversial topic and, combined with the lack of precision of TMS, we feel it unnecessary to clarify further what area we are referring to.
7) By matching the duration at the control site to the pair of experimental sites, we were controlling for the amount of experimental cTBS. Had we used half this amount, there could have been a criticism that our control protocol was underpowered. However, this is all moot, given that we didn’t find any effects either for the experimental or control site.
8) Simply because accuracy and confidence were reported simultaneously doesn’t stop metacognitive judgements from being collected effectively. And confidence is a minor change from awareness. If such a subtle difference is the reason for the effect to disappear, then this would suggest it was a terribly fragile effect to begin with.
9) We were constrained to follow the procedure of Rounis exactly, and believe we have given sufficient details to explain the AMT used here.
10) Again we followed the protocol exactly as was done with the Rounis study, using a similar TMS machine, so should have induced the same type of current.
11) Our cTBS protocol was exactly the same as the standard type and that used by Rounis. We believe we have made clear what we used, but if the reader wants more details, they can refer to the original technical paper.
12) *We will amend the paper to include this information and correct the figure, as suggested.
13) Metacognitive sensitivity pre-TBS for DLPFC, PPC and Vertex are all very similar and close to zero. It’s only for the left and right (DLPFC and PPC) subjects that metacognitive sensitivity pre-TBS is strangely raised. We put this down to potential extra noise with these conditions (possibly relating to these being close to or actually the smallest sample sizes). The critical issue though is the comparison between pre and post-TBS conditions, of which there is no difference in each condition.
14) We limited behavioural learning by including a more rigorous training setup than in the Rounis study, to ensure stable performance. In addition, the paradigm is designed to attempt to keep objective performance at a constant level by constantly titrating contrast, so that performance is approximately 75% for all conditions for all subjects (we rejected those subjects who were 10% better or worse than the 75% score). We therefore don’t believe that counter-balancing was an issue, especially as we saw no significantly positive results at all here.
15) In experiment 2, we were trying to give ourselves the best chance of rigorously replicating the Rounis effect within subjects, even in the single subject case. Had a large proportion of our 27 subjects showed a metacognitive impairment in the first DLPFC condition, none in the first VERTEX condition, and then impairments and no effects in the second DLPFC and VERTEX conditions respectively, statistics on a proportion of the 27 would have been appropriate and demonstrated a replication. As it was, not a single of our 27 subjects (or 17 that showed no instabilities) showed this pattern, so statistics weren’t necessary.
16) We had no reason to believe that activation of the VERTEX should have influenced metacognition, and know of no papers that support this claim. Therefore it was an appropriate location for an active control.
17) In referring to “this paradigm” we meant the paradigm that Rounis used and found positive results with. We agree that there could be methodological issues with this paradigm, or indeed any similar TMS paradigm exploring metacognition, and explore these issues at length in the discussion.
18) The thresholds used in this study were identical to that used by Rounis, and is highly comparable to most studies in the area.
19) *Thank you for pointing out these interesting references, of which we were unaware at the time of submission. Although the Ko paper includes an active control (vertex again), the Schicktanz paper does not. The Chiang paper doesn’t involve cTBS at all, so is not relevant. Finally, the Rahnev study, although highly relevant, was only not included because it was published just as we were submitting this preprint. However, we note that it found the opposite results to that of Rounis, namely a boost to metacognition following cTBS to DLPFC (or APFC). We will therefore now discuss both the Ko and Rahnev papers, and briefly mention the Schicktanz paper.
20) *Yes, site order was counterbalanced. We will add this detail to the manuscript.
21) *The point we are making here is that our repeated within-subjects design, focusing more on single subject effects, gave us 27 chances to hit DLPFC reliably, and find corresponding metacognitive effects. The fact that no subject in experiment two demonstrated this suggests that the issue wasn’t just about accurately hitting the DLPFC with CTBS. We will adapt the wording of this section to reflect this point.
22) Again, we were constrained to repeat the methods of the original Rounis study, which included 300 pulses per site, and reported inhibitory effects. Had we found the opposite results to that expected from the Gamboa study, because of 300 pulses instead of 600, we should have seen excitatory effects, which we did not.