Recursive Hierarchical Embedding in vision is impaired by posterior Middle Temporal Gyrus lesions

In language, left Inferior Frontal Gyrus (IFG) and left posterior Temporal Cortex (pTC: STS/MTG) are considered hubs for syntactic processing. However, it is unclear whether these regions support computations specific to language or more generally support analyses of hierarchical structure. Here, we address this issue by investigating hierarchical processing in a non-linguistic task. We test the ability to represent Recursive Hierarchical Embedding (RHE) in the visual domain by contrasting a recursion task (REC) with an iteration task (ITE). REC requires participants to correctly identify continuations of a hierarchy generating procedure, while ITE applies a serial procedure which does not generate new hierarchical levels.

In a lesion-based approach, we asked 44 patients with left hemispheric chronic brain lesion to perform REC and ITE. We modelled accuracies and response times with a drift diffusion model and for each participant obtained parametric estimates for the velocity of information accumulation (drift rates) and for the amount of information needed to make the decision (boundary separation). We then used these estimates in lesion-behaviour analyses to investigate how brain lesions affect specific aspects of RHE.

We found that lesions in the posterior temporal cortex (pTC) decreased drift rate in RHE, suggesting an impaired process of rule extraction from recursive structures. Moreover, lesions in IFG decreased boundary separation. The latter finding does not survive conservative correction but suggests a shift in the decision criterion. Since patients also participated in a grammar comprehension experiment, we performed explorative correlation-analyses and found that visual and linguistic RHE accuracies are correlated when the latter is instantiated as sentences with two nested embedding levels.

While the roles of IFG and pTC in linguistic processes are well established, here we show that pTC lesions slow information accumulation (drift rate) in the visual domain. This suggests that pTC is essential to acquire the (knowledge) representations necessary to parse RHE in visual structures, a finding mimicking language acquisition in young children. On the contrary, IFG lesions seem to affect RHE processing by interfering with more general cognitive control (boundary separation). This interesting separation of roles, rooted on a domain-general taxonomy, raises the question of whether such cognitive framing is also applicable to other domains.

Keywords: Hierarchy, Visuospatial, Lesion, Syntax, Inferior Frontal Gyrus, Middle Temporal Gyrus