A key challenge for neurobiological models of social cognition is to elucidate whether brain regions are specialised for that domain. In recent years, discussion surrounding the role of anterior temporal regions epitomises such debates; some argue the anterior temporal lobe (ATL) is part of a domain-specific network for social processing, while others claim it comprises a domain-general hub for semantic representation. In the present study, we used ATL-optimised fMRI to map the contribution of different ATL structures to a variety of paradigms frequently used to probe a crucial social ability, namely ‘theory of mind’ (ToM). Using multiple tasks enables a clearer attribution of activation to ToM as opposed to idiosyncratic features of stimuli. Further, we directly explored whether these same structures are also activated by a non-social task probing semantic representations. We revealed that common to all of the tasks was activation of a key ventrolateral ATL region that is often invisible to standard fMRI. This constitutes novel evidence in support of the view that the ventrolateral ATL contributes to social cognition via a domain-general role in semantic processing, and against claims of a specialised social function.