In their paper "Do bilinguals automatically activate their native language when they are not using it?", Costa, Pannunzi, Deco, and Pickering (Cognitive Science, 2016) proposed a reinterpretation of Thierry and Wu's (2004, 2007) finding of native language-based (Chinese, L1) ERP effects when they tested Chinese-English late bilinguals exclusively in their second language (English, L2). Using simulations in a 6-node Hebbian learning model (three L1 nodes, three L2 nodes), Costa and colleagues suggested that form overlaps in L1 between otherwise unrelated words create a persistent relationship between their L2 translations. In this scenario, words in the nascent L2 lexicon overlapping in their L1 translations would become linked during learning because of the form overlap in L1; once the L2 words are learned, the direct link between them would be sufficient to generate robust apparently 'L1-mediated' priming without requiring any activation of L1 translations. Costa et al. contend that links between L2 words remain beyond the learning phase, even after links to L1 representations have been severed, and thus that their model affords an alternative account to (but not a rebuttal of) Thierry & Wu's claim of language non-selective activation -or automatic activation of translation equivalents- in late bilinguals. In this response, we build on Costa et al.'s original simulation code, showing that it can only reproduce L1-independent priming when implementing the L1 disconnection in their particular way. By contrast, when severing inter-language connections bi-directionally, their model fails to retain any sizeable influence of L1 form overlap on L2 activations. The model is not the theory, however, and we discuss several issues that would need to be addressed in further attempts to model language non-selective activation in late bilinguals.