The Inferior Frontal Occipital Fasciculus (IFOF) is a major anterior-to-posterior white matter pathway in the ventral human brain that connects parietal, temporal and occipital regions to frontal cortex. It has been implicated in a range of functions, including language, semantics, inhibition and the control of action. The recent research shows that the IFOF can be sub-divided into a ventral and dorsal branch, but the functional relevance of this distinction, as well as any potential hemispheric differences, are poorly understood. Using DTI tractography, we investigated the involvement of dorsal and ventral subdivisions of the IFOF in the left and right hemisphere in a response inhibition task (Go/No-Go), where the decision to respond or to withhold a prepotent response was made on the basis of semantic or non-semantic aspects of visual inputs. The task also varied the presentation modality (whether concepts were presented as written words or images). The results showed that the integrity of both dorsal and ventral IFOF in the left hemisphere were associated with participants’ inhibition performance when the signal to stop was meaningful and presented in the verbal modality. This effect was absent in the right hemisphere. The integrity of dorsal IFOF was also associated with participants’ inhibition efficiency in difficult perceptually guided decisions. This pattern of results indicates that left dorsal IFOF is implicated in the domain-general control of visually-guided behaviour, while the left ventral branch might interface with the semantic system to support the control of action when the inhibitory signal is based on meaning.