Exudation of organic acid anions by plants as well as root-induced changes in rhizosphere pH can potentially improve phosphate (Pi) availability in the rhizosphere and are frequently found to occur simultaneously. In non-calcareous soils, a major proportion of Pi is strongly sorbed to metal oxi(hydr)oxides of mainly iron (Fe) and aluminium (Al) and organic anions are known to compete with Pi for the same sorption sites (ligand exchange) or solubilize Pi via ligand-promoted mineral dissolution. Root-induced co-acidification may also further promote Pi release from soil. The relative efficiency of these different solubilization mechanisms, however, is poorly understood. The aims of this study were to gain a better mechanistic understanding of the solubilizing mechanisms of four carboxylates (citrate, malate, oxalate, malonate) in five soils with high and low P surface site saturation. Results indicate that at a lower P saturation of solid phase sorption sites, ligand-promoted mineral dissolution was the main Pi solubilization mechanism, while ligand exchange became more important at higher soil P concentrations. Co-acidification generally increased Pi solubility in the presence of carboxylates; however the relative solubilizing effect of carboxylates compared to the background electrolyte (KCl) control decreased by 20–50%. In soils with high amounts of exchangeable calcium (Ca), the proton-induced Ca solubilization reduced soluble Pi, presumably due to ionic-strength-driven changes in the electric surface potential favoring a higher Pi retention. Across a wider soil pH range (pH 3–8), Pi solubility increased with increasing alkalinity, as a result of both, more negatively charged sorption sites, as well as DOC-driven changes in Fe and Al solubility, which were further enhanced by the presence of citrate. Overall, the relative efficiency of carboxylates in solubilizing Pi was greatest in soils with medium to high amounts of anionic binding sites (mainly Fe- and Al-oxy(hydr)oxides) and a medium P sorption site coverage, with citrate being most effective in solubilizing Pi.