Modern domestic horses display abundant genetic diversity within female-inherited mitochondrial DNA, but practically no sequence diversity on the male-inherited Y chromosome. Several hypotheses have been proposed to explain this discrepancy, but can only be tested through knowledge of the diversity in both the ancestral (pre-domestication) maternal and paternal lineages. As wild horses are practically extinct, ancient DNA studies offer the only means to assess this ancestral diversity. Here we show considerable ancestral diversity in ancient male horses by sequencing 4 kb of Y chromosomal DNA from eight ancient wild horses and one 2,800-year-old domesticated horse. Both ancient and modern domestic horses form a separate branch from the ancient wild horses, with the Przewalski horse at its base. Our methodology establishes the feasibility of re-sequencing long ancient nuclear DNA fragments and demonstrates the power of ancient Y chromosome DNA sequence data to provide insights into the evolutionary history of populations.