Nano-sized and filterable microorganisms are thought to represent the smallest living organisms on earth and are characterized by their small size (50–400 nm) and their ability to physically pass through <0.45 μm pore size filters. They appear to be ubiquitous in the biosphere and are present at high abundance across a diverse range of habitats including oceans, rivers, soils, and subterranean bedrock. Small-sized organisms are detected by culture-independent and culture-dependent approaches, with most remaining uncultured and uncharacterized at both metabolic and taxonomic levels. Consequently, their significance in ecological roles remain largely unknown. Successful isolation, however, has been achieved for some species (e.g., Nanoarchaeum equitans and “Candidatus Pelagibacter ubique”). In many instances, small-sized organisms exhibit a significant genome reduction and loss of essential metabolic pathways required for a free-living lifestyle, making their survival reliant on other microbial community members. In these cases, the nano-sized prokaryotes can only be co-cultured with their ‘hosts.’ This paper analyses the recent data on small-sized microorganisms in the context of their taxonomic diversity and potential functions in the environment.