Idiopathic infertility is the most common individual diagnosis in male infertility, representing nearly 44% of cases. Research studies dating over the last half-century consistently demonstrate a decline in male fertility that is incompletely explained by obesity, known genetic causes, or diet and lifestyle changes alone. Human exposures have changed dramatically over the same time course as this fertility decline. Synthetic chemicals surround us. Some are benevolent; however, many are known to cause disruption of the hypothalamic-pituitary-gonadal axis and impair spermatogenesis. More than 80,000 chemicals are registered with the United States National Toxicology Program and nearly 2,000 new chemicals are introduced each year. Many of these are known toxins, such as phthalates, polycyclic aromatic hydrocarbons, aromatic amines, and organophosphate esters, and have been banned or significantly restricted by other countries as they carry known carcinogenic effects and are reproductively toxic. In the United States, many of these chemicals are still permissible in exposure levels known to cause reproductive harm. This contrasts to other chemical regulatory legislature, such as the European Union's REACH (Registration, Evaluation, Authorization and Restriction of Chemicals) regulations which are more comprehensive and restrictive. Quantification of these diverse exposures on an individual level has proven challenging, although forthcoming technologies may soon make this data available to consumers. Establishing causality and the proportion of idiopathic infertility attributable to environmental toxin exposures remains elusive, however, continued investigation, avoidance of exposure, and mitigation of risk is essential to our reproductive health. The aim of this review is to examine the literature linking changes in male fertility to some of the most common environmental exposures. Specifically, pesticides and herbicides such as dichlorodiphenyltrichloroethane (DDT), dibromochloropropane (DBCP), organophosphates and atrazine, endocrine disrupting compounds including plastic compounds phthalates and bisphenol A (BPA), heavy metals, natural gas/oil, non-ionizing radiation, air and noise pollution, lifestyle factors including diet, obesity, caffeine use, smoking, alcohol and drug use, as well as commonly prescribed medications will be discussed.