The townsite may have been visited by Leif Erikson and his Norsemen in 1007; a Runic Stone was found at the nearby village of Overton in 1812. It is said to be carved by Erikkson and the stone is located at the Yarmouth County Historical Society Museum.

The region was visited in 1604 by Samuel de Champlain, who named it Cap Fourchu, and it became a French fishing settlement. In 1759 settlers came to the townsite from Yarmouth, Massachusetts, and named it Yarmouth after their former home. The town was founded in 1761, when a larger group of settlers came from Sandwich, Massachusetts. They were then followed by Acadians in 1767 from the Grand Pré district and by United Empire Loyalists in 1785.

Initially called Cape Forchu, Yarmouth was first laid out in 1759 and incorporated in 1890. Through the 19th century it was a major shipbuilding centre, at one point making more ships per capita than any other port in the world.