In December of 1753, George Washington and Christopher Gist came through the area. They were shot at somewhere between present-day Evans City and Ellwood City (the exact location is unknown), through a region above the forks of the Ohio known as Mutheringtown or "Murdering Town". In 2003, the borough commemorated the event as part of the 250th Anniversary Commemoration of the French and Indian War.
Harmony was founded by the pietist Johann Georg Rapp and his Harmony Society in 1804. George Rapp came to America from Württemberg, Germany, in 1803 in search of land for his followers that was free from the religious persecution they faced in Germany. He was searching for a place for his followers to worship, farm and put into effect Rapp's communal religious teachings. In 1804, two groups of Harmonites purchased a tract of land in Butler County, Pennsylvania.
They formally established the Harmony Society in 1805 and lived in Pennsylvania for about 10 years before selling the Harmony property in 1814 to Abraham Ziegler, a Mennonite, and moving west to Indiana Territory, where they built the town of Harmony on the Wabash River (now New Harmony, Indiana). In 1824, they moved back to Pennsylvania, this time to Economy (now Ambridge). The settlements were economically successful, producing many goods in a clothing factory, a sawmill, a tannery, and from their vineyards and distillery. A hotel was also run in Harmony. In Economy, the group aided the construction of the Pittsburgh and Lake Erie Railroad, established the Economy Savings Institution and the Economy Brick Works, and operated the Economy Oil Company, Economy Planing Mill, Economy Lumber Company, and eventually donated some land in Beaver Falls for the construction of Geneva College. In 1869, Mennonite craftsmen built the stone wall around the Harmonite cemetery in Harmony with a unique revolving stone door. The society exerted a major influence on the economic development of western Pennsylvania. But since the group chose to adopt celibacy, they eventually died out when John S. and his wife Susanna C. Duss, the last surviving leaders of Harmony Society, died in 1951.
The Mennonite community in Harmony lasted until around 1904, by which time the town was selling more and more individual plots. Still extant is an old Mennonite meeting house on a hill near Harmony that was built in 1825.
Harmony's history, particularly that of the Harmony Society, is preserved at the Harmony Museum, operated in the town by Historic Harmony, Inc.
On a hill in Harmony, situated above Connoquenessing Creek sits a "seat" of stone (actually a rock formation). This seat was said to be used by Rapp to watch over the commune and its members in their daily lives. Thanks to local Boy Scout Troop 457 of Zelienople, a set of steps were built leading up to the seat as part of an Eagle Scout project for Christopher Van Arsdale of Harmony.