The Thomas Lee House, built circa 1660, is the oldest house in Connecticut that is still in its primitive state. This building is located in the southwestern section of East Lyme, adjacent to Rocky Neck State Park, at the intersection of Connecticut Route 156 and Giants Neck Road. Co-located on this site is the one-room Little Boston Schoolhouse, which was relocated to its current location from across Route 156. The town features six homes from 1699 or earlier, and the Old Stone Church Burial Ground from 1719 located off Society and Riverview roads.
The area occupied by the town was originally inhabited by the Nehantic people, who maintained villages in the present-day Indian Woods section as well as on Black Point, in the McCooks Beach area and near the Niantic River. The tribe allied itself with the colonists in the 1636 war against the Pequot people. The Nehantic eventually died out in the mid-19th century. The 1750s Ezra Stiles map shows the Nehantic village in what is now Indian Woods as consisting of "12 or 13 huts".
East Lyme, then a part of Lyme, had several taverns which offered stopping places for travelers such as Sarah Kemble Knight. These included Calkins Tavern on what is now Boston Post Road, Royce's Tavern, and Taber Tavern near present-day I-95. At least 25 Revolutionary War veterans are buried within the borders of East Lyme, and countless more veterans from East Lyme found resting places in upstate New York, New Hampshire and Ohio. Moses Warren, along with General Samuel Holden Parsons originally of Lyme and Moses Cleaveland, set out to survey the Ohio Territory in the latter part of the 1790s and has Warren County, Ohio named after him.