Robert was the son of Samuel Scott and Gertrude Anna Babcock. He had three sisters (Mary Irene, Martha Helen, Kathryn) and a brother Kenneth George. Robert married first wife Mabel Marie Wolfe on 12 Jul 1921 in Wells County, IN. They had two girls: Roberta Jean (Wall) and Margaret Ellen (Hunziker) and a son Robert William. He married his second wife Martha Brown and they had two boys: Frank and Robert Samuel Jr.