James was married first to Clarinda P. Tate, and second to Sophia Lysle. He settled in Callaway county in 1823. The Auxvasse Presbyterian Church was organized at his house that year, and it was the first organization of that religious denomination west of St. Charles.
