Joseph was married February 7, 1896 in Escambia County to  Ida Baldwin Pitt.  Joseph and Ida Keller were the parents of two known children,  Vera (1896-1986) and Audrey Keller (b.ca 1904; mar. David Anton).  Joseph is buried in the Benjamin Pitt family plot.