Son of John and Abigail (Knight) Morrill. He had 6 siblings; Mary, Sarah, Abigail, Hannah, John and Samuel.
Joseph married Katherine Smith, daughter of Isaac and Nancy Codman, on 14 Apr 1811 in Weare, New Hampshire. They had 15 children; Eldridge Gardiner, Gardiner "Kilbourn", Elnora, Jeal, Katherine 1820-1821, Katherine N 1822-1823, Henry Codman, Nancy Codman, Charlotte G, Katherine 1829-1830, Aheny, Frances Ann, Josephine M, Achisia and Joseph H.