Born May 13, 1936, in Pittsburgh, Pennsylvania to John, a government official, and Margaret, an accountant. He married Mary-Alice "Micki" Nash on June 13, 1959, with whom he has four children. Educated at Pennsylvania State University, he received his B.S. in 1958, followed by an M.A. from the University of Pittsburgh in 1960, and a Ph. D. from the University of Chicago in 1964. He is the author of several books, including Homeschooling for Excellence, released in 1987 and coauthored with his wife.
In his 1992 nonfiction account Hard Times in Paradise: An American Family's Struggle to Carve Out a Homestead in California's Redwood Mountains, Colfax chronicles his family's move in the early 1970s to an isolated homestead in the California wilderness. Prevented from teaching due to his civil rights and antiwar activism, he details the story of his family's pioneering efforts to tame the land, build shelters, and learn to survive without electricity, running water, roads, phone service, or schools. Amid the hardships and danger, the Colfaxes strove to homeschool their children; they were so effective that all three boys subsequently were accepted at Harvard University. Referred to as an "engaging story" by a Publishers Weekly reviewer, the book garnered praise from Library Journal's Cheryl Childress as well, who judged it "a well-told tale of what it takes truly to live off the land."