Alma Routsong was born in Traverse City, Michigan, in 1924, and graduated from Michigan State College in 1949. Her pen name, Isabel Miller, is the combination of an anagram for "Lesbia" and her mother's birth name. Her first novel, A Gradual Joy, is the story of a young married couple, MSU students living in a quonset hut apartment.

In 1963, Isabel Miller moved to New York and began writing her best known book, A Place for Us, printed in an edition of 1,000 copies paid for and sold by the author. With this title, based on a true story of a 19th-century couple from New York state, Miller began her career as lesbian novelist. In 1971, the novel won the first annual Gay Book Award of the American Library Association. Under its later title, Patience and Sarah, Miller's novel became the most-cherished lesbian love story of all time. The title story in A Dooryard Full of Flowers and Other Short Pieces is a sequel. Miller died in 1996, shortly before her last novel, Laurel, was published.