He met his wife, Tipper, at his senior prom; Tipper came to the event with another date, the son of a federal judge. Al and Tipper married in 1970 at the National Cathedral in Washington, D.C.
He attended Harvard University, where he and actor Tommy Lee Jones became friends. He initially wanted to be a novelist but graduated with honors in 1969 with a bachelor's degree in government.
