Spencer died following a heart attack in a Los Angeles hospital on December 16, 2005, four days before what would have been his fifty-ninth birthday. West Wing cast mate Stockard Channing was visiting with Spencer at the time of his death. He was buried in Laurel Grove Memorial Park in Totowa, New Jersey. At the time of his death, Spencer had appeared in two of the five West Wing episodes then in post-production. Those episodes were Running Mates and The Cold. His death was subsequently written into the show as his character, vice presidential candidate McGarry, dying of a heart attack on election night. Coincidentally, the fictional character had a disconcerting history of heart problems.
An only child of a working-class family (his father was a truck driver who never finished grammar school and his mother was an occasional waitress), both of his grandfathers were alcoholics. His character on the show was also an alcoholic.
Spencer was also a regular on the televison program L.A. Law and was featured in several films including WarGames, The Protector, Hiding Out, Presumed Innocent, L.A. Law, Forget Paris, The Rock, Cop Land and The Negotiator.