Jesse L. Martin was born in Rocky Mount, Virginia, located in the Blue Ridge Mountains. He is the youngest of five sons. Martin's parents, truck driver Jesse Reed Watkins and college counselor Virginia Price, divorced when he was a child. Ms. Price eventually remarried and the boys adopted their stepfather's surname. When Martin was in grade school, the family relocated to Buffalo, New York, and the move was not an immediate success: Martin hated to speak because of his thick Southern accent and was often overcome with shyness. A concerned teacher influenced him to join an after-school drama program and cast him as the pastor in The Golden Goose. Being from Virginia, the young Martin played the character the only way he knew how: as an inspired Southern Baptist preacher. The act was a hit, and Martin emerged from his shell.

The actor attended high school at The Buffalo Academy for Visual and Performing Arts, where he was voted "Most Talented" in his senior class. He later enrolled in New York University's prestigious Tisch School of the Arts Theater Program. After graduation, Martin toured the states with John Houseman's The Acting Company. He appeared in Shakespeare's Rock-in-Roles at the Actors Theater of Louisville and The Butcher's Daughter at the Cleveland Playhouse, and returned to Manhattan to perform in local theater, soap operas, and commercials. Finding that auditions, regional theater, and bit parts were no way to support himself, Martin waited tables at several restaurants around the city. He was literally serving a pizza when his appearance on CBS's Guiding Light aired in the same eatery. While the show aired, the whole waitstaff gathered around the bar television to cheer his performance. Often, during the dinner rush, he broke out in song. When he gave his customers their dinner checks, he told them to "keep it, because someday I'll be famous!" Many of his coworkers in the restaurants continue to follow his career and are considered his early "fan club".

Martin made his Broadway debut in Timon of Athens, and then performed in The Government Inspector with Lainie Kazan. While employed at the Moondance Diner, he met the late playwright Jonathan Larson, who also worked on the restaurant's staff. In 1996, Larson's musical Rent took the theater world by storm, with Martin in the part of gay computer geek, Tom Collins. The 1990s update of Puccini's La Bohème earned six Drama Desk Awards, five Obie Awards, four Tony Awards, and the Pulitzer Prize. Martin soon landed roles on Fox's short-lived 413 Hope Street and Eric Bross' independent film Restaurant (1998). Ally McBeal's creator, David E. Kelley, attended Rent's Broadway premiere and remembered Martin when the show needed a new boyfriend for Calista Flockhart's Ally. The actor's performance as Dr. Greg Butters on Ally McBeal caught David Duchovny's eye, who then cast Martin as a baseball-playing alien in a 1999 episode of The X-Files that he wrote and directed.

While still shooting Ally McBeal, Martin heard rumors that actor Benjamin Bratt planned to leave the cast of Law & Order. Martin tried out for the show years before and won the minor role of a car-radio thief named Earl the Hamster, but decided to wait for a bigger part. With the opportunity presenting itself, Martin begged Law & Order producer Dick Wolf for Bratt's role. Wolf hoped to cast him, and upon hearing that CBS and Fox both offered Martin development deals, he gave the actor the part without an audition.

Since 1999, he has played Det. Ed Green on the popular television series Law & Order, save for a brief hiatus at the end of the 2004–2005 season while he was filming the movie adaptation of Rent, for which he reprised the role of Tom Collins.