Palma grew up in New York City, and worked as a mortician in the Provenzano Funeral Home, owned by his parents. Eventually, Hollywood called, and Palma headed west. He joined the stock company at Columbia Pictures in 1937, and played scores of bit parts over 30 years.
With his lean build, brushed-back hair, and unassuming appearance, Joe Palma almost always played incidental roles. He was usually in the background, and at most, he would be given only a few lines of dialogue. In the 1945 Three Stooges comedy Beer Barrel Polecats, for instance, Palma plays an angry convict who dares Curly Howard to punch him in the nose.
Palma can be glimpsed in all kinds of movies, including crime dramas, musical comedies, costume epics, westerns, serials and two-reel comedies. Several of his many roles consisted of the following:
Palma's largest speaking role is probably in the Schilling & Lane short Training for Trouble, in which Palma attempts a Jewish dialect: "This is Goldstein, Goldberg, Goldblatt and O'Brien, booking agents. O'Brien speaking" (a gag borrowed from the Stooges' A Pain in the Pullman).
Today, Palma is best known as Shemp Howard's posthumous double. In 1955, Three Stooges member Shemp Howard died of a sudden heart attack. At the time, the Stooges still had four short films left to deliver on their annual contract with Columbia Pictures. By 1955, budget cuts had forced them to utilize stock footage from previous shorts as a matter of survival. As a result, the Stooges managed to complete the four films without Shemp. To do this, Palma was made up to resemble the late Stooge, complete with wig, and would be filmed only from the back or side. On occasion, Palma was required to add a brief line of dialogue or sound (most notably in Hot Stuff). The few new shots Palma appeared in were then edited together with the recycled footage containing the real Shemp, and "new" films were born.
Palma spent his last years in the entertainment industry as an assistant to Jack Lemmon. He appears as "Mr. Palma", the mailman, in Lemmon's 1964 Columbia comedy Good Neighbor Sam. His final film appearance was as a butcher in Lemmon's 1968 Paramount film The Odd Couple.