Accardo studied violin in the southern Italian city of Naples in the 1950s. He gave his first professional recital at the age of 13 performing Paganini's Capricci. In 1958 Accardo became the first prize winner of the Paganini Competition in Genoa.
He has recorded Paganini's 24 Caprices (re-recorded in 1999) for solo violin and was the first violinist to record all six of the violin concerti by Paganini. He has an extensive discography of almost 50 recordings on Philips, DG, EMI, Sony Classical, Foné, Dynamic, and Warner-Fonit. Notably, he has recorded an album of classical and contemporary works in 1995 on Paganini's Guarneri del Gesù 1742 violin, Il Cannone Guarnerius.
Accardo founded the Accardo Quartet in 1992 and he was one of the founders of the Walter Stauffer Academy in 1986. He founded the Settimane Musicali Internazionali in Naples and the Cremona String Festival in 1971, and in 1996, he re-founded the Orchestra da Camera Italiana (O.C.I.), whose members are the best pupils of the Walter Stauffer Academy. He performed the music of Paganini for the soundtrack of the 1989 film Kinski Paganini. In the 1970s he was a leader of the celebrated Italian chamber orchestra "I Musici" (1972-1977).