In no particular order:
Bill Russell: Invented defense in the NBA.
Nate Thurmond: Dominant shot blocker and rebounder in the age of great centers
Dennis Rodman (when he felt like it) - can defend perimeter and the post equally well. Volume rebounder, but lost the desire to do the dirty work and seeked individual glory at the 2nd half of his career.
Scottie Pippen - One of the best perimeter defenders of all time, if not the best. Curiously never won DPoY. Can shut down anyone from PGs to SFs, and even smaller PFs. Long arms, quick reflex, and amazing lateral quickness.
Gary Payton - Lots of steals, and covers his man like, well, a glove. Quick hands and lateral quickness coupled with his defensive fundamentals make him a virtual wall on the perimeter.
Hakeem Olajuwon - Great help defender, shot blocker, and man to man. Quick and agile.
David Robinson - Similar to Hakeem in many ways, but even quicker. Can cover anyone from SFs to Cs. Long arms, great timing, and his strength allows him to guard the strongest centers (Shaq).
Bruce Bowen - Again, never won DPoY, but his perimeter defensive skills are both controversial and effective. Sticks to his man like glue, and tires him out over the course of a game.
Michael Cooper - The only person who ever effectively guarded Larry Bird 1-1. Never quits.
Dennis Johnson - Underrated defender in many ways where he just hawks the PG and distrupts the opposition's offense from the get go.
Sidney Moncrief - Quick and agile, while strong enough to handle any variety of perimeter players.
Bobby Jones - 8 All-Defensive teams, enough said.
Joe Dumars - Jordan's arch nemisis. Never had any impressive defensive stats, but similar to Bowen but without the controversy.
Ron Artest (when sane) - Strong and quick, as well as crazy. Made his name by cracking Jordan's ribs during a summer pick up game.
Notable exceptions:
Dikembe Mutombo: probably the most overrated defender of all time. He was a great weak-side help shot blocker and rebounder, but his man to man defense is so-so due to his awkward mobility (except on Shawn Kemp, for some reason), but somehow had 4 DPoY.
Ben Wallace: Also has 4 DPoY, but probably deserves only two of them. I would choose Hakeem, Robinson, Russell, Thurmond, Wilt, Wes Unseld, Alonzo Mourning and even Mark Eaton over Ben Wallace on the defensive end (and defensive end alone) over the duration of their careers.
Alvin Robertson: Steals a lot, but gambles on passing lane. Can be exception on man to man due to his combination of speed and strength, but again, gambles way way too much.
John Stockton: All-time steals leader, and possibly one of the dirtiest players of all time, but his lateral quickness is suspect and always seemed to have problems with ultra-quick guards (see Iverson, Tim Hardway).