It doesn't surprise me that a lot of people believe good defense is physical. But it does surprise me that a lot of Spurs fans currently still believe that, since the Medium Three Spurs were not very physical at all. A guy like Danny Green proved on three teams he could be a really good defender, and he's never been a guy who absorbed a lot of contact. DPOY Kawhi was the same though he was strong enough to where it didn't matter. Very few players in the league use power offensively. The level to which those M3 teams could shut down opposing players was insane. They were very often not as big as the guys they checked. Height and weight simply aren't that important. Positioning, body control and anticipation do the bulk of the work. That's even true for massive mismatches like Mills on Durant, except at that point literally just shooting over the guy can work. But the 6-5 to 6-7 difference doesn't grant that.
I'm not against the team getting a big defensive four and putting him on Kawhi at all, of course. But it wasn't surprising to me that someone Walker's weight did fine and someone of Johnson's struggled more. But I am talking to the same people shocked --- SHOCKED -- when guys like Poeltl or Splitter don't just back down PGs in the post.