Let's look at relevant starting guards around the league.
Worse rookie seasons than Castle: Garland, Jrue, Derrick, Brown, Brunson, Suggs, Herro, Maxey, SGA, FVV, Jamal Murray, Harden, Conley, Lavine, DDR, Booker, Scoot, Dejounte.
Better rookie seasons than Castle: Mitc ( 21 before his first NBA game), Lillard (22 before his first NBA game), Cade, Trae, Luka, Curry, Morant, Kyrie, Fox.
Around the same level as Castle: Haliburton, Lamelo.
The only guards/playmakers who were better than Castle in their rookie year were all guaranteed offensive engines and Castle is easily a better defender than all of them.
Considering that his projected playstyle and offensive output should be very reliable, I don't see how he misses out on all-star and all-NBA selections in his prime, after Fox falls off.