To be a pro rugby player or even play it at a high level, you do need top tier athleticism combined with a relatively large physical build. Show me all the professional rugby players that stand 5'6" 150lb? Furthermore, rugby's rule set is rather complicated compared to basketball and soccer and it's a more dangerous game than both. I should've been more clear that "accessibility" entails a variety of elements, from cost of entry, to physical demands (i.e. what kind of physical build does it take it to excel), to injury severity, to learning curve of the rule set. Basketball shines on the accessibility scale in 3 out those 4 categories.
I think another thing worth mentioning that plays a role in sport's growth is the backyard/driveway factor. Basketball is the team sport that's most fun to play alone, with its "equipment" (a ball and basket) easily accessible, installed and maintained. Sports like rugby, football, baseball, etc you generally need another person to help you properly practice or expensive equipment that can't be easily set up in a driveway/backyard (a batting cage). Soccer has this going for it, as well. You can practice your dribbling, touch, kicking by yourself. Someone who wants to play linebacker can't really practice anything football related by himself in the backyard (unless he has a tackling sled, which is expensive) outside of working out. QB is really the only position you can somewhat practice alone (throwing a ball through a tire and the like).
I know you want to try to prove soccer ascended to the world's number 1 sport because it's intrinsically more "fun," but the cricket argument knocks that argument down. Again, a combination of factors go into making anything popular. Accessibility, timing, spread by cultural/economic power. Golf is bigger than bowling worldwide, yet I would bet the majority of people find bowling more "fun."
Also, anything is fun if you keep an open mind and learn that something's particular nuances, instead of just dismissing it with shallow arguments like "it's boring."