I don't dismiss it as impossible. I just don't see any evidence of alien influence sufficient to prove such a large assertion.
There have been accounts going back thousands of years that the sun is a flaming chariot, being driven by a magic man. That doesn't mean the nearest star is a flaming chariot.
You have taken some vague evidence, and used cognitive bias to see "visitors from other worlds".
Basically, the same logical flaw that shows "visitors from other worlds", is used to prove "God did it", i.e. the argument from ignorance.
The bigger the claim, the better the evidence required to accept that claim as true, for any rational person.
I have seen the evidence presented for this ancient alien theory, and it is about as weak as it gets, in terms of actually supporting the theory it is supposed to back.