Of course, that's because the problem you have is not with immigration per-se, it's with party politics. You fear has nothing to do with immigrants, but how they'll potentially vote should they ever become citizen, in light of how your current party labels them.
It's understandable, but completely superficial and shortsighted. Immigration will continue happening in whichever ways (legally preferably), and there won't be any IQ test involved. AFAIK, not a single nation requires IQ tests for immigration. It's the wrong barometer too. At least with legal immigration, the idea is that it fills a gap in workers demand. You don't need high IQ for every one of those jobs, what's sought after is know-how and experience.