Not necessarily. Scarlett Witch and Quick Silver are Magnetto's kids. but that could simply explained that the X-men continuities of the past 10 movies (13 after next year) take place in one world where mutants and X-men exist but no Avengers and the Avengers exist in a world where there aren't any mutants. Fox and Disney having to separate the X-Men and the Avengers makes it work out.
Quick Silver was apart of both the X Men and the Avengers but that is probably the only issue that I can think of.