I would say that the main objection to 'alien' intervention is that it devalues the skills and technical abilities of our ancient ancestors.
It suggests that they were too stupid or too inept to produce anything of value themselves, and had to rely on beings from on high to help them. The implication of this sort of thinking is that because we don't fully understand how they constructed their sites, they couldn't have done it without help.
Is that really a fair way to look at the legacy of the Ancients?
My perspective begins with flat statements made by the cultures themselves, based on cross-compared evidence that demonstrates the reasonableness of their view. To my way of thinking, that's the very definition of fairness.