I agree with many of your points. And the same is true for any focused researcher - that they are often in the position of having to prioritize what they perceive as productive issues over non-productive ones. Otherwise, all the time evaporates with no tangible result.
Just as counterproductive - to my way of thinking - would be a policy not to consider or respond. As Velikovsky rightly pointed out, many of the great leaps in human understanding have been initiated by someone from 'out of the field'. Any programmer knows the benefit of having a non-technical bystander ask naive questions about a programming problem. It forces you to reconsider assumptions.