I'm of the opinion that we don't "own" this land. Just because we've named this country Australia, and call ourselves Australians, doesn't really mean we have the moral right to tell people to **** off.
The real issue on hand, is find an immigration policy that ensures we maximise the number of people we let in, while also ensuring our society is not disrupted and that the current citizens are kept safe.