You don't get to choose what peoples intentions and implications are through social theory.
You are quite right I do not have that choice. Not what I am saying.
I regard myself, like most of us no doubt, as not being particularly racist. When I grew up the expression "that's the fellow in the wood pile" had no racist connotation for me at all. It meant no more nor less than that some overlooked fact had or would upset what had been planned. I did not even intend to refer to a person of African ancestory.
Unfortunately for me the PC police came along and said the use of the N word even in that context was offensive. Because others would take the word to be a negative reference to a person of African ancestory. Now I know that use of the N word even in my innocent context, will be taken to be racist, I am bound to accept that if I use the expression then I am being racist - whatever my purported intention may be.
The same applies to people who boo Adam Goodes. Once a fan knows that booing Goodes will be taken as being racist - that knowledge condemns the fan to being a racist if they boo Adam Goodes. It's something to do with eating from the tree of knowlege.
If you don't agree with me, try going around the USA referring to "fellows in the wood pile" and see how far you get. However innocent you claim the expression to be.