I agree with you. The fact that it was a private conversation is what makes it a grey area in terms of workplace sanctions. Everyone would be sacked if private convos were made public.You can think how you want. But it has a boundary and it can exceed/extend into someone elses boundaries.
I.e think you co worker is hot and its a shame their sexual preference means any chance you'd have with them is null and void.
Be wary of saying it aloud is all.
Freedom of thought is still ok. Freedom of expression has never existed.