My initial and trite thought would be for you to send your brother away. That is clearly a non starter.
So we have to look at your brothers behaviour, tell him to get out of the cats space. He doesn't have to "try" and stroke it he doesn't have to be any where near it in fact. Why does your brother think he has to befriend it? People don't necessarilly get on with each other but can be in the same room without having to touch each other or indeed talk to each other.
Pussy cat would be confused, upset and totally disorientated if you sent her away from a loving home, There would be no frame of reference for her, she has done nothing wrong in her eyes (hissing and even scratching at an intruder, in her eyes, is perfectly acceptable) she could be protecting daddy!
Please try and rethink this attitude to your extremely loving feline...hey man she's giving you kisses....she loves you...she just aint too sure about your brov, :)