YES and NO in English are not the same as HAI and IIE in Japanese, when you're answering a negative question:
Isn't it raining?
Yes, it is (raining). ---> the sentence is POSITIVE
No, it's not (raining). ---> the sentence is NEGATIVE
AME FURIMASEN DESU KA?
HAI -> you are right, it is not raining.
IIE -> you are wrong, it is raining.
