Is it wrong of me to have a laugh about this incident...?

No it is NOT wrong...tell him you were laughing at the situation, NOT HIM. He should understand; but if not...then you guys are not each other's type...sorry. I mean he is from Pakistan...sorry.