Years ago, on a trip to Japan, I came across a swastika. Dozens of them, actually, in museums across the country. I was shocked, what Westerner wouldn't be?
No doubt this has happened to many Western travelers in Asia, and no doubt many have gotten the re-assuring answer from tour guides or friends: don't worry, it just means "good luck." Buddhists have been using it as a symbol for luck for more than 2,000 years.