Definitions

Webster's:
Ref`u`gee´
n. 1. One who flees to a shelter, or place of safety.

Encarta:
Refugee, person who has fled or been expelled from his or her country of origin because of natural catastrophe, war or military occupation, or fear of religious, racial, or political persecution.

WordNet:
Noun 1. Refugee: an exile who flees for safety.

A synonym is evacuee.
It isn't the "dirty" word that many would like to make it.