Christians disliked the teaching of the Mormons and most people were against the view of having more than wife as being acceptable. Mormons generally cut themselves off from ordinary society, and some Mormons (Danites) attacked and robbed others in the city.
In general, people were afraid of their growing numbers and feared they would overrun the government, this was a particular concern as they were viewed as very low class citizens. Joseph Smith set up a bank in New York, but during the depression in 1837 this collapsed, and the Mormons were run out of New York city.