I do not know if he moved initially in order to improve his business, but I can think of no other reason. In Malta he continued his trade, hoping to be able to make a good living catering to the travelers who frequented Saratoga Springs and Ballston Spa.
As the financial panic grew, travel to resorts slowed considerably, as did the need for a tailor. Rather than let everything collapse, Ephriam widened his horizons. In the Censusof 1850, Mr. Ellsworth is listed as a "Butcher."
Butchers in the 1800s did not work for a grocery, or even for a meat shop. There was no reliable way to keep meat fresh. A butcher worked by appointment. If a farmer needed a cow or pig butchered, he paid the butcher to come to him, do his work, and leave--often with a nice cut of meat as well as his pay.
I think we should call that "reinventing himself," not "ruined."