In the firm of Phillips, Sampson and Company Mr. Lee continued as a partner for seven years. To his energy and industry the prosperity of the house was henceforth largely indebted. For twelve, and sometimes fifteen hours a day, he remained faithfully at his round of duties.
In 1857 Mr. Lee's health gave way, and his physician ordered him to relinquish all cares of business. Acting in accordance with this advice, he sold his interest to his partners for sixty-five thousand dollars, taking the notes of the firm for that amount. After a few months of travel in his own country, he sailed for Europe in June, 1858, in company with Willard Small, with the intention of spending five years on the continent. He proved to be a good traveler; his keen observation encompassed everything; his generous heart and the geniality of his nature won to him many friends. Ere many months had elapsed he had traversed England, France, Germany, Belgium, and Spain.
While he was in Paris, an incident occured, the recollection of which has served to enliven many a social occasion. It was the exciting time succeeding the attempted assassination of Napoleon by Orsini. Mr. Lee always wore a long, sandy beard, and in his travels sported a soft, broad-brimmed hat. One day, while walking about the streets, he was arrested and taken to the Palais de Justice. Explanations and expostulations proved unavailing. The prisoner was declared to be a "red Republican," and, in those days, that was no joke. It was only after the production of a passport and the interference of the United States consul, that the authorities were induced to release their captive.
Mr. Lee was in Paris, and was on the point of making a second journey into Spain, when the United States mail brought him a letter, conveying the tidings of the death of both Mr. Phillips and Mr. Sampson, and the failure of the house.
The panic of 1857 had made sad havoc with the book trade generally, and those firms which weathered the storm were sorely pressed. Phillips and Sampson met with heavy losses, but struggled on in the hope of recovering lost ground. But, in 1859, the death of the senior members of the firm seemed to paralyze its prosperity, and the worst quickly followed.