As I am to speak of Mr. Sumner in our personal relations, which for thirteen years before his death were intimate, I shall use some words of preface. Never on more than two occasions did we have differences that caused any feeling on either side. Mr. Sumner was chairman in the Senate of the Committee on the Freedmen's Bureau, and Mr. Eliot was chairman of the Committee of the House. A report was made in each House, and each bill contained not less than twenty sections. Each House passed its own bill. A committee of conference was appointed. Its report was rejected. I was appointed a member of the second committee.
I examined the bills, and I marked out every section that was not essential to the working of the measure. Four sections remained. I then added a section which provided for the lease and ultimate sale of the confiscated lands to the freedmen and refugees. President Johnson's restoration of those lands made that section non-operative. The committee, upon the motion of General Schenck, transferred the jurisdiction of the Bureau from the Treasury to the War Department. The bill was accepted by the committee, and passed by the two Houses.
When within a few days I was in the Senate Chamber, Mr. Sumner came to me, and said in substance: "The Freedmen's Bureau Bill as it passed is of no value. I have spent six months upon the bill, and my work has gone for nothing. You and General Schenck cannot pretend to know as much as I know about the measure."
With some feeling, which was not justifiable, I said: "I have not spent six hours upon the measure, but after what you have said I will say that the fifth section is of more value than all the sections which you have written." I did not wait for a reply. The subject was not again mentioned; our friendly relations were not disturbed, and it is to Mr. Sumner's credit on the score of toleration that he passed over my rough remarks, even though he had given some reason for a retort.
My next difference from Mr. Sumner was a more serious difference, but it passed without any break in our relations. He had not acquired the church-going habit, or he had renounced it, and my church-going was spasmodic rather than systematic. Thus it became possible and agreeable for me to spend some small portion of each Sunday in his rooms. The controversy over Mr. Motley and his removal from the post of minister to Great Britain excited Mr. Sumner to a point far beyond any excitement to which he yielded, arising from his own troubles or from the misfortunes of the country. To him it was the topic of conversation at all times and in all places. That habit I accepted at his house with as much complacency as I could command. Indeed, I was not much disturbed by what he said to me in private, and certainly not by what he said in his own house, where I went from choice, and without any obligation to remain resting upon me. In all his conversation he made General Grant responsible for the removal of Motley, accompanied, usually, with language of censure and condemnation. On two occasions that were in a measure public, one of which was at a dinner given to me by Mr. Franklin Haven, a personal friend of twenty years' standing, when he insisted upon holding the Motley incident as the topic of conversation. On one of these occasions, and in excitement, he turned to me and said: "Boutwell, you ought to have resigned when Motley was removed."