From the Personal Memoirs of U.S. Grant:
While a citizen of Missouri, my first opportunity for casting a vote at a Presidential election occurred. I had been in the army from before attaining my majority and had thought but little about politics, although I was a Whig by education and a great admirer of Mr. Clay. But the Whig party had ceased to exist before I had an opportunity of exercising the privilege of casting a ballot; the Know-Nothing party had taken its place, but was on the wane; and the Republican party was in a chaotic state and had not yet received a name. It had no existence in the Slave States except at points on the borders next to Free States. In St. Louis City and County, what afterwards became the Republican party was known as the Free-Soil Democracy, led by the Honorable Frank P. Blair. Most of my neighbors had known me as an officer of the army with Whig proclivities. They had been on the same side, and, on the death of their party, many had become Know-Nothings, or members of the American party. There was a lodge near my new home, and I was invited to join it. I accepted the invitation; was initiated; attended a meeting just one week later, and never went to another afterwards.
I have no apologies to make for having been one week a member of the American party; for I still think nativeborn citizens of the United States should have as much protection, as many privileges in their native country, as those who voluntarily select it for a home. But all secret, oath-bound political parties are dangerous to any nation, no matter how pure or how patriotic the motives and principles which first bring them together. No political party can or ought to exist when one of its corner-stones is opposition to freedom of thought and to the right to worship God “according to the dictate of one’s own conscience,” or according to the creed of any religious denomination whatever. Nevertheless, if a sect sets up its laws as binding above the State laws, wherever the two come in conflict this claim must be resisted and suppressed at whatever cost.
The great bulk of the legal voters of the South were men who owned no slaves; their homes were generally in the hills and poor country; their facilities for educating their children, even up to the point of reading and writing, were very limited; their interest in the contest was very meagre–what there was, if they had been capable of seeing it, was with the North; they too needed emancipation. Under the old regime they were looked down upon by those who controlled all the affairs in the interest of slave-owners, as poor white trash who were allowed the ballot so long as they cast it according to direction.
I am aware that this last statement may be disputed and individual testimony perhaps adduced to show that in ante-bellum days the ballot was as untrammelled in the south as in any section of the country; but in the face of any such contradiction I reassert the statement. The shot-gun was not resorted to. Masked men did not ride over the country at night intimidating voters; but there was a firm feeling that a class existed in every State with a sort of divine right to control public affairs. If they could not get this control by one means they must by another. The end justified the means. The coercion, if mild, was complete.
After the war, during the summer of 1865, I travelled considerably through the North, and was everywhere met by large numbers of people. Every one had his opinion about the manner in which the war had been conducted: who among the generals had failed, how, and why. Correspondents of the press were ever on hand to hear every word dropped, and were not always disposed to report correctly what did not confirm their preconceived notions, either about the conduct of the war or the individuals concerned in it.