When Yankees think of the South, a great deal of them immediately think of the Ku Klux Klan. I'm glad that it appears people in this thread are not of that mindset.
There is a lot of culture and heritage that the South represents. More presidents, for example, were citizens of the Commonwealth of Virginia than any other state in the Union. The South contributed the separation of church and state, the model for the Bill of Rights, and the first secular universities (in the world, I believe, but certainly in the western hemisphere). And really, that's just Virginia.
As a side note, a lot of westerners and Yankees like to say that Virginia is not part of the South because of its geographic position in the mid-Atlantic seaboard. But culturally, if anything, it's the South magnified. The capital of the Confederacy was here, and there's a loyalty to Virginia that oft-times exceeds loyalty to nation. Robert E. Lee, for example, knew that in principle, the North was right. But Virginia was in the CSA, and by God, he was a Virginian. So he led Confederate armies. There's a deep, deep pride from being in Virginia that goes way back to the Colonial era. If I moved to northern Virginia today, I would not see the attachment to the Commonwealth that I feel because I grew up here for decades now. Fairfax County in particular has changed a LOT -- diverging far from Virginia proper. But since my roots predate much of that change, I still feel myself attached to the rest of it.
We've had our problems, clearly. Slavery and the resulting stain of racism probably foremost among them -- though there are complex historical factors, Reconstruction imposed by the North among them, that did a lot to encourage that.
But there is much in art and literature and lifestyle that comes from the South. The blues, a number of great writers, a relaxed state of mind, courtesy, spirituality, cuisine, family -- these are all aspect of Southern culture that those who grew up here are proud of and wouldn't trade away. The stars and bars remind a lot of us of that shared heritage.
I personally see the flag of the Confederacy in this way. It represents 260 years of culture before the Civil War even started. However, I do not fly it, because there are those who still see it as racist, and I don't want to give that impression.