I think you hit on my reasons, SE. First off, realize that I'm using hyperbole to make my point. However, I really think authors need to understand some things that make writing--or other careers in the arts--different from your standard job.
It is a matter of degree. However, it's a severe matter of degree. Ask yourself this regarding other positions: Can someone else do the job? My roommate Earl is very valuable to his company. He writes code, and makes their products work. However, can someone else do his job? Yes. There are probably lots of people who can do the job as well as he does. That doesn't make him less valuable, but it does make a big difference to the application process.
Writing is different. I sincerely believe that nobody else can write these books. I offer something that NOBODY in the world besides myself can offer. I think this is part of an essential trait an author must have to be successful commercially.
There are around 300 million people in the US. I believe I write better fantasy books than about a half dozen of them. That's a pretty big 'difference of degree.'
I talk about the value of arrogance because I know what it's like to be repeatedly faced by statistics regarding publication, as well as rejection letters. To get a regular job, confidence is necessary. To succeed in this field, I think your confidence needs to be as disproportionately large as your chances of failure when compared to a different job.
This means arrogance. I use the word flippantly, kind of like Oscar Wilde used 'lying' in his famous treaties. However, I think there's an element of truth to it. If you look at the statistical chances of your getting published, it takes reckless, even foolish, levels of confidence to keep going. This is how I define arrogance--confidence beyond what is logical, right, or even normal.
Plus, I just like to argue.
