I find the story, as well as many aspects of the writing itself, to be remarkably compelling. My main issue with ASoIaF, is the characters.
As of A Feast for Crows, there are perhaps three characters left alive that I actually like. Their stories fascinate me far more than the rest, which means I have to slog through many large sections filled with characters I hate or simply don't care about in order to finally get back to a chapter with Jon or Arya or Daenerys. And Martin's habit of killing off his characters, in particular the ones that readers seem to like the most, prevents me from caring too much about someone else. I don't know of the latter has happened to anyone else while reading, but for me the violence and realism of 'anyone can die' has backfired - the ony time I care anymore is if it's one of the characters I like, and there are so few of those left.
When has Martin ever killed off a character that ppl may have really liked and grown attached to? Even Eddie wasn't particulary interesting as far as characters go. As for his son, Martin didn't even give him one POV page so readers barely even knew him before he was killed. And did anyone really care that Joffrey and the Hound died? I really don't understand where this myth got started that Martin is willing to kill off anyone. Truth be told, Martin hasn't killed off anyone that readers could possibly have grown attached to and a majority of the ones who are dead deserved it.
Okay... potential spoiler warning...
I'm serious.
Different people become attached to different characters. I became attached, quite quickly, to pretty much all of the Starks - although Sansa lost me, for the most part, about halfway through A Game of Thrones. I've started liking her a bit more since.
I found Eddard very interesting, personally. Eddard and Robb were among two of my favourite characters in the beginning, and I admit a fondness for Catelyn as well. Oberyn Martell, Khal Drogo, Jeor Mormont, numerous other minor and non-viewpoint characters that, unfortunately, can't remember the names of because it's been how many years since A Feast For Crows came out, and I haven't picked up one of the books since.
A character need not be a POV in order for a reader to become attached to them. Yes, it made the impact of some deaths incredibly powerful, especially Eddard and Robb, but after a while I just got tired of it. And where Martin got the reputation... well, by doing it. Go to ASoIaF forums, and you'll find debate threads about who's going to die next, with the more likable characters considered the most likely.