Brilliant? Really? Huh.
Honestly (and no offense to you intended; some offense may be taken by Mr. Cage if he ever reads this), it seems to me that it takes one of those crazy geniuses for anyone to take this seriously. Any idiot can scrawl out 4'33" of silence. Any idiot can abstain from playing a musical instrument. And if any idiot did, nobody would give him the time of day.
If I go to a concert, if I buy a CD, I want music for my money, not environmental noises. They do sell those, too, but nobody pretends they're music, and nobody calls them brilliant.
Seeing as how he's been dead for 18 years, I don't think you have to worry about him reading this 
Honestly, though, I would have to agree. Although my reaction to someone saying "any idiot could do that" is usually, "well, why don't you?" Not meant to be insulting - it's something I learned in art school; when someone looks at your work and says "I could have done that" or "any idiot could do that", ask them why they didn't/don't.
The difference here, though, if there is one, is that 4'33 isn't anything. The same experience could be had sitting by yourself and not doing anything. It seems really pretentious to me to take doing nothing, literally, and call it art.
Ah. Sorry for defaming the dead. :/
My answer to your "why don't you" would be twofold. First, because it's silly, and second, because it's one of those things that can only be done once. (Runner-up answer: "Because I am no idiot."

)
Actually, the first part of that retort applies generally. If somebody says "Heck,
I could do that," what they're expressing is that if they, who have no artistic talent, could have produced a given artwork, then it wasn't worth producing. "So why didn't you?" "Because it
sucks."
I will agree that the "anyone could do that" mode of artistic criticism is a poor way to articulate a low opinion of a work. However, for someone who doesn't know much about art, it's about all they have.