I can relate. Sadly I don't have much in the way of advice.
There's this weird push-pull that I have between wanting to be a financially successful writer, and wanting to make art.
The two aren't mutually exclusive, but nowadays it's very hard to do both. Entertainment in general, whether it's music, writing, filmmaking, there's a "cool kids club." You know, there are gatekeepers. Succeeding outside the system is very difficult. In terms of gaming, guys like Kojima are typically only able to outgrow the system by having worked in it for so long. And not all are so lucky; very few, in fact.
A little fact about Michelangelo: he didn't like painting, and hated it when popes and other powerful rich people would pay him to paint stuff, like the Sistine Chapel which actually caused him to temporarily go blind from looking upward so much for prolonged periods of time. Even for the greats like Michelangelo, the "system" was always a problem.
I don't like thinking in terms of this amorphous "audience." I don't want to think in terms of boxes, like genre. I don't want to conform my story to random cultural dictates. I want to write. I wish for people to read my writing and not care about the category, but whether or not it means anything to them, if the story in and of itself, by itself, had any value to them.
Of course, I don't want to sound too pretentious. Of course I am aware in the general sense of romance, sci-fi, fantasy, and what's generally expected. I know that I tend to like certain genres of music more than others. But time and time again I think to that quote: "The work, which becomes a new genre itself, will be called... Cowboy Bebop."
I'm not trying to do something avant garde. I just do not wish to chase trends, or to sacrifice something genuine about my story for the sake of appeasing some agent or critic. Obviously I'd like for as many people as possible to enjoy my writing, but that's only after sticking to my guns.
Ultimately, I have to write for myself. If I can't read my own writing, if I can't re-read it, if I can't be excited about my own writing, if I don't feel confident about my own writing, then none of the rest matters. I find it very hard to clearly see and judge my own writing when I am in a bad headspace, so in such times I have to step away, or I will beat myself up and spiral really, really badly.
You've finished two books, so you're already ahead of me. But I wonder about tying our worth to whether or not people will buy our art or whatever... That seems to basically be the same as turning ourselves into a product, since I see writing as an extension of myself. Tolkien originally wrote The Hobbit for his own children, and nobody else. Rembrandt made a distinction between art that was for his own creative interest, versus things he produced with a commercial intent; even when he was poor, in immense debt, and basically homeless, there were paintings and etchings that (as far as we know) he never made any effort to sell.
EDIT: I realize now that you never mentioned money or finances factoring into your assessment of being a "failure" in your original post. That's my bad, I got a little carried away. However, that being said, I understand the frustration of feeling like you're stuck, trapped, unable to improve. You define success for yourself. It's easier said than done, but... well, let's just say comparing yourself to other writers is risky business, not something I'd encourage unless you can really narrow down the comparison to something specifically technical and compartmentalize it.
Our emotions can be fuel. But that fuel can just as easily flood the engine. I forced myself to write 50k words for the last official Nanowrimo, and since then haven't written any fiction let alone finished that specific WIP. In the past i've sworn off writing, found myself in a similar place as you are now, and eventually ended up writing again... and not because of anything I did or didn't do, it just happened. Maybe the same will be true for you, I cannot say for certain. But the way you feel is certainly valid. I've found that it is seldom (note: I didn't say "never") useful to wonder whether it's valid to feel the way that we do, as that often leads to a much greater psychological or even existential crisis, as opposed to acknowledging you feel x or y and being left with only the issue of what to do about it. People may not relate, and I suppose are even free to disagree with the way you feel about something, but in the end nobody else is in a position to say you don't have permission to feel a certain way. And if you say to yourself "I am not allowed to feel this way" but you feel that way anyhow, then all you've achieved is being at civil war with yourself.
