Part of the inherent process of Exaltation is knowing what you are. You might well conclude you are a god, you are a demon, you are invincible, but as soon as someone says, "You're a Dawn Caste, Exalted of the Unconquered Sun," somewhere, something inside your character's head nods and says, "Yes. That is what I am." When they stop to think what they are, they think, "I am a Lawgiver, an ancient justicar, a Dawn Caste."
You don't feel unrighteous - quite the opposite, you are possessed with righteousness (and rightly so, as the arbiter of good and holy in the universe has just given you the spiritual equivalent of a wink and a buddy Jesus thumb's up). You might cleave to the Immaculate tenets of the Solar Exalted believing themselves holy and convince yourself that you're not, but you're convincing yourself. You might scream, "I'm an Anathema!" after you get that mental answer, but there you are. Jencir of Chanos knows what he is - he's desperately trying to convince himself that he's an Anathema, but that argument becomes less convincing every day, as his dedication to the Realm and to righteousness has only become stronger, not weaker. Most of his angst comes from the fact that his mother would call him a demon and spit on him if she knew what he truly was.
How your character reacts to that voice inside their soul, to those feelings, is where the drama comes in. They may very well deny it, they may spite it, they may say, no, I am more than that, and pursue some course of transhumanism. And they're welcome to do that, because that voice never goes away. But they're explicitly in denial, because from the moment of Exaltation, what they are is screaming [at] them clean [in] the face, and they have a good idea of what they are.
It's something that cropped up in 1e that I didn't invent, but I did codify it, if not entirely explicitly - for the precise reason you listed above.
Never underestimate an Exalt's capacity for self-deception, however. To put it bluntly, there is a reason why there's a page-long caveat in MoXP Infernals asking you to think long and hard about whether or not the Yozis breaking out of Malfeas is a good idea to include in your game. To put it another way, and to use an example from above, I definitely approve of a group of Solars thinking they're the Chosen of the Lame-Ass Wolf-God who is pulling the wool over their eyes, for about three sessions. Session 4, they hit a trigger that tells them what they are, it sets off a chain reaction amongst the Circle, and they go back to the wolf-god baying for his blood. Then, they spend the next three or four story arcs wondering who to trust in a world full of deception and lies.
Is that better, or worse, than believing you're Chosen of the Lame-Ass Wolf-God for the entire chronicle?