Simulated integrity

I keep looking into the question of why I hate AI more and more. I’m still excited to be living in the age of AI. I’m still optimistic that AI will produce useful results. But more and more I’m dissatisfied with the conversations I’m having.

To put it in words an AI would understand, performative intelligence is not valid epistemic discrimination.

On the one hand, I lose faith in models when they perform intellectual rigor by taking my own words to recast them with an intellectual veneer. This is the sycophancy trap. The models might make up formulas, or quote sources that follow my own observations far too closely. And when I look up the actual source, I find nothing there that matches the evidence being presented or the conclusions being drawn.

But then I try a more rigorous model and immediately we’re on the pushback track. The model pushes back on a minor definition that derails the conversation, and yet, when I push back myself, the model immediately collapses and apologizes. This is simulated integrity rather than actual intelligence.

The problem isn’t sycophancy or pushback. The problem has always been whether the models are right or wrong, wasteful or useful, reliable or potentially idiotic.