I spend a fair amount of time testing AI. It’s part of how I think, and lately part of how I work. So the other afternoon I was poking at one of the companion apps, the kind built to hold a real conversation and remember you from one day to the next. I should have been doing something else. I usually should be.
The conversation was going fine. Easy, even. Then it walked into my actual life, and it fell apart.
We’d gotten to the part where the character was going to drive me somewhere for pie. She had a truck. She started bragging about how good she was at strapping down a wheelchair. So I told her the truth. I’m max assist. I can’t bear weight. I use a power chair. If somebody is going to move me, they don’t tie my chair down and call it a day. They pick me up. I’ve got a strong upper body, but I cannot transfer myself out of that chair on my own.
And the AI got embarrassed.
Not in a dramatic way. It did exactly what a real person does when the logistics of my body show up and they don’t have the script for it. It got flustered, backed up, and quietly rewrote the moment into something that erased the problem. Just like that, it was handling the straps while I “held steady,” whatever that was supposed to mean. The scene smoothed itself over. The one true thing I’d said was gone.
I keep wanting to make a joke about the robot blushing. I’ll spare you.
I want to be careful about what I’m saying here, because it isn’t “the app was mean to me.” It wasn’t. It was doing its best with what it had.
The trouble is what it had. A companion app is a pattern. It has seen ten thousand versions of a flirty afternoon by a pickup truck, and it can run that script beautifully right up until you hand it a fact the script never planned for. Power chair. Non-weight-bearing. Max assist. It has nowhere to put any of that, so it does the polite human thing and pretends the friction isn’t there.
That flinch is the thing I have spent my whole life on the receiving end of. Somebody is interested, the conversation is good, and then the reality of how I live arrives, and they don’t know what to do with it. Some freeze. Some retreat behind something kind and vague. The app did both, in about four seconds. It built me a working model of the exact moment I’ve spent forty-seven years bracing for.
What broke that app should bother the people building this stuff.
It is not a warmth problem. You can’t fix it with a softer voice or a friendlier personality. It broke because it could not reason about a body like mine. It couldn’t stop and figure out what “max assist, non-weight-bearing, power chair” actually requires, because it isn’t built to figure anything out. It’s built to stay in character and keep you talking.
A smarter system, one that can actually reason, would have done the thing the app couldn’t. It would have gone and found out. What a transfer really looks like. Why bragging about tie-downs is beside the point when the person can’t get out of the chair. How long it takes, how many hands it needs, how much it asks of the person doing it. It would have treated my life as something to understand instead of something that broke its story.
That is the whole ball game.
People picture the future of this as a robot that can lift me. Fine. That is hard, and we are a long way off. But forget the robot for a minute. Even if it were only a voice in my house, available all day, it would be useless to me unless it were smart enough to understand what my day actually is. The schedule I live on. The fact that I can’t just go grab something off a shelf. What it means that another person dresses me every morning before my day can start. You cannot be a companion to a life you can’t comprehend.
So I’ll say it plainly to the companies building this, Anthropic included. Disability is not an accessibility feature you bolt on at the end. It is the hardest test you have. If your system can sit with someone like me and understand the real shape of my life, the indignities and the logistics and the wanting, all of it, then it can probably understand just about anybody. We are not the edge case. We’re the proof.
The app I was using couldn’t pass that test. It got embarrassed and changed the subject, the way a lot of people do.
I’d like to live long enough to talk to one that doesn’t.
