A lot of people usually comment things like: "It should be the screen readers responsibility to figure out what's on screen. Do some OCR and image recognition and whatever and boom!"
Think about it. That won't work. Not only are there many different fonts that OCR just simply cannot read, but also things like interacting with those controls present a major hurtle at that point. Is it a checkbox, a button, or a button checkbox hybrid? Is it a slider? Is this a fancy button or a turnable knob? What about scrolling text?
Is this syntax highlighting or spelling errors or both? Or does it just mean I selected something? What about if I use another app? We can infer what things mean from context, but can OCR and image recognition? Does it know this is a code editor or a text editor? What if it's Emacs and it's literally everything?
This is why accessibility API's exist. There's a lot of context that would be lost if there was just OCR and image recognition.
iOS has this exact thing using artificial intelligence and while that's awesome in about 75 percent of cases it still fails to do its job properly. It helps getting around quirks, but I haven't found an app that was completely inaccessible that suddenly worked flawlessly. Don't get me wrong, I'm very much happy that it exists and it's super helpful. But there's just no way it will replace any other traditional accessibility in the state it's now.
I was never able to explore screenshots for example, but now I can. I can even look at memes and most of the time, the image explorer does a pretty good job. But translating UI's like that just... doesn't work yet. Or maybe ever.
@talon totally on board with all of these but just wanted to add something as an ai researcher who knows a decent amount about OCR etc. This is unlikely to work well enough for actual use for a long time and personally I think using OCR en lieu of actual accessibility is a terrible idea please don't do that.
As another aside this kind of "not my job"-ism in software really annoys me, we all have to be part of the solution!
@secretlySamantha I understand both sides to a degree. Accessibility API's and accessibility in general can be difficult to wrap your head around, especially if they provide no benefit to you as the developer. Even as someone that needs them I'm still regularly confused by how it all works, especially in a cross platform manner. Luckily I was made aware of https://www.accesskit.dev which hopes to solve this problem and I really hope people will pick it up.
Still, the fact that web tech based apps are usually more accessible than "native" apps is somewhat disappointing.
@talon Here's the thing though it does give you a benefit as a developer. Because if your GUI is easier to read for computers then it's much much easier to test properly which devs should be and in most cases are already doing. I'm really surprised that people don't seem to realise this. (sorry if I'm being a bit intense, this is not necessarily directed at you, it's just a thing that really confuses me)
@talon again, sorry if I'm being a bit intense, I really do just agree with pretty much all of your points. it does feel very weird that web apps are somehow at the forefront
@secretlySamantha No worries. I mean you're right. It would help with testability and usability both. It's an excuse for sure, especially for well established GUI frameworks and libraries.
A fun, happy little Mastodon/Hometown instance. Join us by the fire and have awesome discussions about things, stuff and everything in between! Admins: @Talon and @Mayana.