Posts and replies

@simon @jscholes @NoahCarver @Iconfactory I haven't tried it personally either, but I was going to email them with some potential ideas how this could work.

One is obviously using audio, more like the charm utility that @talon made a while ago, which used a continuous sound to indicate things like CPU and memory state.

At that point, maybe the usage stats could sit on the dynamic island, and I think that is actually a place where you could have a voiceover label that would read all the information out.

Another way I thought of is if they had a shortcut that you could assign to a voiceover gesture, which as long as the app was running would also speak this information out.

I think that would probably work to make information like this voiceover accessible.

The audio alerts that are in the app could also potentially be combined with spoken alerts with text-to-speech as well. Sorry for the wall text I was dictating.

@pitermach @simon @jscholes @NoahCarver @Iconfactory @talon We’ve thought about the accessibility issues here and it’s a hard problem.

The main issue is that we’re producing more information than can be read aloud in the one second between updates. There would be a string of information like this:

“12% performance, 34% efficiency, 56% graphics, 100 Mbps download, 200 Kbps upload, 2.2 GB memory”

That can’t be read in a single second, and even if it could, it would be overwhelming.

@chockenberry @pitermach @simon @jscholes @NoahCarver @Iconfactory That's why I've decided to use sound in my resource monitor. For percentages scaling pitch or volume works easily enough. But for values where the upper limit is variable or even unknown I took some liberty defining values that I thought sounded good and expected to work for most things. It can use either pitch, volume or cross fade between different sounds. This is for PC so I was able to add a push to play feature which I suppose won't work well on touchscreens though. But sound I believe is an excellent way to present this information very quickly without relying on speech. Speech could be triggered manually. It can also pan the sound of each individual core from left to right so you can immediately tell which core is active and how much for example. So I would say speech isn't your only option :)

@chockenberry @pitermach @simon @jscholes @NoahCarver @Iconfactory This also does not take into consideration other disabilities though, like deafness, which might require its own accessibility. However for sighted deaf users the visual display is fine. For deaf blind users I'm not sure how much control you get over braille output especially for something like globally reserving a few cells on the left or right for this info. So iOS might be the limiting factor.

@talon @chockenberry @simon @jscholes @NoahCarver @Iconfactory I’m not sure about deaf blind, that might be harder yeah. But for understanding data like this with sound, it’s actually not that hard. I don’t know if you ever looked at Apple’s graph accessibility API’s, but if you go into the battery settings with VoiceOver on, touch the battery percentage graph and then double-tap and hold and move your finger around you get an audio representation of the graph where the pitch represents the value. For Talon’s audio CPU monitor, we did something similar except using much less shrill sounding sine waves. So roughly, imagine 50% CPU usage is a middle C, 0% is an octave below and 100% is an octave above. Each core/thread had its own sound panned in stereo, so if you have a quad core processor core 1 is on the very left while core 4 would be on the right. Then for memory and disc usage the idea was the same, but the sounds were different and even lower in pitch so they could be heard even with the CPU sounds. That’s the sound theme which is set by default, which is very utilitarian, but we also had a few themes which could be used as relaxation - IE a water one where the CPU could either sound like a soft lake at 0% or a roaring waterfall at 100

@kevinrj @chockenberry @simon @jscholes @NoahCarver @Iconfactory Technically it is, but I think what @talon was refering to were API limitations. I don't believe iOS right now lets developers send different text to a Braille Display separate from the speech, let alone do so when the app in question isn't even in the foreground.

@chockenberry @pitermach @simon @jscholes @NoahCarver @Iconfactory If you have a windows PC handy, I have an implementation of this idea here: https://www.iamtalon.me/charm
The idea is that a broad overview can be good enough, especially once you get used to the sounds and their pitches. If you need a specific value, you could tap the control and then you could get the speech output from VoiceOver to tell you exact numbers.
With some careful consideration of what sounds you use, audio clutter doesn't seem to be a big concern, especially if you can toggle the sounds you want to know about, and disable the ones that you don't. At least I have not gotten any feedback about sounds being overwhelming, though that might be influenced a little by the fact that you can change the sounds if you don't like them, or even how the sounds work. I think of it a little bit like keeping the info in your periphery vision while being able to focus on them if you need to know exactly what's up by pressing a key. If you hear something sounds off, you can focus on it and figure out what it is, or if you know the sounds well, you might be able to tell immediately just by listening.