edited to add: over the past few weeks Apple and their accessibility approach has come up a few times on HN, and when I responded there have been so many questions that I really really wanted to answer. However, not had the energy to be able to respond in a timely manner and certainly not quickly enough and set the article is still on the front page of HN. Basically because quadriplegia.
So I know that there are are a lot of questions about use my computer, and that's largely because The people on HN are genuinely and wonderfully curious.
The question that seems to fascinate able-bodied engineers goes something like this: "imagine coming home and finding somebody had removed every switch, button and lever from every device in your environment; What would you do?" That's the question I had to answer before I could get to where I don't get a job and have to start my own company as a result!
So my question is how best do people think I should share this information? Blog post, Twitter thread (eww) or messenger pigeon?
I would imagine a series of blog posts that you then turn into a book? I love your question about “...what would you do?” It reads like a writing prompt.
A podcast I listen to (maybe CodingBlocks.Net or ATP?) talked about a dev who uses a camera to track a shiny bit of tape over the bridge of their glasses as a mouse and speech (with their own phonetic alphabet variation) for typing and is able to create software at the same pace as an able bodied person - I’d love to hear all of those stories.
Could design a website that was the equivalent of "imagine coming home and finding somebody had removed every switch, button and lever from every device in your environment; What would you do?"
i.e. where "able-bodied" visitors would have to figure out how to interact with the site without all their usual means.
If you designed it right and maybe even gamified it (in a good way that doesn't trivialize, that challenges you to learn), it might entice young and old to explore what it means to have to interact with the world without the benefit of X.
A series of blog posts could serve as in invaluable resource for us caring but ultimately able-bodied code monkeys to refer to. If you go ahead with those, post them here. :)
If you could find a CHI researcher to work with it would make an amazing case-study for a research paper, and that would give it some longitivtiy as well as be a good basis for a more compact blog post.
I would just like to say that there isn't a company on the planet making consumer electronics that leans in harder than Apple does, as a quadriplegic I can get an able-bodied monkey to take my new iShiny out-of-the-box and from then on I can do everything you can do on that device.
They really are world leaders at this stuff, I can use my right index finger and my mouth and with just that I can use Apple products to run my own company and talk to people on HN (you know, the important stuff!).
---
My money doesn't seem to fit in any of the slots on my computer...
Apple just keeps knocking it out of the park and into a completely new sports metaphor assistive technology provision, they really do. Any measure, I'm a massive nerd/geek/hacker and I really have looked into alternatives both in the open source and proprietary basis and nobody at all is assistive technology as well as at all right now.
It really is as simple as whenever I have a new iShiny, I pounds on the nearest able-bodied monkey and get them to take total box and connected to Wi-Fi and then it's all me. From then on I don't need any more able-bodied assistance whatsoever with setting up the new device, it's awesome. And whilst it's been a few months, the last time I looked known of the other major players were able to come anywhere close to this level of frictionless setup for quadriplegics like me.
ALSO! ALSO! Let's not forget that this isn't some crappy subset of functions that quadriplegics have to settle for whilst everybody else gets the full fat version of the software. Nope, with very few exceptions pretty much anything an able-bodied person can do with their iShiny I'll be able to do as well. Just a little slower.
Apple, and specifically their approach to accessibility is only one of many reasons I'm able to work, see my family and generally engage with society as well as I currently do.
Also, I quite like the term cripple, differently abled or on a good day Stuart. WTF cares what label you give me, I am much more interested in whether this laptop enables me to do the shopping independently and see my Nieces without having to check with an able-bodied person first.
If anybody from the project is reading this, could you tell me where I should send the application, entrance form, fee or even straight up bribes to get my brain and this technology smushed together please?
I'm quadriplegic and since lockdown began last March I've not left this bed once. That's not hyperbole, I literally mean that I haven't left the spot in this bed I'm in right now, writing this comment to you all.
So Yeah, connecting with the world like this is something I need to be a part of. I really don't have words to describe how life changing this would be.
That's not cool man. Hope you have a good quality bed at least. I can just imagine what an impact this would have on your life. As you said, no words can describe but I feel you man.
Eye trackers are used as well for those who unfortunately cannot move anything but their eyes.
That's the case in very bad/late ALS cases.
This technology might really be life changing for the very advanced cases of ALS - they often lose the ability to vertically move their eyes, which makes it very very difficult to work with eye trackers.
Really hope they'll make something like this available to the public.
Depending on if they can move their neck much they might have a forehead dot to track movement for a mouse cursor and a tongue/lip device for clicking. Then it's just onscreen keyboards. Other devices I've heard of are cheek movement or even eye tracking.
In addition to the options other commenters have mentioned, it is possible to fully control a computer with dictation, although it can be very arduous.
The term quadriplegia does not always refer to 100% paralysis, and some have limited use of their hands.
I had to give up my degree in CompSci because I was becoming quadriplegic beginning in the early 2000's, And was unable to type. I then spent a decade working out how to become a computer scientist/robotic/terrible drone flyer. If that does not sound difficult, imagine coming home from work and finding out every single light switch, but, lever and key from your keyboard been removed. What do you do? I asked, you are also in pain.
Also, at the time there was no academic structure available to help, so I had to do it myself.
(I find this comment incredibly hard to write because I am a northern man from the UK and emotion is not something we do easily, this feels like boasting to me even though it answers your question. I am writing it Everybody else who might be struggling with some insurmountable problem with a very simple point: Do. Not. Give. Up.
Not ever, and you will either achieve your goal or die trying. You will almost certainly surprise yourself though.)
EditedTo Add: well, that was meant to be a minor comment and request for pointers but turned into somewhat of a rant. My apologies HMN, my apologies. This kind of dashboard really does have the possibility of reducing the overdoses I am subjecting amongst any number of other mistakes brought about by lack of clear and understandable documentation. And once we have fixed it for me, well from THEREON IS WHERE WE TAKE OVER THE WORLD!
On a slightly related note, does anybody remember that about a year or so ago somebody posted his "Personal Analytics Dashboard" I think he called it. Basically it detailed every aspect of his life. it was things like heart rate, steps walked, miles run, cycles cycled along with all manner of other nutritional input.
The thing that marked it out for me was that it took all those desperate threads of information and display them in one easy to understand and beautiful layout. But I cannot for the life of me find it!
At the moment I want to get to the state that when one of the nurses comes on duty, she can get the iPad out of the server cupboard (after first pressing her flic button to login) and see on one or two pages things like but not limited to:
* Is the catheter turn on or off
* What is the bosses current heart rate (BPM)
* What is today's resting heart rate and how does it fit in over the week, the month and the year
* Which of the doors and windows are open and are they locked or not
* Did the boss get into his wheelchair today, and if so for how long and how far did he travel
* How many doses of his medication has he had today, and how many does that leave him with for the rest of the day
* Have there been any incidences of Autononmic Dysreflexia[1] or any other life-threatening shenanigans other parts of the team will need to know about
And on and on and on, so it would need to be expandable. It may be possible to skin home assistance with something simple enough but I am not a designer.
I've lots of this data coming in already from lots of different sensors, I am however just struggling with a way of turning those data into information that non-technical users will not be put off by. Because if they are put off by it then they will not use the system, and as this is going to be one of the systems monitoring medication and my health et cetera I would really like them to use. Non-technical users make up the huge majority of people who will be using this system.
And to pre-empt what I know one of you will mention because you care about me, I do not ever put my life in the hands of anyone person, process, widget or anything else! It's nice to know you care. <3
I also hate spreadsheets. I mean, I will continue to use them obviously because there is no choice. Just thought world should know that I hate them.
So yes I am pretty much totally comfortable purchasing sensors and and getting my work PA to install them, I'm also having enormous fun creating new sensors but what I am not enjoying is being quadriplegic and trying to draw out nice designs.
Have a feeling it is not necessary and that there will be a simple way for me to plug in all of my data streams and be given a very shiny dashboard that I could maybe put on the wall of my office on some 17 inch screen something similar.
---------
I've included a couple of links describing autonomic dysreflexia, but be warned it's not a nice condition and it is not fun having one of these episodes. Not happy reading, just put them here for completeness
I just wanted to saythat I totally agree with part:
> However, the power of HA comes from the third step: automation. The best interface is the one you can forget exists.
I have the interface just for my carers and PAs et cetera, almost all of my interactions with the house are done via voice or other scripts triggered by various things. I would post a picture of my dashboard but it is horrendously messy contain hundreds things!
Just wanted to say that my life as a quadriplegic would be 13.4 million percent more crap without Home Assistant. Being quadriplegic and having something as open as Home Assistant is absolutely amazing, I have automated absolutely everything in the house and home assistant has not choked once.
Couple that with one of the friendliest communities for newbies I have come across in a long time and you have something really awesome.
I've been using it for a couple of years, I have tried all the other open source alternatives but nothing really comes close for me. I'm actually fiddling with my installation right now as it were.
I cannot plug my phone in to charge it up myself obviously, so I am writing a little automation that will check who is in the house and announce through the speakers my phone needs charging up or send them a text message if I have their phone number when my mobile phone charge gets below 20%.
Totally cool beanz and I am totally serious about how much easier this makes my life as a quadriplegic.
Thanks for your comment. I came to this discussion with my blinkers on, so to speak, pre -judging how I need less automation in my world, and you give concrete evidence how someone's "meh" can be balanced by untold advantages (13.4 million, in fact) of such a system.
Thanks for making me eat humble pie and broadening my views. Sometimes more tech is helpful indeed.
Ps: The way you're applying it is creative and awesome too!
No need to eat humble pie at all! it has been my experience thatEngineers And other reprobates like the ones that haunt can use don't know anything about this stuff, but the second it is brought to their attention I pretty uniformly had positive responses along the lines of "hmm, that is an interesting engineering question" and away we go. :-)
It is one of the wonderful things about this place, who gives a crap about the emotion being quadriplegic (I certainly don't) let's just solve the logistical problems! And as far as I'm concerned all of the problems that disability has introduced into my life have solutions that revolve around logistics, which project/process/company/widget do I need to buy/invent/collaborate on to make this thing work. It's great.
I'm automating my house because I nearly ended up being quadriplegic[1] and I want to be prepared next time if I loose my mobility.
I'm largely doing it with 433Mhz hardware, Raspberry Pi and python scripts. I didn't use Home Assistant because I presumed large project might be overhead and inhibit my choice of hardware; your comment has made me to rethink my strategy.
Raspi is definitely a good starting place for Home Assistant (the Hassio distro makes Home assistant particularly easy to pick up). If / when you grow out of it, an older laptop or desktop would probably be more than enough for you. I’m a big fan of Home Assistant!
And how do you interface to it? Do you send what you caught to MQTT?
I am considering having something else as the RF receiver but would need to forward what it received to MQTT (I can write that part myself - it is really the interface to the dongle I would like to understand)
I cannot plug my phone in to charge it up myself obviously
Do you mind if I ask how much precision your conveyance of choice has? I've built docking stations for small mobile robots by mounting qi receiver pads [1] on the robot and compatible phone chargers [2] on walls where the robot can drive up to them to recharge. If you have enough precision you might be able to do the same thing, mount a pair of pads somewhere, run a cable from the pad to your phone, and dock them together to recharge.
Hmm. Not sure what to do if you don't have enough precision to dock a pair of charge pads like that. Maybe... put the one on the wall on a horizontal rail at the correct height and add a motor to run it back and forth until it starts charging? Maybe mount it on a small robot arm?
Can you clarify what you mean by "conveyance of choice has" please?
Apple has a very very awesome accessibility software, it is seriously the best of the world and I have tried all in the decade since I became a full-time wheelchair driver. I use a chin controller to drive my chair, and I also use the chin controller to access Apple's Switch Control[1] using the Tecla Bluetooth bridge. As soon as funds permit I will be updating to the new Tecla as it is an amazing device, it really is the missing link.
This means that anything you can do on an iPhone I can do the same, it is not some subset of commands like most other mobile platforms. Once a new phone or any of the other Apple offerings is taken out of the box, I can get my designated hands to turn on the accessibility software and from then on I can do it all myself. There is literally no the company where this is true. As far as I know.
If I understand you correctly, it might be cool when my iPhone is connected to my wheelchair to be able to drive up to something while I am working and have it charge. It would have to be pretty robust as I will drive into it at some point!
Ah, sorry, I didn't know what kind of wheelchair or control scheme you were using so I didn't know whether you'd be able to align yourself precisely enough to get the thing to charge.
> If I understand you correctly, it might be cool when my iPhone is connected to my wheelchair to be able to drive up to something while I am working and have it charge. It would have to be pretty robust as I will drive into it at some point!
I think that that's doable, yes! Above, I linked the amazon page for a receiver pad that's designed to be mounted to cell phones to let them use wireless charging even if they don't have it built-in. If you mount one to your wheelchair somewhere and connect it to your phone, you should be able to mount a matching phone charger pad on your desk or the wall so you can drive up to it and your phone will charge as if you'd placed it on the pad. It might be a bit slow - it'll probably only charge at 5W, which looks like it'd take about 4 hours to fully charge an iPhone while it's in use - but it should work.
The hardware itself is pretty durable. The receivers are designed to be mounted underneath phone cases, so you can do the same thing to the one you mount on your wheelchair. The pads are designed to have phones tossed at them every afternoon for several years; I've never had one get even close to breaking. If it does have trouble, the wireless charging tech is designed to be able to run through cell phones' shells and cases and you can build version two with a polycarbonate cover that'd survive a charging battlebot.
You probably could. I personally wouldn't attempt it because I mostly do software and wouldn't want to risk draining it dry overnight or voiding the warranty or something, but if it's a just a car battery it shouldn't be hard for someone with more EE knowledge.
Another option might just be to mount an entirely sperate power system just for the phone and any other devices. Buy a huge USB power bank off Amazon and mount it to the carriage or something, might not even need duct tape.
That was my first thought too. If there's a 12v battery on the chair, a car USB adapter could power the qi pad. Then the issue is how to move phone to pad and move it off the pad for use. There would be virtually no chance of running down the wheelchair battery with this setup; the motors use much more energy than a USB charger.
I thought somebody might ask, so I am just sanitising my config files and then they will be available on my Github[1] hopefully in about an hour. Was there any particular automation you were interested in?
No problem, in about an hour things will start appearing And I will include usefulness et cetera.
The most commonly used automation by a very long way are the lights, because I use voice operated computer people were continually coming into my asking if I was okay because they could hear me talking. This obviously was crazy makin.
So now they ignore all talking unless they see:
All of the lights pulse on and off in a blue colour, which means can you please give me a hand when you have a minute I would like a drink or something similar and it is not urgent.
Or
All of the lights pulse on and off in a red colour, which means drop and come running as I am having and episode of autononmic dysreflexia, my hair is on fire or something equally problematic. It is a very much urgent.
after that I would say that these three are the most used, I'm sure there is a way of pulling that information out of the database but I'm not sure how. If someone from home Assistant Team would like to tell me, that would be great!
* Battery Level on iPhone is below 20%, get someone to plug it in
* Notification asking that the basement be closed when it starts raining, or if someone opens the window when rain is predicted to please close it
* Logging whenever the medication draw is opened and closed and keeping a running tally (this one is super important)
It (or something in my stack) has choked for me a few times. The power went out at my house for an hour last week. and my kitchen lights somehow dropped off my network? I had to reconnect my lights to my network, and then i had to completely remove TPLink from my server, reboot the server, and then connect TPLink back to be able to control my lights.
Barring an extended power outage though, I would agree, it is pretty amazing.
Was looking at going with OpenHAB for a bit, but something about home assistant ease of use really got me.
Aeotec and Fibaro Tend to be my go to companies when I want to buy some sort of sensor, anything from door and window sensors to motion sensors. Overall they have been the most solid and well performing in my experience.
I pretty much completely agree with the content of what you said as well.