Unable to type because of serious illness, Google engineer homemade voice recognition software, just need to order the computer to write the code on his behalf

A software engineer who has devoted 10 years to a giant like Google but software engineer James Stout doesn't have to type like other colleagues.

As a 10-year senior employee at Google, every programming time, James did not touch the keyboard, but just use his voice to command the computer is done. His desk is also quite special when equipped with eye tracking system, microphone, monitor and a specially designed keyboard. Underneath his feet was the pedal system, which, whenever he stepped on, displayed the number on the screen.

It sounds easy and simple, but behind this unique way of working is an emotional story. James suffers from Ehlers-Danlos syndrome, an extremely rare syndrome with a ratio of only about 1: 2500 – 1/5000. The wicked disease severely affects the patient's connective tissues, causing them to experience persistent muscle injuries. It has also prevented the dream of becoming a programmer of James when typing every time the hands are so painful that they cannot move.

The keyboard is designed specifically for James.

Currently, James and his colleagues are researching ways to make it easier for users to access and use Google products, such as what makes voice applications for example. In addition, he founded the blog Hands-Free Coding, specializing in sharing the process of voice programming instead of typing as traditional.

Share with Business Insider, James said: “ I have overcome my difficulties, and now it is time for me to help people who are in the same situation as me before. There have been many commenters on the blog that my share saved their programming career. It was just a small blog that was so fun and helpful, I thought of dedicating all my time and effort to this new aspect – voice control ”.

And time has proved that James' thoughts are completely accurate, with many virtual assistants on the market today such as Amazon Alexa, Google Assistant and Apple Siri. Life of users is now much easier and more convenient, when they just need to say a few commands to be able to listen to music, watch news or even book movie tickets, schedule a haircut. , shopping, … The voice programming system invented by James not only helped him continue with his dream job, but also opened up a new opportunity, a new initiative to help people overcome. physical barriers.

In addition, this pedal system also helps James a lot in writing code by voice.

Of course, there have been many times, James felt extremely desperate and said that his career as a software engineer is about to throw away, the programmer who can't type is still able to do anything.

" What I am most worried about is that hand pains will reduce my speed and productivity, " James said . I might lose my job, maybe I won't be able to keep up with the deadline, and at the worst, I won't be able to promote all I have .

But after all, Google and his colleagues have always supported and helped him, his manager did not hesitate to "green light" for him to start testing voice programming.

The physical disadvantages open up new opportunities for James Stout at Google.

James started work at Google as an infrastructure engineer, then moved to Waymo – the company's self-driving car project. But this is also the time when his illness got worse, when the pain of bone pain continued to come instead of just a little numb as before, just typing a key is extremely difficult.

" For a software engineer, this is really a nightmare ."

In fact, Ehlers-Danlos syndrome has persisted with James ever since he was a child. The evidence is that he always feels his joints are loose, and especially that he can bend his fingers backwards without feeling any pain.

But it was only at the time of intense typing that the disease broke out. Not only does it affect his work, but James' daily life is also very difficult when he is always in pain every time when cooking, driving or simply picking up the phone. In addition to his fingers, when he was a student, he had to face many times the pain from dislocated shoulder and knee dislocations.

The illness was so severe that there was a time when James thought he was about to lose his job. Many doctors also shook their heads, some even advised him to consider changing to another job to ensure his health.

But it is easy to call it passion. James shared: “ I chose computer science because that was my dream. I always like what I do, I want to be a software engineer. When I heard the doctor told me to change careers, I was really desperate and tried not to listen to his advice. When everything in your life begins to decline, you will certainly doubt yourself. It will always haunt your mind, and only disappear when you really overcome it yourself . ”

" I once thought I had nothing to lose, " James said . It is easier said than done, but watching videos of my predecessors, who have succeeded despite not possessing perfect physical condition, really helped me to wake up and gave me the motivation to continue the fight. duel ”.

So James strongly suggested to the manager, allowing him to use the voice to program instead of typing once a week. And perhaps his whole life will never forget this person, when his manager was extremely psychological: “ You can rest and try this method as many times as you want. Your health is the most important thing. If you want, don't say it once but 5 times a week ”(ie, voice programming completely, no need to type any more keys).

The psychology boss allowed James to test voice typing code whenever he wanted.

Initially, James used the Dragon speech recognition software, which allows recording the speech and converting it into text form. It sounds good, but the software is still too limited, can only work well with simple tasks such as writing mail or editing Microsoft Word, but to write code is not enough.

Undeterred, James continued researching, creating his own software and constantly updating new and more advanced features. In addition, he also invented a special set of commands to help the process of writing code easier. For example, programmers often use parentheses when programming (left parentheses / right parentheses). Instead of being so verbose, James has shortened it to two "leap" and "reap" statements that are much simpler.

It took up to 3 months for James's voice programming software to write code at half the speed of his keystrokes. But just that is enough for him to have more confidence that: I can still be a professional software engineer!

“ For me, it was a turning point, because the improvements in technology have really affected my life so much, it decided to make a career. Gradually, I thought of other people who were having the same difficulties as me. How can I help them here? ”

Despite many difficulties, the results show that he can still be a professional software engineer.

At this time, the blog that seemed to be written for fun of James became extremely useful. There have been many readers commenting that his sharing really helped save their career.

For example, one reader shared that his 12-year-old sister also suffered from a strange disease, unable to type. So James made a video call for the two brothers, sharing what he had learned from studying voice programming so they could overcome the physical difficulties.

Recognizing the tremendous influence of the blog, James had just thought of turning it into an official career. And so he decided to leave the Waymo project and join Google's Accessibility team. He is also particularly interested in machine learning – an artificial intelligence program that allows computers to load knowledge for themselves. Although his forte is voice control, James thinks machine learning is the key to helping other people with disabilities.

He said: " Machine learning allows us to better understand what is happening inside the" brain "of a computer. That's the best part of Google. What I have been doing is very interesting, but there are many limitations. I want something more explosive . ”

Currently, in addition to official work, James is still persistent in creating new features for Dragon – the "life buoy" software that has helped him a lot before: "I plan to continue sharing for free. what I did and what I discovered. I hope what I'm doing at Google will help everyone in the world, not just a developer. I also hope that the ideas and ambitions that I am cherishing about machine learning can inspire the world of programming industry ”.

James's blog site has been helping other similarly difficult people.

At present, James's ability to control computers via voice has reached a peak, when he can write mail and type code faster than typing by hand. However, with software related to image editing such as Photoshop, or making slides like Powerpoint, it is still quite limited.

Another difficulty James faces is multitasking – he cannot both command a computer, nor plan his work because both processes occur simultaneously in the same hemisphere. Brain. Ordinary people on the contrary, they can easily think of other things and still type deliciously.

But again and again, James's productivity has improved markedly: Not only can he type code faster, he can also better understand all aspects of his work. He said: “ I am always thinking about each of the commands I give and wonder how I can perform this task more quickly and simply? ”

Most importantly, James will never have to worry about giving up his software engineer dream, and what he is doing still has a lot of potential for future development: “ I did it myself. Overcoming difficulties, but there are still many things that need improvement to make the work faster, more convenient and simpler. I feel I still have many shortcomings, and that is the motivation for me to keep getting excited every day . ”