Object-Level
What does a normal day in your field look like? Can you give me a “day in the life” run-down?
“My day usually starts with an hour-long meeting with all the members of my team, where we resolve problems we’ve dealt with over the previous few days, and talk about what we’re doing today. Then our manager will assign us new tasks and tell us what to prioritize. After that, it's mostly individual work. Sometimes there are meetings throughout the day, and you need to arrange what you will present on top of completing your programming tasks. Other days, I have to prepare for a webinar where I present my company’s current projects to the public or our clientele. Those usually consist of live demos of Jupyter Notebooks where I show how to train deep learning models on my company’s hardware.”
Aside from meetings and presentations, what do you spend most of your time doing?
“The majority of my time is spent running experiments to acquire performance and accuracy data about our hardware, or optimizing the configuration of models. We want to make sure that anytime someone uses our company’s hardware, it’s performing better and more accurately compared to the competition.”
How does this differ from the average engineer?
“An average engineer may not present so much content to a live audience or work so often with R&D teams, but aside from the fact that I am working in a niche field, it's not too different.”
What does a bad day look like for you?
“Bad days happen when you get stuck debugging rare, esoteric problems – where you can’t find out what’s gone wrong even when you google the error messages. You have to dive deep into what’s going on beneath the surface to resolve your problem, and often keep your customer waiting.”
What does life look like for a software engineer outside of work? How often does your work bleed into the weekends, and how much time off does an average engineer get?
“Initially, when my team was new, we would work longer hours, but we’d hardly ever be working on the weekend unless there was an emergency. Now, I make it a point that I don't work more than eight hours a day, and that I find time for other activities, like cricket. I play college level cricket as an alumni representing my university and also in a larger league for the entire South California region.”
What helps an engineer negotiate the lifestyle they want with their employer?
“The ability to debug and solve your problems quickly will take you far. That leaves you with spare time, and it also helps you build trust with your managers. If they can give you a task and rest assured that you are smart and savvy enough to be able to solve the problem independently, and do whatever it takes – like diving into the software base or debugging hard problems – you have leverage with them. Building that skill takes time, and it comes with experience.”
What is your physical environment like? How does the software engineering environment vary?
“These days I work from home. My setup is a laptop hooked up to a couple monitors and a sit-stand desk. Other engineers either work the same way, or they have to drive to work and sit in a cubicle. I live in a large city with heavy traffic, so I’m grateful I don’t have to do that anymore.”
Do you use Github Copilot? If so, has it improved your productivity appreciably? How do you foresee future versions of Copilot changing the nature of software engineering?
“I've tested GitHub Copilot, and I found it could give me some quick answers. Other than that, it's been restricted at my company because they don't want to share our code base with GitHub. I'm not sure what the current Copilot model is capable of, but I think it can tighten the debugging and prototyping workflow. Other than that, I don't see how useful it could be.”
Do you expect Python to ever be replaced as the de facto programming language for machine learning? What would a dream language look like for data-driven programming?
“I think Python is the dream language. You couldn't ask for more. I don't see it being replaced in the next 10 years or so.”
Sometimes medical students intent on becoming surgeons find out they can’t take the sight of blood. They may shift towards a less hands-on specialty or abandon medicine altogether for that reason. Are there any “deal-breaker” stressors of that kind unique to your work?
“Deal breakers for me would be working inside a cubicle five times a week a minimum of eight hours a day, and working on weekends as a support engineer. I know some companies that provide you with a pager and expect you to work weekends, whereas I provide extended support every one month or so. Anything further would be stretching it.”
Are there any hair-raising moments in software engineering? What in your work lights up your adrenal glands?
“When there's a release coming up and you find a major bug with no immediate resolution.”
For computer sciences students, or people about to join this field – what has the potential to go most wrong? How could someone otherwise competent squander their career?
“You could squander your career by being lazy at work, and spending too much time surfing the internet. As a software engineer, you’re always a few keystrokes away from the whole internet, so you must be conscious about distractions. And then there’s the stock market. If you're speculating too much on the markets with the kind of volatility around today, you could squander all your money. Not being careful about money is a really bad way to go about things.”
You graduated college long before the internet democratized education to the extent it has today. To what degree can one acquire an equivalent education to yours with resources freely available online?
“With shelves of free textbooks stretching out forever, all you need is a structured plan. If you have that, and you follow it, you don’t need a college education. Absorbing the core abstractions of machine learning can be done rather swiftly. Everything you stack on top of that can come from what you learn on the job.”
What would you do if you had to relearn everything you use for your job – including math – using the internet? How would you show employers what you’ve learned?
“I wouldn’t use a certificate course, though there’s nothing wrong with them. The primary way I would show off my skills would be with a portfolio made using GitHub, LinkedIn, or a simple website with clear explanations of the projects and what I aimed to accomplish with them.”
Do you have any online resource recommendations for programmers learning deep learning?
“Machinelearningmastery.com, Fast.ai, Udacity, and Geeksforgeeks.org. Stanford also has some online courseware, but you don’t necessarily need anything so intensive.”
Meta-Level
Do you find your work meaningful? Is meaningness contingent on specific things, or is it intrinsic to the work?
“Yes, I find my work meaningful. When I'm able to help customers train models faster and at better accuracy than the competition, I'm vicariously enabling a lot of things. I'm enabling better technology. We have clients that work in medicine, and through supporting them, I'm enabling the refinement of medical science, allowing doctors to detect diseases they wouldn’t have noticed before. These things happen because of my work, colouring it with meaning.”
If you were trying to dissuade a software engineer from specializing in deep learning, what would you tell them?
“I would tell them to stick to the basics. You don't need to hop onto every new technological bandwagon. Don't feel pressured because things are changing so fast in machine learning, with all the new innovations coming out every week. Get good at the fundamentals of what you do, and don't get overwhelmed.”
On the other hand, if you were trying to persuade someone to join your speciality, what would you say?
“I would say that it is a unique field, with major surprises behind every corner. Machine learning unlocks a whole new dimension of computing. I think it's essential that you have some basic understanding of the concepts as we are ushered into the new era.”
What is the rate of change of information, important paradigms, and established thought in your field? How often do earth-shaking things get introduced?
“Earth-shaking things used to get introduced every six months, but now I think it’s every three or four months. I’ve never seen this rate of innovation in any field in all my life. It’s always been getting faster.”
What would you say is the seminal paper of the last “epoch” of deep learning?
“Anything major happening right now will be considered seminal in the future. It started with AlexNet in the ICCV competition back in 2012 – that was one of the first papers that brought out the revolution that we see today. And then Google's transformers in the NLP space in 2018. And now stable diffusion.”
Do you think we will hit a plateau soon, or will this continue all the way until general intelligence?
“I believe there's going to be a plateau pretty soon. You're going to find diminishing innovation in the general space, while the roots of research will draw from progressively more variegated niches. More will happen, but it's not going to be generally applicable to all other scenarios, and that will be the situation very shortly. That's my personal opinion. I know there's millions of people who would actually differ.”
Do recent innovations change the way you look upon your education? Did your years in school prepare you for how the world of machine learning changes?
“My schooling was not in machine learning. I took one AI course in undergrad, but it was all textual, and had no correlation to the real world whatsoever.”
Do you feel college taught you to keep up with new changes?
“Nope, not at all. I learned that on my own.”
How would you recommend someone go through college – given that most of what they need to know, they will need to learn on their own?
“I would tell them to focus on their tech interviewing ability. At the end of the day, that's all that's going to matter. Even your courses are not going to matter so much. It's a good idea to take the most popular courses, but you will move no earth if you can’t do well in interviews. So it’s LeetCode all the way! At the end of your undergraduate, when you go for your interviews, they should be able to immediately say yes to you, and you have plenty of time to make that happen. Other than that, college projects will teach you enough.”
If you had to distill the process of acquiring machine learning skills, how would you do that? What’s the eat/sleep/repeat loop that you go through to get better?
“My work keeps me up-to-date, so I don’t fret over improvement. However, I do check out the YouTube channel Two Minute Papers and the website paperswithcode.com for general information.”
What makes an engineer of any kind excellent?
“The ability to seek help when you think you've exhausted all your revenues. The right temperament is to have the thirst to be able to solve your problems, but not so much that they consume you. There’s a balance to be found between the prototypical Google engineer and incompetency. In anything in life, I always take the middle path.”
If I transported your mind back into the body of your 18 year old self, what would you do differently given what you know now regarding your work?
“I would actually do less than I did. When I was 18, I wasn’t spending all my time in front of a screen, and that was good. I would definitely spend more time on leisure. Work is going to be part of your life eventually, so go and play some games when you have the time for it. I would play even more cricket than I did. What I regret right now is not spending the extra time with that, because I’ve learned so much recently, and if I had learned those skills back then, I might have even become a professional player by now.”
What general skills has cricket taught you?
“Cricket – or any sport for that matter – teaches you how to gracefully accept failure and success, and that transfers everywhere else. There’s also the self-study element. You have to look back at the things you did during the match, and figure whether you took the escape route when your team was in trouble, or did you stand and fight? It teaches you about yourself more than anything else, and the more you know yourself earlier in life, the better you are prepared for what’s to come.”
Domain
To what extent does talent matter for succeeding in software engineering? This could be a very vague question, but it’s known that journalism and athletics require a high amount of initial talent, and something like dog walking does not. Where does your field fall in between those two extremes?
“Talent in software engineering is having the knack to quickly link things together, and know how they work behind the scenes. It can be trained, of course, but it might take some time.”
How could someone find out if they have talent or potential as a software engineer? What kinds of interests or hobbies usually indicate someone will be good at or enjoy working as a software engineer?
“The people I know who work at the best tech companies usually spend time solving puzzles on their own. There’s a strong correlation between the quality of a tech company and the geekiness of their employees. I sometimes find it obnoxious – that self-absorption that comes with the geeky personality.”
What prevents talented people with a good fit to the field from becoming excellent?
“The extremes of laziness and overdoing things. If you're overworking, let's say you're chasing money, and promotions, and you're doing too much stuff, you're going to get burned out for sure. People who don't know how to maintain that balance are going to get burned out more often and sooner every time. Eventually they'll quit or go less far than if they had maintained that balance.”
Relatedly, what falsely fruitful activities do your colleagues engage in? Why do people fall into these traps?
“Doing projects on your own outside of work. If you’re working in a demanding job, and you still have time to sit in front of a computer and do more projects, that’s going to burn you out. If it's a casual project, then constrain it to the weekends. But then there’s hobby game developers. They’re the worst! Why would you build a game when you could spend that time playing a wonderful game outside? Why would you go through the pain of building a game when games exist out there that do pretty much the exact same thing as what you're building? So for me, game development is the most wasteful thing to do.”
The most wasteful hobby?
“Yes, most wasteful.”
Does it not have any advantages as a portfolio project?
“If you're early in your career, that's the only time you should do something like that – or if you're working in game development obviously. Outside of those edge cases, I don't see how game development will translate into your work or any other aspect of your life. It's will only drive you into greater isolation because game development takes time and effort – and you will never be satisfied.”
What traits do different specialties of software engineering favor?
“The general software engineering skills – having a balanced amount of tenacity combined with the knack for linking things together – should take you far. Every general skill in engineering transfers well to other areas…except game development. Because I hate game development. There’s zero skill transfer with game development!”
Contrarianism
What decisions did you make under great uncertainty that worked in your favor?
“I stuck to my field even though nobody else I knew in school was pursuing the same thing. I did my master’s degree in graphics because I couldn’t stand the other kinds of engineering – staring at a static wall of text all day didn’t entice me. I wanted to see the result of my work immediately, and I wanted to work on something visual.”
What are some things you believe about deep learning and artificial intelligence that other engineers don’t?
“General AI is BS as a concept. Many people sometimes believe that AI can become self-aware and all that nonsense. It's a machine at the end of the day. It's a program. It has no other sense to it. I would say that AGI is entirely a fiction.”
How would you respond to the argument that humans are just programs, that our consciousness is just a product of a physical law?
“You’re munching words together that have no meaning.”
Do you think about models reaching human-level performance ubiquitously? When do you expect this will happen?
“Like I said, I don't like general AI as a whole concept. Yeah, computers have beaten humans in Go, chess, cancer detection, and other tasks – that always turn out to be narrow problems under the hood. I don't think there's ever going to be a general purpose thing that can do it all. I don't even know why people chase general models. That's crazy.”
Do your colleagues entertain these ideas?
“Nope. Nobody talks about this. Everybody has a life and a wife and a family and a home to go to and they don't discuss these things at home either. When you see the same thing every day, you get sick of talking about it. You just want to unwind and stay as far from the computer screen as possible.”
Aaron Swartz once observed that the smartest people he knew always talked about fine details, and that charlatans always talked about big, fancy abstractions. Are there any tell-tale signs that someone doesn’t know what they’re doing?
“Anytime you see the words ‘serial entrepreneur’ on someone’s LinkedIn profile, run away. If anything, it means they started a long string of failed businesses. And, of course, anyone who calls themselves a ‘sapiosexual’ is a hack of the highest order.”
What is the most common useful fiction engineers in the deep learning specialty have, and why do you think they have it?
“The belief that everything is soluble using Google. It helps me solve about half my problems, but sometimes it’s sifting through goldless sand.”
What paradigms won’t change in machine learning and computer vision, or perhaps technology more generally? Are there things you don’t see budging?
“We’re going to use the 2D format – 2D screens, 2D images – for a long time. Apple may claim to have AR goggles or whatever, but monitors aren’t going anywhere…unless they’re VESA mounted.”
How will the experience of being a software engineer change in the near future?
“I think being able to learn new technologies will always be paramount. You will always have to be on your toes, holding a fully developed set of interviewing skills, ready to interview anytime. Things are changing, the world is becoming more volatile, company projects coagulate and dissolve like mishandled yeast cultures, and you are a greeble on the corporate Star Destroyer. Soon, you might start to find yourself in the market every six months, even if you’re a competent engineer.”
People who’re tapped into a specific field might take breakthrough innovations for granted because they’ve known what to expect long before results materialized. If you could step back, what in the nearterm might the world find revolutionary coming out of machine learning and computer vision?
“NLP technology will improve, data about individuals from multiple sources will be kneaded together, and corporations will be able to deduce ever more surprising things about you.”
How do you disagree with everyone about the future of your field?
“Aside from AGI, I don't disagree or agree with anyone or even talk to anybody about the future of my field.”
Conclusion
How might someone figure out more about what being a software engineer involves? What books, interviews, or nontechnical media would you recommend they check out?
“On YouTube, there’s a bunch of video blogs about people’s day-in-the-life at Google or Microsoft. So you can get a general idea about software engineering from there. I would recommend a channel called BoyAlgo – new on YouTube. He's the most pessimistic engineer with the most imposing imposter syndrome I’ve seen. He talks about his real experiences and how he feels, and he says it the way it is. Although he works at Google, he thinks he's the dumbest guy there, and he's completely candid about it. It might initially appear really dark, like this guy is so depressed, but he's just being candid and honest and he's frustrated. And that comes out in his videos and it's hilarious.”
Thanks to Chlorokin for contributing questions about GitHub copilot and AGI.