DATA STUFF
My Data Engineering Journey: Pivoting from a Physics Major
A doctor wannabe became a programmer, who would have thought?
Inspired by my recent webinar session as a speaker (link), I decided to write a post about my journey in becoming a data engineer, especially as I come from a not-so-IT background, which is a Physics major. In fact, I used to dream of becoming a doctor but ended up enjoying my current role as an “experienced” data engineer (at least by the number of years, which might not represent my actual skill, haha). Looking back, I might say that my journey could be categorized into the following periods:
- Pivoting from a Physics major (link)
- Shifting from Software Engineering (WIP)
- Hardening the Data Engineering (WIP)
In this story, I will share about the first period, which is the period in which I’m certain that IT is the path that I have to take for my future career.
Pivoting from a Physics major
This is essentially my time pursuing a Bachelor’s degree in Physics. You might think the word “physics” is related to the equations of force, acceleration, and speed, or you might think that it’s related to the general theory of relativity as this article’s picture suggests, or even the term “quantum” in those superhero movies. Well, those are partially correct, which is actually not only those terms but far more beyond that. As pure physics is one of the roots of science, it is somehow applicable to all of our everyday life, including the medical field, electronics, and even ensuring that your ladder is placed appropriately to avoid falling. During this period, I see that it is also divided into some sub-periods:
- Early physics
- Robotic team
- Undergraduate thesis
Each period held its own story.
Early physics
As I scratched out my initial plan of becoming a doctor, I decided to take on the Physics major instead, with the thought of taking the medical physicist minor when the time comes as it is available as one of the options in the university that I’ve attended, and would still allow me to contribute in the medical field.
The truth said otherwise, as it turns out the electronics class sparks more joy in me, and I decided to take the control system and instrumentation minor instead. It somehow also has the optional class of “medical instrumentation”, a class that might be my last straw to contribute to the medical field, but let me stop you there as I didn’t take that path.
Apart from my fuss in the medical field, this instrumentation minor is where met “coding” for the first time, in which I had to code a program for an embedded system, one of them being in the C language. Not as fancy as you might think in software engineering terms, but from my perspective, I code, without knowing for sure where my future will take me.
Interestingly, my friends and I in this minor held an open discussion about what’s our plan for ourselves towards the path we took. Me, not knowing exactly what I wanted to do, and kinda felt that the medical field didn’t excite me anymore at that time, decided to share my love for robotics back in my early life (I love LEGO!). I also shared how I used to plan to join the Mechatronics Major before deciding to take the Physics Major. One of my friends, who already started to be involved in the robotics team of our college, asked me to join his newly created team to compete in the new competition of Unmanned Aerial Vehicles (UAV), or we might be more familiar with the term of drones. I took the opportunity as I found this sparks joy for me. As simple as that, I became part of the robotic team.
Robotic team period
The robotics team’s initial recruitment has three roles we could choose from, which are mechanics, electricians, and programmers. I might be closer to choosing the electrician role as I am an instrumentation minor, but somehow followed my gut to take the programmer role instead. A kinda crucial choice I suppose, as it’s one of the decisions that led me to where I am now. I then (un)officially became a programmer!
I was somehow categorized as a relatively inexperienced programmer, so I was assigned to do the low-code programming, with most of the logic being set via a UI to “program” the drone. Regardless, I also had to learn from the programmer who actually did the full-code programming, which at that time used C++ programming language (whoa!). I could say that I still got the coding exposure as I might have been asked to modify the script when the mentioned programmer wasn’t available, but I was not the one who developed the script from scratch.
The competition itself was using a drone to find fire hot spots and attempt to extinguish them. This led us to utilize the image processing logic to find it (hello there, image processing!). This also led us to find a supervisor from our lecturers, which turned out that one of our physics lecturers has the research experience of using a drone. Long story short, I had a great time reprising my role as a programmer in the robotic team, nurturing my interest in the IT field.
Undergraduate thesis period
It turns out that the supervisor I previously mentioned was my supervisor for my undergraduate thesis. Not really that surprising as he’s looking for a student to be supervised, while I was looking for a supervisor to guide my thesis, and as the topic suits my interest, I don’t find any reason not to seize this opportunity. Regardless, I was aware that I lacked the programming skills I need to have to be more convenient in doing my research.
To overcome that challenge, I decided to take an external subject of “Programming Basics” from the Computer Science Faculty. This is of course optional, but as I’ve somehow been firm in seriously doing my research, I took the class even though there have been rumours that this is a killer class that will surely affect my GPA if I get a bad grade. The rumour was somewhat true, as I couldn’t even get a “B” grade. I made peace with myself and said that even a part of the Computer Science students themselves were not passing this class, so it was already great for me to pass this class with decent grades.
As per my research topic, my supervisor suggested extending what we’ve achieved in the robotic team, which was then continuing my journey in C++ programming language. In this case, I was also going to implement AI in image processing (whoa!). As this research will be for my undergraduate thesis, I need to dive deeper into the theory behind it. So, now, not only do I have to code, but also the image processing and AI theory (apart from the aerodynamics and the other physics theory, which is not that relevant to the programming thing).
I was using the artificial neural network as a Convolutional Neural Network (CNN), and I only managed to use an existing tool instead of tweaking it for the best result. It is hard, of course, especially considering that it is not taught in the physics curriculum, but I had to do it to graduate (haha). Well, hello there AI. I was then a Physics graduate with limited knowledge of code to do image processing and AI.
This moment was an “Aha” moment for me to see that “IT is the future”. It’s also worth mentioning that it was in 2016 when tech stuff wasn’t as fluid as now. As per my limited understanding at that time, I could pursue my career in the IT field by diving deeper into the programming world, which led me to pursue the software engineering role (data scientist wasn’t that “sexy” at that time, let alone data engineering). This encouraged me to take a master’s study in IT not long after I graduate, which will be the next story for me to tell.
What I learned from this period
- What you’ve planned might not be what will we do in the future. From planning to become a doctor to becoming a programmer, who would have thought?
- Networking opens up opportunities. If only I refused to join my minor’s meeting, I might still be struggling to decide what I want to be, let alone finding a supervisor for my undergraduate thesis.
- Don’t be afraid to try. It’s especially the case when you’re still unsure of what you want to do. Opportunities don’t come right away, as we need the effort to make the opportunity appear, and that’s by trying.
I hope that you found my story relates to you, although might not be the same, but could somehow be a starting point for anything that you are (or will be) doing. See you in my next post!
