The Timeline so far
Level 0 - noob [Sept 2019 - March 2020]
I joined my first company right after graduating from college. I was a clueless idiot, like a deer in headlights. I first started out working on a Spring 2.0 legacy project with code from the late 2000s. Being a Pythonista in college, this was quite a shock. Tutorials were almost non-existent (YouTube wasn’t a thing back when this project was written). My only reference was the codebase and the official Spring documentation. But slowly, by doing small changes in the codebase, I got a hang of it.
Data Engineering Stint 1 [March 2020 - March 2021]
After six months in the legacy backend project, I had the opportunity to be part of a brand-new initiative to revamp the reporting and analytics side of my department. The catch? We had to use an internal big data platform as a mandate. My thoughts on mandates are well documented here. The requirements on the functional logic and data transformations did not exist; instead, we were handed a 1000+ line SQL Python turdball and our job was to migrate it to Java & Spark. As expected, this failed miserably. I left the project using my connections, and ultimately this initiative died a slow and painful death without ever achieving business go-live.
Good Ol' Backend [March 2021 - September 2021]
At this point, I am just shy of 2 years into the job, have worked on 3-4 projects, and am no longer a noob. I can implement features end-to-end, write documentation, and even participate in hiring, albeit all my work had been in projects of no significant functional complexity. But this project is different. A team of 20 developers working on a monster monolith. An extremely complex codebase with a fully custom dynamic form renderer and workflow engine, full of fragile connections that break often. Sounds scary? This is where I learned how to play in the big leagues—how to follow a structured process with the involvement of Product Owners, Business Analysts, QA, and Production Support teams.
React Junkies [October 2021 - December 2022]
My next challenge came in the form of a UI modernization initiative that not only revamped the UI but the entire business process while having to maintain backwards compatibility. I became part of a core group of developers that built a ReactJS adapter to the previously mentioned dynamic form renderer written in Java, thereby bringing the project into the world of modern frontend development. As a result, we built a very well-thought-through system even though we were unfamiliar with React at the start of the project. It was through this sequence of events that I briefly considered becoming specialized in Frontend engineering. I have to admit, looking in hindsight, I was somewhat enamored by React at that time and have realized how naive I was then.
We all live in the Manager’s World [January 2023 - April 2024]
After the modernization initiative had wrapped up, I expressed my desire to specialize in Frontend engineering to my manager multiple times. Even though initially I received some positive encouragement, later on, it was made clear to me that my services were needed in other technical initiatives that my manager had in mind. Thus began a journey of complete dismantling of everything that I had built up till then in that project. I was drafted into a team designed to execute refactoring and tech debt initiatives. This team did away with Product Owners and Business Analysts, and the requirements were directly dictated by my manager and the way to implement them was also dictated by him. The collaboration aspect of the work completely disappeared. This was probably the saddest phase of my career. While I was learning a lot, I felt sad, depressed, and suffocated. A good thing that came out of it was that I used this negative energy to focus on my health and fitness. (That’s a story for another time.)
Data (Platform) Engineering Stint 2 [May 2024 - June 2025]
After I reached my breaking point, I decided to have a 1:1 with my manager. We went back and forth for an hour at my year-end performance meeting, after which I realized the only way forward was out. So I used my reputation and goodwill I had built up, directly approached my skip-level lead, and orchestrated a move to another team within the same department. It was a return to the revamped reporting and analytics project. A lot had changed during the time I was away: we had a new team lead who was my former colleague, new teammates including experienced data engineers, and a new (and hugely improved) tech stack. The project was structured such that I would be part of the “platform team” doing the software engineering to build the common abstractions used by data analysts and data engineers alike. It was during this time that I realized that I was quite good at Developer Experience. I also worked as a data engineer on a few initiatives and honestly, it wasn’t so bad as the technology was much more bearable now. We also had proper documentation and requirements and most importantly I had a free hand at implementation design.
But a question always kept haunting me - Is Data Engineering the thing I was meant to do? What does a Data Engineer make? Is getting data in a certain format a fulfilling endeavor? These questions have plagued me for the better part of 2025. The underlying question of picking my specialization for a lot longer than that.
Realizing what I want [July 2025 - Forseeable Future]
To be continued…