I just published a piece for DataExpert that I’m particularly proud of. Click here to read it.
Like many of my pieces, it makes a few interlapping arguments:
- Data Engineers are actually product owners for data
- This is particularly true in the age of AI that can do a lot of the engineering work for you
- Seniority is correlated with the influence/people/non-technical parts of the job
- Data systems are your product, your coworkers are your customers, you’re looking for product/market fit
- Pay attention to what surfaces/numbers the principals actually use, and treat them as services that need 100% uptime (even as you gently and carefully update them)
- Showmanship and reframing your work is a big (and fun) part of the job
- (and more)
Plus a few different war stories from my time in the trenches.
I was spurred to write this piece years ago, when I was brought on to consult with a particularly beloved computer company. There was a lot of DE work to do, and a big question was where to start. We zoomed in on the jank google sheet company leadership was using to track numbers every day. Thinking about how to (carefully, slowly, with care) get better processes to power the “backend” of that critical artifact framed our entire plan.
I think about that a lot.
Okay, so here’s a link to the article, and later I’ll update this post with an archival copy for posterity.