Ask HN: How to become better at software "engineering"?

> Writing design documents, properly tracking decisions, careful planning, building infrastructure and deciding on expensive infrastructure projects, all sorts of compliance stuff.

These are important skills as a data scientist as well and as such shouldn't be new for you. I'm not sure what sort of strategy you've been employing to get any serious data science work done.

> how do I get better and find joy in the more managerial tasks of software engineering?

Maybe you don't and that is okay. You don't have to find every single part of software engineering joyous. This goes for non-managerial work as well. Not all problems are blessed with being fun. Just do it regardless.

> Does confidence just come with experience?

Yes.

> Can I get better faster?

Yes. You get faster at any task you repeat often enough.

> Or is it all worthless anyways and I should focus on the code?

Some of it, maybe, maybe not. You'll know what kinds of non-programming activities help your programming after trying them out for a while. I for example don't love design documents for every little mundane feature. However, for larger, complex tasks the act of writing something, anything at all helps me bring clarity to my thoughts. I've also found development diaries extremely useful for any project where I go more than a week between development sessions. The important point is that I found this out by testing it out and reflecting on my experience and/or results.

Do the boring parts first. I mean this seriously. You need to build personal routines of accomplishment for the more administrative tasks. Some of that can be worked in parallel with other things.

Don't use AI.