That is all true but I don't think the n-dimensions analogy applies to the particular essay the author wrote.

He didn't look back on his accumulation of learning and suddenly notice that it was smaller part of a larger knowledge space. (E.g., programmer stumbles across "monads" and suddenly he realizes that's just the tip of the iceberg in a never-ending horizon of category theory.)

Instead, his essay is lamenting the all-too-common "joy vs drudgery" in projects. It's a similar sentiment that aspiring writers who don't finish their "great novel" have. He then grades himself on his lack of motivation (words like "lazy") to stay on top of programming tasks that are not pure intellectual fun. On his idiosyncratic standard of "programming competence", he wonders if he is a "worse programmer".

Of course, he can choose whatever "success metrics" he wishes to apply to his personal projects but some of us might think that staying on top of documentation (that nobody will read) or adding features to Lily (that a large population of programmers haven't adopted) can bring undue self-criticism.

If we are to plow ahead into boring unpleasant tasks, there usually has to be a compelling motivation -- e.g. we think the personal project is the basis of a great business startup... or the code is part of a PhD paper due next Friday, etc.

So it seems like one problem/feature with high dimensional spaces is that you have more regions which are orthogonal to one other. So, it's possible to have quite a bit of motion, which to a human mind might give the impression that one is doing quite a lot—but then you might discover a new way of turning around and find that space extends in another direction, too, at which point you realize you covered a smaller fraction of the available space than you previously believed.

What I don't get is how the volume of an N-ball with fixed radius going to zero as N goes to infinity plays into the analogy. When you say, "when you compare it to a hypercube of the same dimensional count," I don't get what in the programming analogy would link to the N-ball rather than the high dimensional hypercube. I also get the algebraic justification for this property of N-balls, but I still have no real geometric intuition for why it would be the case.

Could you clarify what's meant by, "as the number of dimensions increases the closer the 'halfway' point of the volume of the N-ball gets to the edge of the N-ball"? Not sure what's meant by 'halfway' point here, or edge of the N-ball. My guess is that you're talking about distribution of volume, like if you have a tapered cup that gets small at the bottom, most of the volume will be at the top.

https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

I tend to focus on the fact I am constantly expanding my knowledge, instead of worrying about all the stuff I don't know yet.

I still consider myself a generalist, however. I can't say I'm an expert in any one area. Maybe that's why I feel the way I do.

This certainly applies to programming but I've felt it too in my Aerospace career. I used to feel like the smartest guy, I always knew how to do things or learnt them really fast, and I couldn't imagine where the horizon was. Now, the more I see, the further I feel the horizon is, and it's pretty frustrating as it makes me feel less intelligent.

Your comparison helped me to see things a bit clearer and to be a bit more motivated.

There are always stories about innovators who simply apply established solutions from problem space A to problem space B and everyone is amazed. The surface area problem certainly exists for the entire domain of human knowledge, but domains pretty much exist to wrangle problems that are manageable in the first place, even if it makes you myopic in the process.

I get frustrated in interviews because everyone is just so convinced that the latest utility module is so new and unique and it's really just a retread of five other things. It might have some interesting semantics that change how you use it, but even that isn't always the case.

Plus ça change, plus c'est la même chose.

I've just started telling people that 'how' ages faster than 'why'.