Why I am not getting better at programming


#1

I do not see programming as an art. I approach it with too much demand for exactness. I do not know how to begin to go about building a compiler until I’ve seen someone else do it. I do not want to need to consult previous hackers’ works to begin mine. I want to be shown a set of axioms and theoretical underpinnings, set in stone, from which we all work with little deviation in idea and technique. I want it to be a natural science so bad so that I would not feel as dumb as programming currently makes me feel.

You can do anything in programming. True. False. You cannot write an API without validating inputs. You can. You can’t. I want the programming environment to tell me about that without me having to pore over pages of online tutorials to learn how to do something so obvious. We slap tool atop tool in an ill-fitting mess of dependencies that it begins to feel like a very raw craft.

Where can I escape this field of boring uncertainties a the implicit? I want to write a text editor. I should not have to hack for hours on end, hoping to be led by the Muse to discover tree data structures. Which I never will on my own. There should be a catalogue of the most essential tools of thought from which every new apprentice must depart having thoroughly schooled himself on how to handle them and combine them in new ways.

Every hacker should begin by knowing what he desires to make. But also what he desires to be able to understand. I don’t want to make websites. I want to understand compilers. And then, incidentally, make them. I want to understand a lot of things. And I cannot without first learning how to make them. Attempting to make them. It is this exercise of attempting to create that causes me to freeze. The devotion to surviving failure and the stamina required are not merely excruciating because of the difficulty but because of the hopelessness. The vagueness you must endure when trying to make something you have never made. Vague because you do not know what the rules are, when are you getting better? Is this the best way to learn X?

I want to focus on learning about programming languages, how they are implemented. This desire to understand programming languages is enough to see me through the dark tunnel of learning. But I must become great at building backends too. I’m not curious about backends. There is no theory of backends that survives gracefully when the more practical things are stacked on top of it. Behind the monstrosity of modern frameworks, it’s hard to believe I can walk into that castle with little guidance.

How to fix this?


#2

You likely won’t learn programming by learning to write a compiler. That’s a huge jump for someone who still has a lot to learn. But if you want to dig into the roots of programming and get a better idea of the kind of programming used to develop compilers, even other languages, you could learn C programming. A number of the biggest languages, systems, and compilers are written in it. If you can’t learn C, you probably won’t be writing compilers anytime soon.

Just my few cents.


#3

You can. I’ve been programming for about 4 years now. But a beginner can be taught how to write an interpreter, at least. The essentials of any large software system can be taught.


#4

From what I could read between the lines, I think you would be happy with the certainty that low level programming has to offer. I would focus on C, Linux, embedded stuff. I think you will be happy doing that.


#5

A human learns to walk by falling and controlling the fall. We learn math and science because so many before us have ventured out, failed, and failed again until they found what they were looking for.

I feel what you’re saying.
Relying on other people’s discoveries and knowledge is honoring the strife they went through to bring the knowledge to everyone else.

Don’t be afraid to try and fail. Don’t be afraid to build upon knowledge other people worked very hard to provide to the community.

Failure is just like falling, and if you can’t see the value in the forces taking you down, you won’t successfully master how to avoid them.

Don’t give up. You can do this.


#6

I’d also highlight that programming ‘languages’ are like spoken languages. There are rules of syntax of course, but no rules to how you implement a sentence.

Poetry differs greatly from a set of cooking instructions, and both communicate their purposes in vastly different ways. Rules and language are not easy bed-fellows, as languages are always evolving.


#7

The fix is to control your perfectionism and embrace the unknown. Feeling stupid is only a problem if you think you should be perfect and smart. If you accept that programming is hard and everyone is stupid until they gain experience then it’s a lot easier. That sounds psychobable but everything you say has nothing to do with technology and is more about your perceptions and attitudes to the technology.

In fact, I have a suggested project for you as a way for you to rid yourself of this ridiculous idea that you should be making perfection. If you’re up to it, I suggest the “100 days of hacks” challenge:

  1. You pick 100 projects that are tiny dumb little tools.
  2. Then you set a limit of maybe 45 minutes.
  3. You hack out an absolute piece of garbage that works.
  4. If you hit the 45 minutes and don’t have something that works you stop and delete what you have. It’s too perfect. Your requirements were too much. Tell that perfectionist in you that you want garbage and to tone down the requirements.
  5. Do it again, 45 minutes max, with less requirements, and hack your brains out without worrying about it being perfect. Nothing’s perfect. Get over it and just make it work, barely.
  6. Once you get something working in 45 minutes move on with your life…unless there’s something interesting in there you want to research.

The absolute worst mental attitude I see for beginning programmers is perfectionism. Programming is a continuous brutal reminder that you are stupid and you suck. Everything about programming is demoralizing to the perfectionist. You literally have a computer yell at you all day saying you are constantly wrong. The most important thing is to just get that under control and accept that you’re just wrong, and you’re not as smart as you think, and nobody else is really and that’s alright.

I mean, if it was easy, then why would anyone get paid to do it?

Finally, if you just ignored everything I said and you want to try your hand at “perfection” and learning the core mathematical principles that underpin everything, then you just need to get “The Art of Computer Programming” by Donald Knuth. I’m going to warn you though that you’re going to get that book thinking, “Yes! Now I’ll go through this book and everything I do will be perfect and everyone will know how smart I really am!” Then, you’re going to do it for about 2 days and run into the same problem: programming is hard and the perfectionist in you can’t handle the brutality of having a computer (and now Donald Knuth) tell you what a failure you are all day. Then you’ll give up. Again.

My way is a lot better. Just get used to making garbage and accepting that it’s hard and a struggle and not very well defined and you’ll do a lot better.

I hope that helps.


#8

Thank you, Zed. One question - at what point should the programmer who has done this exercise pick up a book like SICP or CLRS?


#9

I think if you want to learn to get very solid at computer science concepts then the one topic to focus on is how to build a programming language. The algorithms and math used in programming language theory are solid, extensive, and also practical. It also teaches you all of the things you’d learn by going through a book like SICP or similar, but in a way that applies to every language, past or present. I think compiler design is really the only solid no-bullshit part of computer science, which is why I tell people to learn it. After that just about everything in CS becomes easier.

Typically people run to SICP or CLRS because other supposedly smart people from MIT have said that’s why smart people do. It’s actually a lie. There’s quite a few ways to understand computation, and that’s only one very narrow version of it that honestly doesn’t have much of a foundation to it other than “other smart people said it was best”. It’s best to treat these book recommendations as things they were forced to do, but really didn’t benefit from, or they were told other smart people did that so they go around claiming they do it too.


#10

Wow, I’ve never seen those book recommendations that way, thank you. Why are the other ways to understanding computation not as famous as the MIT-anointed canon? And how do you suggest one goes about learning compiler design and programming languages?


#11

Well, I think you might also be grasping for some magic trick or concept as a way to avoid practice and study. You get better at programming and computational thinking by doing more of it. You write more software, and that’s part of what my advice of “lots of quick and dirty hacks” does. It forces you to write a full complete piece of software from start to finish, but doesn’t let you get bogged down in making it perfect. There really isn’t a magic trick to it. It just takes practice and work.

There’s only a few concepts in the simplest version of computation, so let’s try an experiment. I will explain them, and then you see if that somehow magically makes you better without any practice. Then, go get a piece of Python code from my book that has if-statements and while-loops and see if you can see all of these concepts in it. See if that helps. Then, write a piece of software but try to think about everything as these things:

Simplistic Computation

  1. A way to store and retrieve data.
  2. A way to execute instructions.
  3. A way to jump.
  4. A way to test.
  5. And let’s add a way to do input and output (because without that it’s all useless).

Storing Data

In the simplest form that’s just this:

x = 1

You can also go very bare metal in the CPU and do something like store into a register, but for now just act like the = is “store data”. It puts the data 1 into a spot in memory, and then gives it a name x so you can use it later.

Retrieve Data

That’s just getting it back from memory using the name you gave it:

x

That’s it, so things like this works:

y = x + x + x + x

That’s 4 data retrieve and 1 store into y.

Execute Instructions aka The Tape

You then need a way to tell the computer to do stuff, and that’s instructions. In the CPU this can be thought of as a “tape” (think old school cassette tape) with instructions going in order. In Python this is just the lines of your file, so we could have this:

y = x
y += x
y += x
y += x

That is kind of like making a tape out of the larger line above so it is more like how a computer processes it. One single operation/instruction per line, and then think about the lines as having numbers so we can address them. This is important for the next ones.

Jumps

Just starting at the beginning of a tape and going to the end is kind of stupid. If you want to add 4 numbers, fine that’s a 4 instruction long tape (aka a 4 line long script). If we want to add a million numbers we’d need a million line script (million instruction tape). What we really want then is to loop over any set of lines, which is where a jump comes in. A jump just says, “go to line 3”, like this:

1: y = x
2: y += x
3: jump 2

That weird “jump 2” just means, “go back to line 2 and do it again.” It’s made up by me and not a real thing. That will just loop forever adding x to y, continuously increasing it. If Python that’s basically this:

y = x
while True:
    y += x

Tests

A loop that goes on forever is pointless. I mean, sure, we’d like some kinds of software to run forever, but most of our loops should end. We also would need to make decisions on our jumps. That’s where tests come in, and that’s basically your classic boolean algebra tests:

y == 100

That just tests “is y equal to 100”, and returns true or false (or 1 or 0) depending on that equality. You can also do <=, >=, <, not, and many more.

Tests + Jumps == Logic

Typically though, when you do a test the result gets stored somewhere temporarily, so doing a y == 100 would put some kind of 1 or 0 into a “last test register”, which is just a small piece of temporary space.

If you have a way to test, and then jump based on that test, then you can do all the looping and if-statement constructs you want. Imagine this:

1: x = 1
2: y = x
3: y += x
4: test y != 100
5: jtrue 3

What’s that jtrue? It’s a thing I made up that stands for “jump if iftrue”, and kind of comes from assembly language. It’s like saying, “if the last test results in true then go to line 3”. And remember my made up assembler the test tells it to do the boolean test y == 100 and then record if that was true (1) or false (0).

I could kind of pythonify that with this:

1: x = 1
2: y = x
3: y += x
4: if y != 100: jump 3

Now, what do those chunks of code do? Well, this:

x = 1
y = x
while y != 100:
   y += x

What about if-statements? Let’s say I don’t know what’s in a variable p and need to decide something:

1: test p < 100
2: jtrue 10
3: # greater than 100 code starts here
...
9: jump 11
10: # less than 100 code starts here
11: # code is done so we jump here to get out of this test

This is a simple if-else like this:

if p < 100:
   # less than 100 code starts here
else:
   # greater than 100 code starts here

But as with a lot of CPU code we have to invert it because we only have jumping and not kind of fancy nested constructs like with Python’s if-else.

Input and Output

Finally, with all of that we need a way to read in some data and write it out. That’s usually done as input and output, so if I took the above if-statement and had an read and write instructions I could do:

1: read p
2: test p < 100
3: jtrue 6
4: write "That number is greater than 100."
5: jump 7
6: write "That number is less than 100."
7: # continue with the code

Already this is getting nasty but it’s kind of fun. We have to figure out how to do tests, and then jump over lines and also jump over the other lines from there. In Python the above code is:

p = input()
if p < 100:
    print("That number is less than 100.")
else:
    print("That number is greater than 100.")

You can see why we kind of stopped bothering with this raw machine code because it gets really hairy even in simple cases like this. BUT, with just these 5 things you can actually do all of the programming we have right now (although really horribly). At least I think so. I might be missing something but this should cover everything.

Concepts to Application

Alright so you now have the 5 things that are at the center of all computation. Since I can tell that you’re hoping for something like this I suggest you take it and analyze it, and then see if you can break down a very small Python program into this fake CPU language I have here. Then try the reverse and convert a fake CPU into Python.

Once you do that, see if you can find these concepts being used everywhere. Then, it’s time to blow your mind out of the water with this:

https://docs.python.org/3/library/dis.html

THAT is basically Python’s fake assembler code. Try using that to dump the contents of simple functions and see if you can figure out what they do. There’s a complete list of Python’s instruction set. So, maybe learning that will help you figure out what’s going on and start to feel more confident in what you’re writing.


#12

@zedshaw that blows my mind! Awesome!


#13

I see your point now. Knowing how the assembly code works behind a Python function would not magically help me gain the dexterity that only practice and time would.

However, I want to take the 100 days of hacks challege but I would not be very motivated to do these hacks if I didn’t somehow find them interesting. So can you propose to me the first 10 hacks but these hacks must have something to do with compilers/PLs. For instance, hack 1 - Write a simple parser for arithmetic expressions.


#14

Hi @letroot From what I understand from your writing, it seems to me you have learned in your life, that you only can do things if somebody tells you what to do (no pun intended).
But as you write in your posts you already have a clear vision in mind what you want to know and where you want to go.

And you already know what first steps you need to take. Take that first step “write a simple parser for arithmetic expressions” and split it up in 45 min hacks. When needed take a break to investigate topics you don’t understand. So it will be a mixed experience: learning and doing. From what you learn you can then prepare the next steps to reach your goal. It will not get boring, I promise you.

At the moment I do the same thing with learning Django. I know what I want to accomplish and hack my way through. I have to read tutorials, dig through the documentation, watch videos, search on StackOverflow and learn some basics about ORM, testing, etc. On the same time I hack on my desired Django App and it gets closer to the point I want it to be, every day a bit.

Doing this you will learn a lot and discover topics you never heard before and then you have to learn and implement them.

Be your own master, you don’t need anybody else to tell you what to do. You know your goal, go for it! :facepunch:


#15

A simple parser for arithmetic language is kind of hard. I’m a big fan of starting off doing copies of unix command line tools. How about, instead of arithmetic you write a command line options parser kind of like this:

https://docs.python.org/3/library/argparse.html

You can either copy that as exactly as possible, or try to make your own. I recommend maybe trying to copy it then try something different.

After that, try doing copies of:

  • ls
  • find
  • grep
  • cat
  • less
  • time
  • rm
  • mv

Believe it or not, doing copies of those tools will teach you a mega ton of the first steps to understanding little languages, and also how your basic unix system works.


#16

Also, I’m going to hammer home one extra important point:

Making very detailed copies of another work is an entirely valid way to learn and has been used by great masters in every single discipline to learn how to do something.

Don’t let someone tell you that you have to invent something new from nothing or you’re not good enough. All the greats copy and steal every idea they can. So can you.


#17

I have worked through those Unix tool copying exercises in LMPTHW, that’s why I wanted to move on to the exercises I proposed. What do you think?


#18

Thank you @DidierCH for the encouragement.


#19

Ah, sweet, you have More Python, then jump to the little languages section:

https://shop.learncodethehardway.org/paid/more-python/part4.html

Give it a shot, but keep in mind you’re going to want to do it slowly, and you’re going to want to try doing the machine code dump to get an idea of how it’s working internally. Try it out. Might just be all you have to do is trick your brain into thinking you’ve learned everything and then it will let you relax and use it.


#20

Okay, thank you. I’ll get back to it.