I’m not really sure how to describe it other than when I read a function to determine what it does then go to the next part of the code I’ve already forgotten how the function transforms the data

This sounds to me like you could benefit from mentally using the information hiding principle for your functions. In other words: Outside of the function, the only thing that matters is “what goes in?” and “what comes out?”. The implementation details should not be important once you’re working on code outside of that function.

To achieve this, maybe you could write a short comment right at the start of every function. One to two sentences detailing only the inputs/output of that function. e.g. “Accepts an image and a color and returns a mask that shows where that color is present.” if you later forget what the function does, all you need to do is read that one sentence to remember. If it’s too convoluted to write in one or two sentences, your function is likely trying to achieve too much at once and could (arguably “should”) be split up.

Also on a different note: Don’t sell your ability to “cludge something together” short. If you ever plan to do this professionally or educationally, you will sadly inevitably run into situations where you have no choice but to deliver a quick and dirty solution over a clean and well thought out one.

Edit: typos

It’s a lot to take in at first, but the problem isn’t that you’re really doing something wrong or that you’re lacking some piece of information. You just need to actually write more code. It will help you understand why something is set up the way that it is, and also help things make a lot more sense. Once you have that foundation of experience, a lot of other things fall into place on their own for lack of a better explanation. In other words: practice makes perfect

Yes, I too used to struggle with this.

---

Debugging

Learn how to debug. For me, it’s a lifesaver for me to be able to step through some code to figure out what it actually does, instead of me trying to read the code to figure out what it may do. Yes, I do this for my own code too, because we tend to make assumptions, and I want to confirm those, always.

That means learning how to setup your IDE of choice - I presume you use vscode, so you’ll then have to google for “vscode debugging”. Maybe you’ll have to install some addons to add the support, probably setup some launch.json in a local .vscode folder. It all depends on your language of choice.

---

Learn how to test. This goes great with debugging. I write code in Python, so I end up creating src/ and tests/ folders. src/ for my actual code, and tests/ for my tests. I can use either pytest on the terminal, or just the vscode test addons to run tests.

Anyway, my tests end up being something like this:

src/my_app/main.py or something, with src/my_app/__init__.py existing to turn that folder into a module:

def main():
    # some code I want to run

Then in tests/test_main.py (mirroring the src/ folder; adding test_ makes the file findable for pytest, and I call it main to make it easier to link to the main code):

from my_app import main

def test_main():
    main()

This is how I always start off with - just a simple piece of code that does a thing, and a test with almost the same name as the function I’m trying to test. I can now add a breakpoint inside test_main and run the test within vscode, which means I have a way of hooking into the main function.

---

Think about the process of your application

Think about how to cut up the steps to create your application into smaller and smaller steps. Whenever something feels insurmountable, I’ll just have to stop in my tracks and mentally cut up a task into smaller and smaller steps, until I feel comfortable to take some steps.

I’m a data engineer, which means I tend to write code to ‘ingest’ data (which means, grab it from source A and put it into target B (where B is some centralized location to store all raw data).

So the main task is:

1. Write ingestion

I then have to figure out “what is the source”, because that dictates how I grab the data (do I have to loop over all folders in an SFTP server? Is there a state file that makes my life easier? Do I use an API instead?)

1. Write ingestion
   1. figure out what the source is
   2. Is there an SFTP state file? is it an API?
   3. Do I need a username/password? Some API key?

I then start writing a small piece of code that connects to the source, or just grabs some random data to show the connection works.

So now I can grab some data. Is that data too large to ingest all at once? If a file is super large, I may not be able to hold it into data, which means using a buffer. And how many files are there to download? Should I batch those?

1. Write ingestion
   1. figure out what the source is | SFTP
   2. Is there an SFTP state file? is it an API? | there is a state file
   3. Do I need a username/password? Some API key? | usename/password
   4. How big are the files?
   5. How many files are there?

and this is how I slowly grow my applications from an idea “ingest all data from some source” into something that can actually run.

Now, I do have some experience and know that filesize and filecount are important to take into account, but that’s something I learned along the way.

My first applications just loaded whole files into memory (a bad idea if your memory limit is 4 GB, and I’m trying to load multiple 1GB sized files into memory 😆), and taking local state (which files have I already downloaded) and external state (which one have updated or been added?) into account, etc.

Anyway, you’re already on the right path: You already know a weak point, and you’re smart enough to know your limits and ask for help when you’re stuck. That’s one of the fastest ways to grow as a programmer.

I’ll often cludge something together just to make it work but I don’t feel like I made any progress

That’s a good first step! I’ve been programming for ~25 years and that’s still usually where I start. Get a little code that compiles and produces some kind of output or tracing. Then compare the output to your requirements and tweak the code to get it closer to the right behavior. Run it and repeat till it’s doing what you want. Do this cycle with small changes, like a handful of lines or a short function, not 20 mins of coding at a time.

Test-driven development can also help with breaking down tasks. It takes a good amount of practice to learn the right patterns, but it’s an approach that forces you to work with small narrowly scoped tasks. Then you chain those testable tasks together to create more complex behaviors to create robust testable code.

Experience takes time. Junior developers frequently ask me after I’ve helped them “but how did you just know how to do that? I’ve been trying to solve that for an hour and you did it in 10 seconds!!” The answer is because I’ve solved that exact problem before. More than a few times.

Her is my take to try to help you.

If it’s your own code, you can add docstrings comments to your functions, so you don’t have to re-read the function body everytime. Also, name functions to be understandable more easily when possible.

If not your code, write on a piece of paper (not on computer) the in and out of a function, maybe like so:

 [1,2,3] -> (sum function) -> 6

Then, you can even connect the functions together and see the whole algorithm:

[1,2,3] -> (sum) -> (multiplyBy2) -> (...) -> final_result

When projects get more complex, paper will not cut it, then some note taking app of some sort will help. (logseq could help, but some mind mapping or sequence diagram programs would help as well)

Also, I don’t know what language your are working with, but learning LISP (maybe clojure) could help.
Why? Because you have to connect your functions together, and it forces you to do so.
At first, it might be harder compared to what you’re used to, but it’ll give you better fundations to keep learing.

The name of the function, what goes in and what goes out in most cases should be enough to get a good idea on what the function does.

It also helps to make a diagram of how everything ties together. Just boxes and arrows is enough.

When writing your own code, it takes a bit of experience to know when to put something in its own function. It’s very obvious when you’re replicating code. It’s also very common to cut things up when a function gets too big. Look for bits of functionality that you can give a good name.

You might be trying to hold too many details in your head at once. Once you’ve read a function, try to reframe it in a simpler and easier way; for example, instead of remembering all the steps it makes, just remember what goes in and what comes out (and any side effects). This will let you think about how different functions interact, not just different lines of code.

Write it down when you figure something out. Draw arrows to it from some other part you figured out. Scratch it out when you realize you had it wrong and then put the arrows to a new place where you doodle how it actually works. Never look at the notes again.

Is this with your own code or someone else’s? It’s always harder to understand someone else’s code (at least at first). Everyone thinks and writes in a different way.

In either case, I think you could benefit from stepping through the code in a debugger. Depending on what the code is, give some data as input where you know/can guess what the output is. Using the debugger, step through each line to see what happens to the data. It can help break down long or complicated functions into simpler chunks.

Recognition of functions or snippets of code will come through repetition and exposure. Writing code helps more than reading as well. Even with all of that, it’s still okay and common to have to look things up or review. I constantly have to check the syntax of C++ library functions, like snprintf, which I have used but not enough to memorize (and that’s okay). Don’t be discouraged. I’ve been in my career for 11 years now, around 9 of which is working with embedded C++, and I still feel like an imposter.

The main part you need to pick up is being able to establish the mental hooks around the ideas that are central to programming. Do you know how you can watch a choreography session and see the dancers just pick up the moves as they’re described/demonstrated? That’s because they’ve learned the language of dance. It’s an entire (physical) vocabulary. It’s the semantics of dance.

What you need to do is do that with programming. There’s a number of getting started with books and videos, but you’re going to want them to learn the fundamentals of not just a language but of programming.

If you’re talking about using other people’s functions (like in an api), then the function name should give you a clue about what it does. The cool thing about functions is that you don’t have to know how they’re doing their thing, just what they’re doing. If you have the source code, you will find you remember more if you use comments to make notes for yourself (it engages more of your brain than just reading).

If your problem is writing your own code using functions, start out more slowly. Write a program that’s just a giant block of linear code. Once that’s working, then take a look as to how to break it down into functions. If you have a block of code that sorts a list, for example, and you had to copy and paste it into three different areas, that would mean it should be a function.

Use comments very often as you’re going. Before you write a block, write a comment about what it’s supposed to do. You’ll start to see some generalities, which will be you learning programming, not just a language.

There is a great similarity between watching the variables in a function and juggling. Psychologists will tell you, ‘you only get seven variables’, but yiu’re a coder, you might have ten to twelve( especially if some are boolean), but it’s finite, so chunk. Interestingly learning to juggle may help, mirror neurons go brrrr., also chess with moves ahead. If it works it’s not stupid, but is it repeatable, testable…

Difficult concepts are made of simple parts. Find the parts. Find ways to play with them.

Example: TCP/IP performance is complicated, but you can look at specific parts of it like packet latency, retransmission behavior, and the various timeouts. Eventually you understand the “gears” of the system well enough that it’s obvious that if the minimum retransmission timeout is 100msec, that a single packet loss means your whole transaction cannot possibly have <100msec latency.

Removed by mod