Question

Is there value in writing everything down?

Answer

I had this question when I started my research journey towards AGI. At one point I realized I wouldn't be able to hold everything in my brain and that I needed to write down what I was thinking about so that I would be able to come back later, re-read what I wrote and continue making progress on what I was working on, whether it had been a day, a few weeks, or many months.

There is definite value in writing everything down. It frees your mind, letting you delegate the task of remembering to the computer. A thought I might have had more than five years ago can still be accessible to me if I wrote it down, but it's unlikely I'll remember exactly what I was thinking five years ago by "thinking really hard". Nowadays I have trouble even remembering what I was working on a few hours prior due to the high volume of interruption I face in my work. Writing things down allows me to keep a record of what I'm doing.

One of the main concerns of writing everything down is that it can be seen as a waste of time. We often write and never read what we wrote again. As I write more and more, it becomes less and less likely that I will read what I've written in the past because the volume of content increases requiring more time to read. For the same amount of time spent reading, I cover the same constant amount of content, but the amount of uncovered content keeps growing, assuming I review prior content more than once.

I don't view writing that is not read again to be wasteful. Sometimes writing down our thoughts allows us to figure out an issue that is blocking us. Sometimes expressing what we feel makes us realize that the feelings themselves are very hard to explain. Sometimes we just need a buffer where we can jot things down and get rid of it once we're done.

Writing (and sharing) has the benefit that your thoughts can be reused by others. They can also be reused by your future self if you end up facing the same situation again.

References

• How often should I review my personal notes?
• Research methodology and organization

Question

What is the most complex sequence that can be made from a n-long binary string?

Answer

0....0 and 1....1 are easily compressible, thus what is the most unpredictable sequence?

Let's proceed by induction.

1-long binary string

0 and 1 have a 50% to be selected in a 1-long binary string, thus they are both the most complex examples of a 1-long binary string.

2-long binary string

With two bits, we introduce two patterns: repeating and alternating. Repeating means that the bits are repeated for the complete string. Alternating means that the bits are alternating between 0 and 1 given a certain periodicity.

From our initial observation we suggested that the repeating pattern was likely to be the simplest because it consists of defining the symbol (0 or 1) that is repeated and the number of repetitions. We can say that this function has two parameters: length and symbol.

The same can be said about the alternating pattern. If you provide the initial symbol and the length of the pattern, you've defined its parameters. We can say that this function also has two parameters: length and starting symbol.

Using those two descriptions, we can observe that the alternating pattern is in fact a special case of the repeating pattern, where the symbol is 2 bits instead of one.

00 = repeating(2, 0)
01 = alternating(2, 0) = repeating(1, 01)
10 = alternating(2, 1) = repeating(1, 10)
11 = repeating(2, 1)

3-long binary string

001, 011, 100, 110 can be described as alternating and repeating, which is more complex than the previous examples as it requires the composition of two operations.

4 complex examples (001, 011, 100, 110) have been introduced at this stage.

4-long binary string

Here we first observe the presence of the alternating pattern with a periodicity of 2. We also observe the first case of a mirror pattern with 0110 and 1001. It could be claimed that 010 and 101 is the uneven equivalent of the mirror pattern for 3-long binary strings. The remaining "complex" examples are again displaying alternating and repeating patterns at the lower level.

One observation is that the complex patterns appear to be the variants where 1 bit is the opposite of the remaining bits, the only special case so far being for the 3-long binary string where the pattern is considered alternating. Instead of considering it a complex pattern, it could be described as the operation "fill X bits with 0/1, then flip by Y".

From those few inductive iterations, we can see that the first and last string are always the repeating pattern. We also observe that the first half of the strings exhibit the same pattern of the second half of the strings, which is expected due to the binary nature of the string.

At this point there is still not enough information to be able to derive an answer to the original question, given that the strings can be described using 4 patterns: repeating, alternating, mirror, and fill & flip.

8 complex examples are found at this stage.

5-long binary string

The second half of this table was not written down due to the previous observation that the patterns are the same, simply inverted.

At this stage, the complex patterns that emerge are difficult to express using the prior language. Composition needs to be used, e.g. 00101 can be described as 00 then 101.

14 complex examples are found at this stage.

At this point in my study, I would summarize as follows:

• Repeating, alternating and fill & flip are low complexity. For a n-long binary string, this implies 2 repeating patterns, 2n fill & flip patterns, 2n alternating patterns of periodicity=1.
• Complexity appears to emerge from alternating patterns with varying periodicity. As we're working with longer and longer strings, the periodic patterns will start to become periodic themselves (I believe this is leading to fractal).
• The fill & flip pattern appears to be able to cover most of the complex cases that appear as the length of the binary string increases.
• Some of the patterns can be expressed as a few alternatives, for example 01110 as alternating, periodicity=1,3,1, but also as 0 and fill & flip (or even an "odd" mirror). My intuition would lead me to think that those cases that can be expressed in multiple ways are less complex.
• It wasn't done during this analysis, but it may be possible to apply operations overlayed on one another, such as two alternating patterns, one with a periodicity of 2 and one with a periodicity of 3, and use logic operators such as AND/OR/XOR (and their negative variants) to recreate the pattern
• The sequence of complex examples seems to be as follows: 0, 4, 8, 14, ...
• It would be interesting to look at those strings from a logic circuit point of view. For example, 00000 can be produced from a single signal (0) routed to 5 exits, 00001 would be a single signal (0) routed to 4 exits and to one not gate which is then routed to an exit. Assuming the input signal is always 0, producing 11111 would require 1 not gate.

Question

What can cause us to learn inefficiently?

Answer

There are many reasons which can cause someone to learn at a slower pace than what should be possible for them. This can be caused by many factors, amongst them:

• Learning the same material under different wordings: You are going through material which you've already read, thus this "new" material is not providing you with any fresh information.
• Learning outdated material: You are learning material that has already been replaced with better material. It may be useful to learn outdated material for the historical purpose, but you should always aim to know what is the latest material.
• Learning incorrect material: You are learning material that is effectively not providing you with the information you want to learn. It may appear like you are learning something, but learning the wrong facts is a waste of your time.
• Learning from a source that makes the material difficult to understand: Learning efficiently is about learning from the best sources. If the material you have makes it difficult to understand what you are trying to learn, you should look for content elsewhere.
• Learning material for which we do not know the prerequisites: Learning a topic when we do not have the prerequisite knowledge will generally lead to confusion and make it difficult, if not impossible to learn.
• Learning from multiple sources with contradictory information: Learning from sources that disagree on facts makes it difficult to learn because you have to either accept that both answers are true, which can be complicated when they contradict each other, or that you need to establish which answer is the correct one, which requires additional effort.
• Learning while not being completely focused on the task: When learning, it is imperative to dedicate our focus to the task. Not doing so is likely to lead to information falling through the cracks of focus, leading to gaps in understanding, leading to requiring to go through the material again.

What is important to do while learning is to recognize when certain behaviors or circumstances make learning less efficient than desired and writing those behaviors or circumstances down so that you may have your own list of causes for learning less efficiently. As you discover those causes, make sure to put in place some contingency plans that will help you reduce their impact on your learning abilities.

Question

How often should I review my personal notes?

Answer

The simple answer is to do it as often as possible.

The more often you can go through your notes and review them, the more they will be fresh in your mind.

However, as you write more and more notes, it will be difficult to go through them all. At one point you will need to start prioritizing what you want to review and at what frequency. I see writing the same way I see programming: certain parts of the code work well and don't need to be touched for a while, so they are left alone. However, sometimes certain parts have become neglected with time and need to be looked at again.

All my notes on this blog, as well as my private project and personal notes are committed to git automatically every time I switch out of VS Code. This means that I have a very granular history of the last time a file was touched. This allows me to look back at the list of files I have and to find the files that haven't been touched in a while. Sometimes I'll read them again, think about them a bit, then close them if I have nothing to add. Other times I'll write about something that I've learned since then. Doing this feels a lot like improving some part of a codebase after you've learned how a piece of code would be used.

Another way you can schedule reviewing your notes would be with a spaced repetition system like Anki. It wouldn't be too much work to have a system that automatically creates an Anki note/card with a link to the file itself. You would click on the link, which would open the file in your text editor of choice, read the note, change it if needed, then close it. You would then go back to Anki and pick between hard/good/easy to decide when is the next time you would review the note.

I'd suggest dedicating a small amount of your week going through what you wrote and refreshing your memory. It is definitely useful to simply write things to get them out of your mind and to attempt to express your thoughts clearly. It's also useful to review what you thought about because your subconscious might have kept working on those thoughts well after you wrote them down and there might be more to contribute on a topic.

Question

What is the point of asking ourselves many questions when we only answer a few of them?

Answer

The purpose of asking yourself questions is to prompt your subconscious to look for the answer.

Asking yourself questions is a way to determine if you know or do not know the answer. If you don't know the answer, then nothing will come on the top of your head quickly. On the other hand, if you know an acceptable answer, it will show itself rather quickly.

Asking yourself questions is a way to explore the domain of your knowledge and look for areas where you might need to do additional research.

If you ask yourself a lot of questions on a certain topic you're studying and you don't know the answers, then it can let you know that your knowledge is lacking and that you need to study more.

In some cases you might ask yourself questions that you don't really want to know the answer to.

One of the main reason a lot of questions remain unanswered is because asking questions is easy, but answering is hard.

References

• Answering your own questions