a lot of stuff

1 files changed, 582 insertions, 0 deletions

diff --git a/blog/cognition.org b/blog/cognition.org
new file mode 100644
index 0000000..f331ba0
--- /dev/null
+++ b/blog/cognition.org
@@ -0,0 +1,582 @@
+#+title: Cognition
+#+author: Preston Pan
+#+description: Other languages are inflexible and broken. Let's fix that.
+#+html_head: <link rel="stylesheet" type="text/css" href="../style.css" />
+#+html_head: <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
+#+html_head: <link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
+#+html_head: <link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
+#+html_head: <link rel="manifest" href="/site.webmanifest">
+#+html_head: <link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5">
+#+html_head: <meta name="msapplication-TileColor" content="#da532c">
+#+html_head: <meta name="theme-color" content="#ffffff">
+#+html_head: <meta name="viewport" content="width=1000; user-scalable=0;" />
+#+language: en
+#+OPTIONS: broken-links:t
+
+* Introduction
+Cognition is an active research project that Matthew Hinton and I have been working on for the past
+couple of months. Although my commit history for [[https://github.com/metacrank/cognition][this project]] has not been impressive, we came up with
+a lot of the theory together, working alongside each other in order to achieve one of the most generalized
+systems of syntax we know of. Let's take a look at the conceptual reason why cognition needs to exist, as
+well as some /baremetal cognition/ code (you'll see what I mean by this later). There's a paper about this language
+available about the language in the repository, for those interested. Understanding cognition might require a
+lot of background in parsing, tokenization, and syntax, but I've done my best to write this in a very understandable way.
+The repository is available at https://github.com/metacrank/cognition, for your information.
+* The problem
+Lisp programmers claim that their system of s-expression code in addition to its featureful macro system makes it a
+metaprogrammable and generalized system. This is of course true, but there's something very broken with lisp: metaprogramming
+and programming /aren't the same thing/, meaning there will always be rigid syntax within lisp
+(its parentheses or the fact that it needs to have characters that tell lisp to /read ahead/). The left parenthesis tells
+lisp that it needs to keep on reading until the right parenthesis in order to finish some process that allows it to stop
+and evaluate the whole expression. This makes the left and right parenthesis unchangable from within the language (not
+conceptually, but under some implementations it is not possible), and, more importantly, it makes the process of retroactively
+changing the sequence in which these tokens are delimited /impossible/, without a heavy amount of string processing. Other
+langauges have other ways in which they need to read ahead when they see a certain token in order to decide what to do.
+This process of having a program read ahead based on current input is called /syntax/.
+
+And as long as you read ahead, or assume a default way of reading ahead, you fall into the trap of having some form of syntax.
+Cognition is different in that it uses an antisyntax that is fully /postfix/. This has similarities with concatenative
+programming languages, but concatenative programming langauges also suffer from two main problems: first, the introduction
+of the left and right bracket character (which is in fact prefix notation, as it needs to read ahead of the input stream),
+and the quote character for strings. This is unsuitable for such a general language. You can even see the same problem
+in lisp's C syntax implementation: escape characters everywhere, awkward must-have spaces delimit the start and end
+of certain tokens (and if not, it requires post-processing). The racket programming language has its macro system,
+but it is not /runtime dynamic/. It still utilizes preprocessing.
+
+So, what's the percise solution to this connundrum? Well, it's beautiful; but it requires some /cognition/.
+
+* Baremetal Cognition
+Baremetal cognition has a couple of perculiar attributes, and it is remarkably like the /Brainfuck/ programming language.
+But unlike its look-alike, it has the ability to do some /serious metaprogramming/. Let's take a look at what the
+bootstrapping code for a /very minimal/ syntax looks like:
+#+begin_example
+ldfgldftgldfdtgl
+df 
+ 
+dfiff1 crank f
+#+end_example
+And *do* note the whitespace (line 2 has a whitespace after df, and the newlines matter). Erm, okay. What?
+
+So, our goal in this post is to get from a syntax that looks like /that/ to a syntax that looks like [[file:stem.org][Stem]].
+But how on earth does this piece of code even work? Well, we have to introduce two new ideas: delimiters, and ignores.
+
+** Tokenization
+Delimiters allow the tokenizer to figure out when one token ends and another begins. The list of single character tokenizers
+is public, allowing that list to be modified and read from within cognition itself. Ignored characters are characters
+that are completely ignored by the tokenizer in the first stage of every read-eval-print loop; that is, at the start of
+collecting the token, it fist skips a set of ignored characters. By default, every single character is a delimiter, and
+no characters are ignored characters. The delimiter and ignored characters list allows you to toggle a flag to tell it
+to blacklist or whitelist the given characters, adding brevity (and practicality) to the language.
+
+Let's take the first line of code as an example:
+#+begin_example
+ldfgldftgldfdtgl
+#+end_example
+because of the delimiter and ignored rules set by default, every single character is read as a token, and no character
+is skipped. We therefore read the first character, ~l~. By default, Cognition works off a stack-based programming language
+design. If you're not familiar, see the [[file:stem.org][Stem blogpost]] for more detail (in fact if you're not familiar this /won't work/
+as an explanation for you, so you should see it, or read up on the /Forth/ programming language).
+Though, we call them /containers/, as they are more general than stacks. Additionally, in this default environment, /no/
+word is executed except for special /faliases/, as we will cover later.
+
+Therefore, the character ~l~ gets read in and is put on the stack. Then, the character ~d~ is read in and put on the stack.
+But ~f~ is different. In order to execute words in Cognition, we must take a look at the falias system.
+** Faliases
+Faliases are a list of words that get executed when they are put on the stack, or container as we will call it in the future.
+All of them in fact execute the equivalent of ~eval~ in stem but as soon as they are put on their container. Meaning, when
+~f~, the default falias, is run, it doesn't go on the container, but rather executes the top of the container which is ~d~.
+~d~ changes the delimiter list to the string value of a word, meaning that it changes the delimiters to /blacklist/ only
+the character ~l~ as a delimiter. Everything else by default is a delimiter because everything by default is parsed
+into single character words.
+** Delimiter Caveats
+Delimiters have an interesting rule, and that is that the delimiter character is excluded from the tokenized word
+unless we have not ignored a character in the tokenization loop, in which case we collect the character as a part of
+the current token and keep going. This is in contrast to a third kind of tokenization category called the singlet, which
+/includes/ itself into a token before skipping itself and ending the tokenization collection.
+
+In addition, remember what I said about the /blacklist/? Well, you can toggle between /blacklisting/ and /whitelisting/
+your list of delimiters, singlets, and ignored characters. By default, there are no /blacklisted/ delimiters, no
+/whitelisted/ singlets, and no /whitelisted/ ignored characters.
+
+We then also observe that all other characters will simply skip themselves while being collected as a part of the current
+token, without ending this loop, therefore collecting new characters until the loop halts via delimiter or singlet rules.
+** Continuing the Bootstrap Code
+So far, we looked at this part of the code:
+#+begin_example
+ldf
+#+end_example
+which simply creates ~l~ as a non-delimiter. Now, for the rest of the code:
+#+begin_example
+gldftgldfdtgl
+df 
+  
+dfiff1 crank f
+#+end_example
+~gldf~ puts ~gl~ on the stack due to ~d~ being a delimiter, and ~f~ is called on it, meaning that now ~g~ and ~l~ are
+the only non-delimiters. Then, ~tgl~ gets put on the stack and they become non-delimiters with ~df~. ~dtgl~ gets
+put on the stack, and the newline becomes the only non-delimiter with ~\ndf~ (yes, the newline is actually a part of
+the code here, and spaces need to be as well in order for this to work). Then, the space character, due to how delimiter
+rules work (if you don't ignore, the first character is parsed normally even if it is a delimiter)
+and ~\n~ gets put on the stack. Then, another ~\ \n~ word is tokenized (you might not see it, but there's another
+space on line 3). The current stack looks like this (bottom to top):
+#+begin_example
+3. dtgl
+2. [space char]\n
+1. [space char]\n
+#+end_example
+~df~ sets the non-delimiters to ~\ \n~. ~if~ sets the ignores to ~\ \n~, which ignores these characters at the start
+of tokenization. ~f~ executes ~dtgl~, which is a word that toggles the /dflag/, the flag that stores the whitelist/blacklist
+distinction for delimiters. Now, all non-delimiters are delimiters and all delimiters are non-delimiters.
+Finally, we're put in an environment where spaces and newlines are the delimiters for tokens, and they are ignored at the
+start of tokenizing a token. Next, ~1~ is tokenized and put on the stack, and then the ~crank~ word, which is then executed
+by ~f~ (the ~1~ token is treated as a number in this case, but everything textual in cognition is a word).
+We are done our bootstrapping sequence! Now, you might wonder what ~crank~ does. That we will explain in a later section.
+
+* Bootstrapping Takeaways
+From this, we see a couple principles: first, cognition is able to change how it tokenizes on the fly and it can do it
+programmatically, allowing you to program a program in cognition that would theoretically automate the process of changing
+these delimiters, singlets, and ignores. This is something impossible in other languages, being able to
+/program your own tokenizer for some foreign language from within cognition/, and have
+/future code be tokenized exactly like how you want it to be/. This is solely possible because the language is postfix
+and doesn't read ahead, so it doesn't require more than one token to be parsed before an expression is evaluated. Second,
+faliases allow us to execute words without having to have prefix words or any default execution of words.
+
+* Crank
+The /metacrank/ system allows us to set a default way in which tokens are executed on the stack. The ~crank~ word takes
+a number as its argument and by effect executes the top of the stack for every ~n~ words you put on the stack. To make
+this concept concrete, let's look at some code (running from what we call /crank 1/ as we set our environment to
+crank one at the end of the bootstrapping sequence):
+#+begin_example
+5 crank 2crank 2 crank
+1 crank unglue swap quote prepose def
+#+end_example
+the crank 1 environment allows us to stop using ~f~ in order to evaluate tokens. Instead, every /1/ token that is
+tokenized is evaluated. Since we programmed in a newline and space-delimited syntax, we can safely interpret this code
+intuitively.
+
+The code begins by trying to evaluate ~5~, which evaluates to itself as it is not a builtin. ~crank~ evaluates and puts
+us in 5 crank, meaning every /5th/ token evaluates from here on. ~2crank~, ~2~, ~crank~, ~1~ are all put on the stack,
+leaving us with a stack that looks like so (notice that ~crank~ doesn't get executed even though it is a bulitin because
+we set ourselves to using crank 5):
+#+begin_example
+4. 2crank
+3. 2
+2. crank
+1. 1
+#+end_example
+~crank~ is the 5th word, so it executes. Note that this puts us back in crank 1, meaning every word is evaluated.
+~unglue~ is a builtin that gets the value of the word at the top of the stack (as ~1~ is used up by the ~crank~ we
+evaluated), and so it gets the value of ~crank~, which is a builtin. What that in effect does is it gets the function
+pointer associated with the crank builtin. Our new stack looks like this:
+#+begin_example
+3. 2crank
+2. 2
+1. [CLIB]
+#+end_example
+Where CLIB is our function pointer that points to the ~crank~ builtin. We then ~swap~:
+#+begin_example
+3. 2crank
+2. [CLIB]
+1. 2
+#+end_example
+then ~quote~, a builtin that quotes the top thing on the stack:
+#+begin_example
+3. 2crank
+2. [CLIB]
+1. [2]
+#+end_example
+then prepose, a builtin like ~compose~ in stem, except that it preposes and that it puts things in what we call a VMACRO:
+#+begin_example
+2. 2crank
+1. ( [2] [CLIB] )
+#+end_example
+then we call ~def~. This defines a word ~2crank~ that puts ~2~ on the stack and then calls a function pointer pointing
+us to the crank builtin. Now, we still have to define what VMACROs are, and in order to do that we might have to explain
+some differences between the cognition stack and the stem stack.
+** Differeneces
+In the stem stack, putting words on the stack directly is allowed. In cognition, words are put in containers when
+they are put on the stack and not evaluated. This means words like ~compose~ in stem work on words (or more accurately
+containers with a single word in them) as well as other containers, making the API for this language more consistent.
+Additionally, words like ~cd~ as we will make use of this concept.
+
+*** Macros
+Macros are another difference between stem quotes and cognition containers. When macros are evaluated, everything in
+the macro is evaluated, ignoring the crank. If bound to a word, evaluating that word evaluates the macro which will ignore
+the crank completely and will only increment the cranker by one, while evaluating each statement in the macro. They
+are useful for making crank-agnostic code, and expanding macros is very useful for the purpose of optimization, although
+we will actually have to write the word ~expand~ from more primitive words later on (hint: it uses recursive ~unglue~).
+** More Code
+Here is te rest of the code in ~bootstrap.cog~ in ~coglib/~:
+#+begin_example
+getd dup _ concat _ swap d i 
+_quote_swap_quote_compose_swap_dup_d_i eval 
+
+2crank ing 0 crank spc
+2crank ing 1 crank swap quote def
+2crank ing 0 crank endl
+2crank ing 1 crank swap quote def
+2crank ing 1 crank
+2crank ing 3 crank load ../coglib/ quote
+2crank ing 2 crank swap unglue concat unglue fread unglue evalstr unglue
+2crank ing 1 crank compose compose compose compose VMACRO cast def
+2crank ing 1 crank
+2crank ing 1 crank getargs 1 split swap drop 1 split drop
+2crank ing 1 crank
+2crank ing 1 crank epop drop
+2crank ing 1 crank INDEX spc OUT spc OF spc RANGE
+2crank ing 1 crank concat concat concat concat concat concat =
+2crank ing 1 crank
+2crank ing 1 crank missing spc filename concat concat dup endl concat
+2crank ing 1 crank swap quote swap quote compose
+2crank ing 2 crank print compose exit compose
+2crank ing 1 crank
+2crank ing 0 crank fread evalstr
+2crank ing 1 crank compose
+2crank ing 1 crank
+2crank ing 1 crank if
+#+end_example
+Okay, well, the syntax still doesn't look so good, and it's still pretty hard to get what this is doing. But the
+basic idea is that ~2crank~ is a macro and is therefore crank agnostic, and we guarantee its execution with ~ing~, another
+falias (because it's funny). Then, we execute an ~n crank~, which standardizes what crank each line is in (you might
+wonder what ~ing~ and ~f~'s interaction is with the cranker. It actually just guarantees the evaluation of the previous
+thing, so if the previous thing already evaluated ~f~ and ~ing~ both do nothing). In any case, this defines words that
+are useful, such as ~load~, which loads something from the coglib. It does this by ~compose~-ing things into quotes and
+then ~def~-ing those quotes.
+
+The crank, and by extension, the metacrank system is needed in order to discriminate between /evaluating/ some tokens
+and /storing/ others for metaprogramming without having to use ~f~, while also keeping the system postfix. Crank
+is just one word that allows for this type of behavior; the more general word, ~metacrank~, allows for much more
+interesting kinds of syntax manipulation. We have examples of ~metacrank~ down the line, but for now I should explain
+the /metacrank word/.
+** Metacrank
+~n m metacrank~ sets a periodic evaluation ~m~ for an element ~n~ items down the stack. The ~crank~ word is therefore
+equivalent to ~0 m metacrank~. Only one token can be evaluated per tokenized token, although /every/ metacrank is incremented
+per token, where lower metacranks get priority. This means that if you set two different metacranks, only /one/ of them
+can execute per token tokenized, and the lower metacrank gets priority. Note that metacrank and, by extension, crank,
+don't /just/ depend on tokenized words; they also work while evaluating word definitions recursively, meaning if a word
+is evaluated in ~2 crank~, one out of two words will execute in each level of the evaluation tree. You can play around
+with this in the repl to get a sense of how it works: run ~../crank bootstrap.cog repl.cog devel.cog load~, and use stem
+like syntax in order to define a function. Then, run that function in ~2 crank~. You will see how the evaluation tree
+respects cranking in the same way that the program file itself does.
+
+Metacrank allows for not only metaprogramming in the form of code building, but also
+direct syntax manipulation (i.e. /I want to execute this token once I have read n other token(s)/). The advantages to
+this system compared to other programming languages' systems are clear: you can program a prefix word and ~undef~ it
+when you want to rip out that part of syntax. You can write a prefix character that doesn't stop at an ending character
+but /always/ stops when you read a certain number of tokens. You can feed user input into a math program and feed the
+output into a syntax system like metacrank. The possibilities are endless! And with that, we will slowly build up the
+~stem~ programming language, v2, now with macros and from within our own /cognition/.
+* The Stem Dialect, Improved
+In this piece of code, we define the /comment/:
+#+begin_example
+2crank ing 0 crank ff 1
+2crank ing 1 crank cut unaliasf
+2crank ing 0 crank 0
+2crank ing 1 crank cut swap quote def
+2crank ing 0 crank
+2crank ing 0 crank #
+2crank ing 0 crank geti getd gets crankbase f d f i endl s
+2crank ing 1 crank compose compose compose compose compose compose compose compose compose
+2crank ing 0 crank drop halt crank s d i
+2crank ing 1 crank compose compose compose compose compose VMACRO cast quote compose
+2crank ing 0 crank halt 1 quote ing 1 quote ing metacrank
+2crank ing 1 crank compose compose compose compose VMACRO cast
+2crank ing 1 crank def
+2crank ing 2 crank # singlet # delim
+2crank ing 1 crank #comment: geti getd gets crankbase '' d '' i '\n' s ( drop halt crank s d i ) halt 1 1 metacrank
+#+end_example
+and it is our first piece of code that builds something /truly/ prefix. The comment character is a prefix that drops
+all the text before the newline character, which is a type of word that tells the parser to /read ahead/. This is our
+first indication that everything that we thought was possible within cognition truly /is/.
+
+But before that, we can look at the first couple of lines:
+#+begin_example
+2crank ing 0 crank ff 1
+2crank ing 1 crank cut unaliasf
+2crank ing 0 crank 0
+2crank ing 1 crank cut swap quote def
+2crank ing 0 crank
+#+end_example
+which simply unaliases ~f~ from the falias list, with ~ing~ being the only remaining falias. In cognition, even these
+faliases are changeable.
+
+Since we can't put ~f~ directly on the stack (if we try by just using ~f~, it would execute), we instead utilize some
+very minimal string processing to do it, putting ~ff~ on the stack and then cutting the string in half to get two copies
+of ~f~. We then want ~f~ to mean false, which in cognition is just an empty word. Therefore, we make an empty word by
+calling ~0 cut~ on this string, and then ~def~-ing f to the empty string. The following code is where the comment is
+defined:
+
+#+begin_example
+2crank ing 0 crank #
+2crank ing 0 crank geti getd gets crankbase f d f i endl s
+2crank ing 1 crank compose compose compose compose compose compose compose compose compose
+2crank ing 0 crank drop halt crank s d i
+2crank ing 1 crank compose compose compose compose compose VMACRO cast quote compose
+2crank ing 0 crank halt 1 quote ing 1 quote ing metacrank
+2crank ing 1 crank compose compose compose compose VMACRO cast
+2crank ing 1 crank def
+2crank ing 2 crank # singlet # delim
+2crank ing 1 crank #comment: geti getd gets crankbase '' d '' i '\n' s ( drop halt crank s d i ) halt 1 1 metacrank
+#+end_example
+Relevant: ~halt~ just puts you in 0 for all metacranks, and ~VMACRO cast~ just turns the top thing on the stack from a
+container to a macro. ~geti~, ~getd~, ~gets~ gets the ignores, delims, and singlets respectively as a string; ~drop~ is
+~dsc~ in stem. ~singlet~ and ~delim~ sets the singlets and delimiters. ~endl~ is defined withint ~bootstrap.cog~ and just
+puts the newline character as a word on the stack. ~crankbase~ gets the current crank.
+
+we call a lot of ~compose~ words in order to build this definition, and we make the ~#~ character a singlet delimiter in
+order to allow for spaces after the comment. We put ourselves in ~1 1 metacrank~ in the ~#~ definition while altering
+the tokenization rules beforehand in order to tokenize everything until a newline as a token while calling ~#~ on said word
+in order to effectively drop that comment and get ourselves back in the original crank and metacrank. Thus, the brilliant
+~#~ character is written, operating on a token that is tokenized /in the future/, with complete default postfix syntax.
+With the information above, one can work out the specifics of how it works; the point is that it /does/, and one can test
+that it does by going into the ~coglib~ folder and running ~../crank bootstrap.cog repl.cog devel.cog load~, which will load
+the REPL and load ~devel.cog~, which will in turn load ~comment.cog~.
+** The Great Escape
+Here we define a preliminary prefix escape character:
+#+begin_example
+2crank ing 2 crank comment.cog load
+2crank ing 0 crank
+2crank ing 1 crank # preliminary escape character \
+2crank ing 1 crank \
+2crank ing 0 crank halt 1 quote ing crank
+2crank ing 1 crank compose compose
+2crank ing 2 crank VMACRO cast quote eval
+2crank ing 0 crank halt 1 quote ing dup ing metacrank
+2crank ing 1 crank compose compose compose compose
+2crank ing 2 crank VMACRO cast
+2crank ing 1 crank def
+2crank ing 0 crank
+2crank ing 0 crank
+#+end_example
+This allows for escaping so that we can put something on the stack even if it is to be evaluated,
+but we want to redefine this character eventually to be compatible with stem-like quotes. We're
+even using our comment character in order to annotate this code by now! Here is the full quote definition (once we have
+this definition, we can use it to improve itself):
+#+begin_example
+2crank ing 0 crank [
+2crank ing 0 crank
+2crank ing 1 crank # init
+2crank ing 0 crank crankbase 1 quote ing metacrankbase dup 1 quote ing =
+2crank ing 1 crank compose compose compose compose compose
+2crank ing 0 crank
+2crank ing 1 crank # meta-crank-stuff0
+2crank ing 3 crank dup ] quote =
+2crank ing 1 crank compose compose
+2crank ing 16 crank drop swap drop swap 1 quote swap metacrank swap crank quote
+2crank ing 3 crank compose dup quote dip swap
+2crank ing 1 crank compose compose compose compose compose compose compose compose
+2crank ing 1 crank compose compose compose compose compose \ VMACRO cast quote compose
+2crank ing 3 crank compose dup quote dip swap
+2crank ing 1 crank compose compose compose \ VMACRO cast quote compose \ if compose
+2crank ing 1 crank \ VMACRO cast quote quote compose
+2crank ing 0 crank
+2crank ing 1 crank # meta-crank-stuff1
+2crank ing 3 crank dup ] quote =
+2crank ing 1 crank compose compose
+2crank ing 16 crank drop swap drop swap 1 quote swap metacrank swap crank
+2crank ing 1 crank compose compose compose compose compose compose compose compose \ VMACRO cast quote compose
+2crank ing 3 crank compose dup quote dip swap
+2crank ing 1 crank compose compose compose \ VMACRO cast quote compose \ if compose
+2crank ing 1 crank \ VMACRO cast quote quote compose
+2crank ing 0 crank
+2crank ing 1 crank # rest of the definition
+2crank ing 16 crank if dup stack swap 0 quote crank
+2crank ing 2 crank 1 quote 1 quote metacrank
+2crank ing 1 crank compose compose compose compose compose compose compose compose
+2crank ing 1 crank compose \ VMACRO cast
+2crank ing 0 crank
+2crank ing 1 crank def
+#+end_example
+Um, it's quite the spectacle how Matthew Hinton ever came up with this thing, but alas, it exists. Then, we use it in
+order to redefine itself, but better as the old quote definition can't do recursive quotes
+(we can do this because the definition is /used/ before you redefine the word due to postfix ~def~, a
+development pattern seen often in low level cognition):
+#+begin_example
+\ [
+
+[ crankbase ] [ 1 ] quote compose [ metacrankbase dup ] compose [ 1 ] quote compose [ = ] compose
+
+[ dup ] \ ] quote compose [ = ] compose
+[ drop swap drop swap ] [ 1 ] quote compose [ swap metacrank swap crank quote compose ] compose
+[ dup ] quote compose [ dip swap ] compose \ VMACRO cast quote compose
+[ dup dup dup ] \ [ quote compose [ = swap ] compose \ ( quote compose [ = or swap ] compose \ \ quote compose [ = or ] compose
+[ eval ] quote compose
+[ compose ] [ dup ] quote compose [ dip swap ] compose \ VMACRO cast quote compose [ if ] compose \ VMACRO cast
+quote compose [ if ] compose \ VMACRO cast quote quote
+
+[ dup ] \ ] quote compose [ = ] compose
+[ drop swap drop swap ] [ 1 ] quote compose [ swap metacrank swap crank ] compose \ VMACRO cast quote compose
+[ dup dup dup ] \ [ quote compose [ = swap ] compose \ ( quote compose [ = or swap ] compose \ \ quote compose [ = or ] compose
+[ eval ] quote compose
+[ compose ] [ dup ] quote compose [ dip swap ] compose \ VMACRO cast quote compose [ if ] compose \ VMACRO cast
+quote compose [ if ] compose \ VMACRO cast quote quote
+
+compose compose [ if dup stack swap ] compose [ 0 ] quote compose [ crank ] compose
+[ 1 ] quote dup compose compose [ metacrank ] compose \ VMACRO cast
+
+def
+#+end_example
+Okay, so now we can use recursive quoting, just like in stem. But there are still a couple things missing that we probably
+want: a good string quote implementation, and probably escape characters that work in the brackets. Also, since Cognition
+utilizes macros, we probably want a way to notate those as well, and we probably want a way to expand macros. We can do
+all of that! First, we will have to redefine ~\~ once more:
+#+begin_example
+\ \
+[ [ 1 ] metacrankbase [ 1 ] = ]
+[ halt [ 1 ] [ 1 ] metacrank quote compose [ dup ] dip swap ]
+\ VMACRO cast quote quote compose
+[ halt [ 1 ] crank ] VMACRO cast quote quote compose
+[ if halt [ 1 ] [ 1 ] metacrank ] compose \ VMACRO cast
+def
+#+end_example
+This piece of code defines the bracket but for macros (split just splits a list into two):
+#+begin_example
+\ (
+\ [ unglue
+[ 11 ] split swap [ 10 ] split drop [ macro ] compose
+[ 18 ] split quote [ prepose ] compose dip
+[ 17 ] split eval eval
+[ 1 ] del [ \ ) ] [ 1 ] put
+quote quote quote [ prepose ] compose dip
+[ 16 ] split eval eval
+[ 1 ] del [ \ ) ] [ 1 ] put
+quote quote quote [ prepose ] compose dip
+prepose
+def
+#+end_example
+We want these macros to automatically expand because it's more efficient to bind already expanded macros to words,
+and they functionally evaluate identically (~isdef~ just returns a boolean where true is a non-empty string, false
+is an empty string, if a word is defined):
+#+begin_example
+\ (
+( crankbase [ 1 ] metacrankbase dup [ 1 ] =
+  [ ( dup \ ) =
+      ( drop swap drop swap [ 1 ] swap metacrank swap crank quote compose ( dup ) dip swap )
+      ( dup dup dup \ [ = swap \ ( = or swap \ \ = or
+        ( eval )
+        ( dup isdef ( unglue ) [ ] if compose ( dup ) dip swap )
+        if )
+      if ) ]
+  [ ( dup \ ) =
+      ( drop swap drop swap [ 1 ] swap metacrank swap crank )
+      ( dup dup dup \ [ = swap \ ( = or swap \ \ = or
+        ( eval )
+        ( dup isdef ( unglue ) [ ] if compose ( dup ) dip swap )
+        if )
+      if ) ]
+  if dup macro swap
+  [ 0 ] crank [ 1 ] [ 1 ] metacrank ) def
+#+end_example
+and you can see that as we define more things, our language is beginning to look more or less like it has syntax!
+In this ~quote.cog~ file which we have been looking at, there are more things, but the bulk of it is pretty much done.
+From here on, I will just explain the syntax programmed by quote.cog instead of showing the specific code.
+
+As an example, here is ~expand~:
+#+begin_example
+# define basic expand (works on nonempty macros only)
+[ expand ]
+( macro swap
+  ( [ 1 ] split
+    ( isword ( dup isdef ( unglue ) ( ) if ) ( ) if compose ) dip
+    size [ 0 ] > ( ( ( dup ) dip swap ) dip swap eval ) ( ) if )
+  dup ( swap ( swap ) dip ) dip eval drop swap drop ) def
+
+# complete expand (checks for definitions within child first without copying hashtables)
+[ expand ]
+( size [ 0 ] > ( type [ VSTACK ] = ) ( return ) if ?
+  ( macro swap
+    macro
+    ( ( ( size dup [ 0 ] > ) dip swap ) dip swap
+      ( ( ( 1 - dup ( vat ) dip swap ( del ) dip ) dip compose ) dip dup eval )
+      ( drop swap drop )
+      if ) dup eval
+    ( ( [ 1 ] split
+        ( isword
+          ( compose cd dup isdef
+            ( unglue pop )
+              ( pop dup isdef ( unglue ) ( ) if )
+            if ) ( ) if
+          ( swap ) dip compose swap ) dip
+        size [ 0 ] > ) dip swap
+      ( dup eval ) ( drop drop swap compose ) if ) dup eval )
+  ( expand )
+  if ) def
+#+end_example
+Which recursively expands word definitions inside a quote or macro, using the word ~unglue~. We've used the ~expand~
+word in order to redefine itself in a more general case.
+* The Brainfuck Dialect
+And returning to whence we came, we define the /Brainfuck/ dialect with our current advanced stem dialect:
+#+begin_example
+comment.cog load
+quote.cog load
+
+[ ] [ ] [ 0 ]
+
+[ > ] [[ swap [[ compose ]] dip size [ 0 ] = [ [ 0 ] ] [[ [ 1 ] split swap ]] if ]] def
+[ < ] [[ prepose [[ size dup [ 0 ] = [ ] [[ [ 1 ] - split ]] if ]] dip swap ]] def
+[ + ] [[ [ 1 ] + ]] def
+[ - ] [[ [ 1 ] - ]] def
+[ . ] [[ dup char print ]] def
+[ , ] [[ drop read byte ]] def
+
+[ pick ] ( ( ( dup ) dip swap ) dip swap ) def
+[ exec ] ( ( [ 1 ] * dup ) dip swap [ 0 ] = ( drop ) ( dup ( evalstr ) dip \ exec ) if ) def
+
+\ [ (
+  ( dup [ \ ] ] =
+    ( drop swap - [ 1 ] * dup [ 0 ] =
+      ( drop swap drop halt [ 1 ] crank exec )
+      ( swap [ \ ] ] concat pick )
+      if )
+    ( dup [ \ [ ] =
+      ( concat swap + swap pick )
+      ( concat pick )
+      if )
+    if )
+  dup [ 1 ] swap f swap halt [ 1 ] [ 1 ] metacrank
+) def
+
+><+-,.[] dup ( i s itgl f d ) eval
+#+end_example
+test with ~../crank -s 2 bootstrap.cog helloworld.bf brainfuck.cog~. You may of course load your favorite brainfuck
+file with this method. Note that brainfuck.cog isn't a brainfuck parser in the ordinary sense; it actually
+/defines brainfuck words/ and /tokenizes/ brainfuck, running it in the native cognition environment.
+
+It's very profound, as well, how our current syntax allows us to define an /alternate/ syntax with great ease. It might
+make you wonder if it's possible to /specifically craft/ a syntax whose job is to write other syntaxes. Another interesting
+observation you might have is that Cognition defines syntax by defining a prefix character as a /word/ that uses metacrank,
+rather than reading symbols and deciding what to do based on symbols. It's almost as if the syntax becomes /inherent/ to the
+word that's being defined.
+
+These two ideas synthesize to create something truly exciting, but that hasn't yet been implemented in the standard library
+(though we very much know that it is possible). Introducing: the /dialect dialect/ of Cognition...
+** The Dialect Dialect
+Imagine a word ~mkprefix~, that takes two input words (say for example ~[~ and ~]~), and an operation, and
+/automatically defines/ ~[~ to apply said operation until it hits a ~]~ character. This is possible because constructs
+like ~metacrank~ and ~def~ are all just /regular words/, so it's possible to use /them/ as words to metaprogram with.
+In fact, /everything/ is just a word (even ~d~, ~i~, and ~s~), so you can imagine a hyperabstract dialect that includes
+words like ~mkprefix~, using syntax to automate the process of implementing more syntax. Such a construct I have not
+encountered in /any other programming language/. Yet, in your own /Cognition/, you can make nearly anything a reality.
+
+Such creative things Matthew Hinton and I have discussed as possibilities regarding the standard library. Right now, the
+standard library has metawords that generate abstract words automatically and call them. This is possible through string
+concatenation and using ~def~ in the definition of another word also (this is also possible in my prior programming
+language Stem). We have discussed the possibility of a word that searches for word-generators to abstract its current
+wordlist automatically, and we have talked about the possibility of directing this abstraction framework for the purpose
+of solving a problem. These are conceptually possible words to write within cognition, and this might give you an idea
+of how /powerful/ this idea is.
+* Theoretical Musings
+There are a couple of things about Cognition that make it interesting beyond its quirks. For instance,
+string processing in this language is equivalent to tokenizer postprocessing, which makes string operations inherently
+extremely powerful in this language. It also has potential applications in Symbolic AI and in syntax and grammar research,
+where prototypes of languages and metalanguages can be tested with ease. I'd imagine that anyone configuring a program
+that reads a configuration file would really want their configuration language to be something like this, where they can
+have full freedom over the syntax (and metasyntax) in which they program in (think about a Cognition based shell,
+or a Cognition based operating system!). Though, the point of working on this language was never its applications;
+its intrinsic beauty is its own philosophical statement.
+* Conclusion
+You can imagine cognition can program basically any syntax you would want, and in this article, we demonstrate the power
+of the already existing code that makes cognition work. In short, the system allows for true /syntax as code/, as my
+friend Andrei put it; one can /dynamically program/ and even /automate/ the production of syntax. In this article, we
+didn't have the space to cover other important Cognition concepts like the /Metastack/ and words like ~cd~, but this
+can be done in a part 2 of this blog post.