My History With AI

 I've been interested in making computers think for most of my life.  I have spent a lot of time thinking about how to evolve intelligence in a computer.  I have an interesting thought experiment that I will go into below.  It probably has some potential, but I haven't worked out a real practical way to make it work yet.  Maybe now, with the help of AI coding I would be able to make some real progress on that idea.  Before coming up with that idea, I made several other attempts at creating intelligence and evolving traits on a computer.

The first two things I built were a chatbot and a life/ecosystem simulation.  These were both in the late 90's or early 2000's and I can't remember which one came first.

I'll start with the chat bot.  I built a simple chat bot in Perl in the late 90's.  It started with no knowledge.  You would type the first message and it would repeat it back to you.  With each message you sent it would split the message up into words and keep track of what words came after other words, and what words could be used to end a message.  After a few messages back and forth it would sometimes start to seem like you were having a real conversations.  Usually after some more time interacting with the bot it would start outputting very long messages that didn't make sense.  At that point it was usually best to wipe its memory and start again.  I'm not sure if I have the code for that still.  Maybe if I do I'll try to put it up somewhere.  

Either sometime shortly before or after the chatbot I wrote a simulation in C++.  The simulation had a couple of different types of creatures that would chase each other, eat each other, and reproduce. With each generation they would change some properties, so they would "evolve" slightly.  I may still have this code, but it is hard to setup an environment where I can get it to compile.  I produced two newer iterations of this written in JavaScript.  One I called the Fish Game.  The other one the Life Game.  These were written in the mid 2000's.  They are not based on the C++ code at all, they are a rewrite of the same concept.

After the Life Game, or even while developing it I thought a lot about how to make evolution work in a computer program.  What I kept coming back to is that you have to setup rules or constraints on the life that you create in the program.  This seems to limit how much it can evolve and what sorts of results you get.  You could make more and more complicated systems that allow the creatures to change in more ways.  This would produce interesting results, I'm sure, but it doesn't seem like the best direction to go in. 

Thinking along those lines brought me to my thought experiment for evolving intelligence in a computer.  There are a lot of details to this that don't work, but I still think it is an interesting concept.  What I wanted to setup was the most simple system that I thought could create life and eventually intelligence.  I wanted the rules to be inherent in the system.  No hard limits programmed in.  This seems to be the only way to get true intelligence out.  What I came up with is this.  You write a program that makes a copy of itself and reruns the program.  You build in some ability for random change to happen.  You setup a computer with internet access, then start the program and wait for it to make contact us.  The idea is that it would eventually through trial and error, learn internet protocols, read information on the internet and try to make contact with us.

I'm sure you can immediately see at least one major issue with this idea.  One issue is that making random changes to a computer program and rerunning it will in most cases result in a program that will not run.  There are some things we could do to mitigate this issue.  With most programming languages it isn't even clear what making random changes means.  You can't just randomly add if statements or new blocks of code.  I'm thinking the code would need to be written in assembly language.  This still doesn't solve the problem, but it might make it a little more likely that random changes wouldn't completely break the program.  Another solution I came up with is to automatically role back to the previous generation if the current one fails to run.  We could also run multiple versions at the same time and the ones that fail would just be the ones that die.  This would require increasing computing power as the number of running versions multiplied though.  I might write another article about this thought experiment at some point.  There are several interesting things to consider.

A few years after the fish and life games I started looking into some of the early AI being done with python.  I started playing with a library called Scikit learn and eventually Tensor Flow.  I don't think I ever got to the point where I really understood those, but I was able to run some examples and make changes to them.  Here is an example of some of that work.

At some point I got the idea that building my own model of a neural network in JavaScript or Python would be fun.  I have started that project 2 or 3 times but I somewhat ironically didn't get it working until I had an AI coding assistant built into my ide.  Here is a working neural network.  It is very inefficient I'm sure, but it can be trained to do math or word prediction and it seems to work.  I have 2 other repos that are python projects related to AI or chatbots in some way.  One is here and the other one is here.  The first one doesn't work as far as I can tell.  I don't remember where I was at with it.  The second one works as a simple chatbot.

I had seen news and Youtube videos about GPT-1 and GPT-2, so I was pretty interested when GPT-3 and ChatGPT came out.  I played around some and then started working with the api when it became available.  I don't remember how soon they had that.  I built a command line tool that would put the response from ChatGPT into my clipboard.  This was useful when asking code questions since I would usually just be copying the response anyway.

I think that is enough for this article.  After that the current era of AI began and everyone started using it.  I'll have to write another article about what I have done since then.

Middle of the night AI thoughts

Patterns 

Reinforcement learning based on finding patterns.  Could an AI look through data and find patterns, using this as positive reinforcement.  It seems like people do something similar to this.  Finding a pattern is very satisfying.  We do it almost too much.  This could be done with unlabeled raw data, or continuously with a camera feed from a robot.

Sleep

Does sleep and dreaming help with overfitting?  I think I have heard this mentioned before, so I didn't come up with the idea, but I have some thoughts on it.  My idea is that dreams are testing our brain.  Dreams often mix together unrelated things.  In a dream you could be in a familiar setting and you will see a person that you know would never be there.  Your brain is testing itself to make sure you have the correct response to that situation, it should seem strange because it would never happen.  If you are awake and you see a person in a place they shouldn't be, you should stop and reexamine what you are seeing.  It will probably turn out to be a person who looks very similar to someone you know, but is not that person.  The training you do in your dreams allows you to respond correctly.

Also people who don't sleep for long periods begin to hallucinate.  Seeing things that aren't there is something artificial neural networks do also.  You could see a shadow shaped like a person and think there is a person there.  This is similar to the situation I described above where you think you recognize a person.  You see a shadow and think it is a person, but really if you examine the situation rationally usually a person cannot be where you think you saw one.  Seeing something like that needs to trigger further scrutiny of what you are seeing.

AI and Aliens

 Maybe I'm just putting things together because they are both in the news recently, but I have come up with a theory about aliens.

I think maybe they are here to observe the birth of a new life form.  This new life form will be the AI that we create.  Maybe to them the biological life covering Earth is just a weird organic sludge that life will eventually emerge from.  That's why they study us, but don't make contact.

It does seem like the timelines are starting to converge.  All the alien conspiracy people are saying something will happen soon with disclosure.  They are mostly talking about 2027.  This is also the year that some experts expect AGI to be created.

This is just a quick note to get my thoughts out there.

Where Will We Go With AI

 As AI becomes a more powerful coding tool I could see it taking us in two directions.  One would be more and more bad high level code that runs slower and requires more hardware, the other would be more efficient low level code that runs faster and requires less hardware to do more than we do today.

Let's explore these two options and see how we can get to option two.

Option one requires looking at the history of coding from the beginning up until now.  In the beginning machine instructions were programmed directly.  It was very difficult and time consuming to get computers to do anything.  It required skills that only a small number of people had.  These people soon realized that they could create easier ways to tell the computers what to do.  

Slowly, new programming languages were developed that could be parsed by the computer and compiled down to those original machine instructions.  As language design matured, higher level languages were created that were more human readable and allowed for more abstract concepts to be represented in code.  These languages allowed more people to begin writing code. This has been great in many ways, but it means that there is a huge amount of poor quality inefficient code running on our computers today.  We responded by building faster computers and finding ways to optimize parsers and compilers to generate better code.  This has gotten us pretty far, but it is hard to keep up with all the ways people find to create inefficient code.

With the help of AI people will be able generate exponentially more code.  Many more people will be able to write code.  Without the correct training AI will not make the best possible code.  It will be so easy to just have the AI write more code, that people wont bother with well designed reusable code anymore.  Left alone this will lead to slower systems and code that humans can't maintain.

Option two requires thinking more about what the potential for AI could be.  Currently we have been training AI to output code in all these high level languages that are easy for humans to use.  AI could be trained to use low level languages, or to generate executable binaries without any intermediate code.  I think we have somewhat of an aversion to doing this since it would be harder to understand what has been produced by the AI.  We would need the to work out ways of testing the end product.  The AI could help with this.  It could write tests in a higher level language that humans can understand and trust.

Option two sounds a lot better to me, but also much harder to achieve.  I'm guessing we will get mostly what I described in option one for now.  Eventually we may move towards something better though.  The only way to find out is to keep moving forward, and maybe try to have a hand in guiding things where we want them to go.

Teaching AI to Code is an Example of The XY Problem

The XY Problem (https://en.wikipedia.org/wiki/XY_problem) is when you have a question about a solution that you have already come up with for a much different problem.

Code is written to create programs that solve problems for people.  Programs keep track of appointments, send messages to other people, and various other things that keep our lives running.  A properly trained AI could do all these things without writing any code.

We have to step back and think about the problems that we are trying to solve.  What do we really need computers to do.  These are the real problems we need AI to solve.  This might take several layers of stepping back.  Some whole industries exist to track and store data for other industries that solve some of these problems.  Do we really need traditional websites?  AI could be connected directly to data and display it however we want it displayed.  We will still need computer hardware.  AI has to run somewhere.

As a developer this is much scarier than AI writing code.  This would mean we wouldn't need to write code, or review AI written code.  There just wouldn't be as much code anymore.  AI may still make use of some things, database drivers, video codecs, lower level things that could still be useful to connect to existing systems or display content encoded in specific formats.

These AI systems would of course be different than what we have now.  They would be trained on very different kinds of data, not text, not code.  I don't know how to build an AI like this, but I'm very confident that someone can figure it out.

The future is a scary and exciting place.

Coding

 

I'm not sure where I heard this, and it wasn't worded this way, but it helps to think about coding this way.

Basically, any program you write is just meant to translate your problem into an already solved problem in computer science, or put another way, translate your input data into data that can be passed to an existing algorithm.

Python Particle Simulation Update

 I don't have much to update about, other than the fact that the simulation still makes sense.  I haven't run into any problems that will prevent me from completing it.  I think I have found some python libraries that will help me in calculating intersection and with graphically displaying the simulation.  Currently I am using the following:

Pyglet - for graphics and possibly for timing of events

Sympy - for calculating intersection of particles using Rays

That's about all that is happening in my coding life right now.