Computers can be programed to sift unstructured texts in order to find connections. If you want to connect two realms of thought, you need data from both realms. You should also have a theory on how to connect the realms. With a fast computer, and the right software, there is a chance to find discoveries between realms of thought. I worked in the stock market trying to find relationships between one stock going up, and another following (tracking stocks). If I could show that Harley Davidson stock prices went up and then Wal Mart stocks followed, I could help investors make money.
Using similar techniques, I seek to find connections in literature, history, culture,… you name it, a link is valuable. With all the data available on the internet (electronic books especially), we are going to see a new era of data mining and connectionism. The book to the left is an invaluable introduction to the topic. I found it while doing research at Reuters on how to connect News Stories to stock movements.
I have been especially interested in applying these ideas to the ancient texts of Egypt, Israel, Assyria, Rome, etc. For example, in the Egyptian Amarna letters, unique phrases appear that also appear in Malachi and Isaiah. There is a connection (at least linguistically) between documents across languages. Finding such connections, and then pointing them out to a researcher, is the work of software.
That is the software I am proposing. The requirements are these: 1) large quantities of ancient texts that may relate at some abstract level; 2) fast computers and 3) advanced algorithms that find connections. The first two are easy to get and the third one is the trick. I will elaborate on all three and suggest a way forward:
1) Large quantities of ancient books stored electronically are readily available from Logos
I have been using Logos for years, and my collection of ancient texts has been growing (I have to purchase each book I unlock). In addition to the Logos collection, I have 1400 Ugaritic tablets along with software that I wrote to display and use them.
2) Fast computer chips and memory are ubiquitous
With the advent of personal computers, and now cloud computing, the ability to process large amounts of data is as close as the least expensive laptop. Computing power is at a point where this kind of project is feasible for the hobbyist. A modern laptop rivals the computing power of earlier super computers and older mainframes.
3) Logos has provided a programming interface to access their LARGE collection of books.
As a software programmer, I need an interface that allows me to access the above mentioned books. They are in a proprietary data format — one not easily accessible. The good news which prompts the writing of this article is this: The folks who provide the important e-books also wrote an interface for programmers. Their interface is freely available and documented on their web site. This means Windows software can be written to sieve, process, connect and collate realms of thought.
This post is the first installment of a series where I will chronicle the use of Logos’s programmer’s interface. I am talking about the start of an ambitious software project — software that gets into Logos’ massive collection of books. As with all of my software experiments, look for scaffolding-like tools to start appearing on my web site. I have written extensively on this subject of connectionism in the past. Specification documents have already been generated along with basic design.
The steps before me are these:
1) Learn the application interface for the Logos system. This will mean sample software; help exists.
2) Implement connectionism algorithms as per the above mentioned book. Some of this has been written. Earlier excursions have yielded wildly different applications. One software tool connects the Hebrew Old Testament to the Greek Old Testament and finally to the Greek New Testament while another operates on the periodic table of elements.
Limitation: Time. I don’t have any free time to dedicate to this project. It will be slow going. Other software projects take priority (even as they are related).