Bigsley the Oaf

Skeleton of Argument

Posted in Uncategorized by bigsleytheoaf on June 25, 2011

Here is a skeleton of the argument I’m trying to put forward in my previous post:

– Given a system S, and a model M of that system, I can say whether M is “good” or “bad,” in a systematic way.

– My system of judgement relies on an analysis of the ontology (system of types of elements) underlying M.

— One way that a system M can be better than a system M’ is if the difference between its cardinality and the cardinality of the “true ontology” is smaller than that of M’ and the true ontology. In other words: abs(C(M) – C(S)) < abs(C(M’) – C(S))

— Physics has evolved along a trajectory wherein it has become “better” with each “revolution.” It serves as an example of a set of ontologies which are better/worse than one another. Each “revolution” in physics corresponds to an action on the ontology of the mainstream conception of physics.

— Models can be “more macroscopic.” A model M is better than a model M’ if the tradeoffs associated with its level of macroscopicity (word?) are good. E.g. if precision is not decreased too much, but computational efficiency is increased greatly.

— Chemistry > Physics in some cases. Although chemistry is less precise than a faithful computation of all the physical properties of a system, such a computation is intractable, so chemistry wins sometimes.

– Therefore, we can say that neural nets are not good. First of all, they have too small of an ontology. Secondly, they aren’t macroscopic enough to be computationally useful for some tasks.

Against Neural Nets as a Computational Model of Intelligence

Posted in Uncategorized by bigsleytheoaf on June 25, 2011

I just sent this email to someone and decided that it pretty nicely summed up my ideas on the subject of neural nets (with lots of ideas which apply equally well to other modern computational models of intelligence – e.g. SVM’s, bayesian nets, etc.)

I think what we ended up converging on, topically, was a comparison between the evolution of our (society’s) ideas about mind to the evolution of our ideas about other things (e.g. physical reality).

The reason for this was that I was asserting that, by various definitions of “bad” which I was unable to effectively elucidate, neural nets are a “bad” model of the human brain/the human mind. I think that you were trying to justify the progress we’ve made (which has culminated in neural nets?) by referring to the overall progress of science.

Let’s look at the progress of physics which goes (very loosely): folk physics -> “elemental physics” (earth, wind, fire, etc.) -> newtonian mechnics -> relativity -> quantum physics -> string theory.

In my way of interpreting science, what has really happened here is a long process of growth and refinement of a finite ontology. We start out with a very coarse ontology:

Folk physics: (stuff)

It grows:

Elemental physics: (earth, wind, fire, water)

->

Newtonian mechanics: (matter (earth + water + wind), mass, force, momentum, energy (fire…kind of), E&M)

->

Pre-relativity: (aether + newtonian mechanics)

->

Relativity: (energy + mass are combined, so are momentum + energy)

->

Quantum physics: (sub-atomic forces, particles + waves are combined, birth of particle physics (sub-ontology of sub-atomic particles))

->

String theory: (well… I actually don’t know that much about string theory)

Anyway, the history of physics has been this really complicated and boring struggle to find just the right number and combination of concepts to compactly describe what we see around us. The two operations – adding an element to the ontology, and combining two elements of the ontology – were used over and over in a complicated way until we arrived at our modern understanding.

I’m not asserting that this is the correct way to understand physics – I’m just saying that it’s my way of interpreting what I know of the history of physics. Of course it’s much more complicated than this very linear picture, but this is more or less the backbone (in the sense that it’s what is taught as the history of physics, and each of these theories is still easily accessible, informationally – unlike theories surrounding flogistam, the aether, etc.)

Now what’s key here is that I would assert that each successive ontology is “better” than the one before it. How is it better? I’ll try to describe the intuition:

The addition of an element to an ontology allows it to “breathe.” Once we add “sub-atomic particle” as something we can think about then we have this interesting combinatorial calculus which justifies the behaviors of objects which heretofore were seen as “atomic” (in the sense of being indistinguishable and composed of exactly one object).

The combination of two elements in an ontology makes its representation more compact – and easier to calculate with. Why think of energy and mass as ultimately being separate things, when they can be thought of as the same thing? Sure, in some situations you can, for convenience’s sake, treat mass purely as mass (in the newtonian sense) and energy purely as energy (in the maxwellian sense), but there are situations (e.g. when things are going very fast, or when things are blowing up) in which it doesn’t really make sense to think of these as different things.

Conversely, the unnecessary addition of an element or combination of two elements is quite burdensome. Imagine if, for instance, someone claimed that energy IS gravity – it would be so hard to wrap our minds around this. Or if someone claimed that there was a fifth force, and tried to justify it.

To sum up: do not try to solve an N-dimensional problem using an N-1 or N+1 -element ontology.

—–

Bringing this back to terra firma:

One issue with neural nets is that I believe that the mind is an N-dimensional object and that neural nets present (in my opinion) an N-m (for some m > 0) -element ontology in attempting to model it. Why?

Well, I’m assuming that the theory of neural nets is really just a 1-element ontology (where the element is the “neuron” or node). Correct me if I’m wrong (or interpreting this strangely).

For one thing, on the cellular level, there are other computational structures in the human brain – namely glial cells. Astrocytes (a common type of glial cell) form a syncytium (effectively a large a multi-nuclei cell), release neuro-transmitters, are electrically active, and are about as plentiful in the brain than neurons (there are about 10^11 of each). Here is a pertinent quote from wikipedia:

“For over a century, it was believed that they did not play any role in neurotransmission. That idea is now discredited; they do modulate neurotransmission, although the mechanisms are not yet well understood.”

This suggests to me that we might want to consider a model with (at least one) other element -> it seems to me that, to represent the cellular structure of the brain, we might need at least 2 (if not more) elements in our ontology.

Another issue with neural nets is that they are trying to model the mind in terms of cellular structures. I don’t believe in spirits or god or magic, but this approach seems to me like trying to figure out chemistry in terms of purely quantum-theoretic or string-theoretic mechanisms.

Namely, macroscopic models exist and are useful.

The ontologies of chemistry have evolved along different (though not entirely disconnected) lines from those of physics. E.g. they started with atom -> plum pudding model -> bohr atom & elements -> quantum atom -> etc. etc.

Similarly, I think that what we have witnessed (mostly in the fields of philosophy, psychiatry, psychoanalysis, cultural anthropology, to the extent that they have been performed responsibly and scientifically) historically has been the evolution of an ontology of mind. It is composed of words like “memory,” “computation,” “thought,” “idea,” etc. If the mind has the brain as its sole substrate then these models should be equivalent in the sense the chemistry and physics are somehow equivalent since they’re both trying to describe the same physical reality.

Let’s say that a model can be “more macroscopic” than another. Then chemistry is a “more macroscopic” description of physical reality than physics. They’re both ‘right’ but, being more macroscopic, Chemistry has higher expectations placed on it for computational tractability, and lower expectations for precision.

This suggests to me that we might want to consider a model that is “more macroscopic” than neural nets. They might be computationally intractable. Which brings me to my next and final point:

The elephant in the room is computational tractability. You said something during the party about “if the neural net is slow then we can just model the environment slowly.”

I think this is ignoring the fact that human intelligence is largely (if not entirely) written in terms of the structure of society. A human which grows up out of contact with society will never be “intelligent” in the same way that you or I are intelligent.

In fact, we do not have a single example anywhere in the universe of anything which we consider “intelligent” which has not had an absurd amount of personal contact with humans.

My belief is that human society is very very good at turning people into “intelligent” creatures, meaning creatures who can do art, who can learn math, who can play go, who can think about the mind…

The structures of society have evolved to allow us to reliably educate a sufficiently “normal” human to levels of intelligence heretofore unseen.

If we can’t connect to society a “creature” whose computational structure is a neural net, then I don’t think we’re going to be able to teach it, and I don’t think that it will be “intelligent” in the same way that you and I are intelligent.

Fractal(ly) Nonsens(e/ical)

Posted in Uncategorized by bigsleytheoaf on June 10, 2011

Modern life makes no sense, fractally. On no level does it make sense. We sit all day – that’s weird. We talk about ethics – that’s weird. We talk about the ethics of sitting all day. We watch programs about people who are only occasionally seen in real life. We talk about the ethics of those programs. We talk about the ethics of the content of those programs and the ethics of whether or not the characters in those programs are sitting.

There are parallel social realities evolving in contradiction of one another. You hate her when you’re with her – you love her when you’re with friends. You feel bad about hating her when you’re with friends. You feel good about loving her when you’re with her (although you don’t).

It makes no sense! We sit in boxes. We visit the forests. We can’t help but visit them. Even if we live in the forests, we don’t really. We’re living in the bubble of our protective culture suit.

There’s no returning to nature. We’re plowing into nature like a jackhammer which creates self-referential cracks. Our culture is the template for our culture. Our culture is the tool we use to carve out reality. Thus the manner in which we carve out reality is defined by the way in which we carve it out. This is to say that our nature fundamentally an internally-consistent but randomly wandering one.

The very essence of culture is this – we don’t know where we’re going, or what we’re doing, but we’re going to do it in style! But we don’t know what style is – how about the way we’re doing things!?!

Surveillance and Simplified Law

Posted in Uncategorized by bigsleytheoaf on June 9, 2011

I was thinking, recently, that I wouldn’t mind massive surveillance in public places given the following two conditions:

1. Massive simplification of law and distribution of materials giving explicit conditions for breaking a law

I simply don’t know what’s illegal. One reason that the expansion of the US’s surveillance powers is so frightening is that I don’t know if/when I am breaking a law. If I have a very good idea when I am breaking a law then I will be afraid regardless of the existence of surveillance powers.

An example is trespassing. E.g. I don’t know when it’s illegal to be on government property. Jen and I were walking around an old naval base at night the other day and I was a little afraid we’d get picked up by the cops.

2. Legalization of victimless “crimes”

This includes drug use, nudity, public drunkenness, “loitering,” prostitution etc. There are many arguments for these, but I think the surveillance one is pretty powerful.

This actually goes hand in hand with #1 above – if damage to another person is the primary standard for whether or not something is a crime then it’s pretty easy for me to check whether I’m breaking a law.

The upswing of surveillance is that it makes the resolution of interpersonal disputes easier. E.g. surveillance footage of a car accident is useful in figuring out if someone is to blame. If you get mugged or robbed then there will be more information available to assist cops in finding the perp and/or your stuff.

The downswing of surveillance is that it’s damned scary. No one likes the feeling that someone is watching them – especially if that someone has the power to throw them in jail or kill them.

The fact of the matter is that people are becoming increasingly alienated from and terrified of government and the law. I think that they would be less scared if there were some reform in the way law is constituted and communicated.

Personal Knowledge and Interpretation of Statistical Data

Posted in Uncategorized by bigsleytheoaf on June 2, 2011

This post is a response to some of the sentiments expressed in response to these articles:

http://news.ycombinator.com/item?id=2607751

http://news.ycombinator.com/item?id=1978295

To summarize, a lot of people are highly critical of the decisions of individuals to take a 1/10 chance of death in summiting Mt. Everest.

This type of criticism causes me deep anguish. On the one hand, given that the people know nothing about the individuals they are criticizing, they are technically correct. On the other hand, a refutation from an individual of the form “Fuck you! I’m gonna climb that shit! I’m strong as hell!” is very difficult to effectively contradict:

One deep problem when applying a statistic of this type to an individual is that a statistic is a statement about the properties of an individual given no information other than intent. Straightforward statistical analysis is egalitarian – it relies on the assumption that people are fundamentally similar. What gets complicated is that, from the perspective of an outsider who has no information, they are, but from the perspective of the individual who has lots of information, they aren’t necessarily.

Suppose you’re confronting a person who’s going to climb Mt. Everest –

Naysayer: “But you have kids! You have a 1/10 chance of dying!”

Climber: “No, actually, the average person who attempts to summit Everest has a 1/10 chance of dying – but I’ve been climbing for 20 years and have done much harder climbs…so my probability is lower”

The Naysayer is now in a very complicated position. He has no knowledge about how the conditional probability of death in attempting to summit Everest varies with experience – and what’s more, we expect that not everyone who climbs Everest has 20 years of experience, that experience is probably useful, and so probably this person has a higher chance of living – but how much higher?

In formal terms, the problem is that all we know is P(Death), we don’t know P(Death | X) for ANY X. So how do we respond when someone says “oh, but X!” (for any X!)

E.g. “oh, but I’m SUPER strong – and I climb 5.13’s in the gym!”

E.g. “oh, but I’ve summited Kilimanjaro!”

E.g. “oh, but I have very good circulation!”

To refute these claims would require a lot of effort on your part – you’d have to do a thorough analysis of the influence of these conditions on probability of success – as well as a quantitative analysis of the subject.

More purely, as soon as you know anything about the person you are forced to evaluate the conditional probability given what you know – but you generally don’t know how to do that.

The real crux of the issue is that you might have beliefs about the action outside of a purely statistics-based expected value calculation. E.g. maybe you “really believe” that the guy can climb the mountain! How do you compare a conditional probability that you can’t evaluate with a qualitative claim? God only knows!

—–

This is to say that statistical analysis is basically useless with respect to its application to a known individual.

It is useful when it comes to decisions regarding a population (e.g. policy decisions) as long as you don’t have any information about how properties of the population are distributed.

It is useful if you think you’re just like anyone else, or if you know nothing about the individual.

—–

This is to stay – stop crossing the bounds, motherfucker!

Posted in Uncategorized by bigsleytheoaf on June 2, 2011

Words that are nice:

Clean, bright, touch, face, mother, mud, prowl, taste, orange.

King, fang, bull, fist, stretch, cacophony, wig.

Harsh, fricative, ebullient, quick, lucidity.

Haggard, smoked, cracked.

Words that are just awful:

Maximize, plethora, jaundice, fantastic, pug, stench, pineapple.

Span, magnet, liquidity, bless, government, beauty.

On Understanding

Posted in Uncategorized by bigsleytheoaf on June 1, 2011

It’s very easy to forget that, speaking colloquially, understanding an assertion does not necessarily correspond to a faithful representation of its underlying beliefs in one’s own brain.

The model I have in mind is the following:

Mind1 -> Formulation -> Statement (transmission) -> Interpretation -> Mind2

If one person’s interpretation scheme is the inverse of the other’s system of formulation, then there has been a connection of minds – that is to say that the idea/thought which appeared in Mind1 can be said to have appeared in Mind2.

For those of us who try to think rigorously, the fact that most people are terrible at BOTH formulation and interpretation makes interaction with other people very difficult. What this means is that only very simple/repetitive information can be transmitted to/gleaned from such a person.

As a man who holds his own abilities in these domains in high regard, to meet a person great at Formulation and Interpretation is a divine joy. To be able to hear what they’re actually thinking and to have them recognize the profundity of your thoughts rather than quibble with details is pure heaven. It’s the fusion of souls. It is, perhaps, equivalent to divinity.