personal essayin'

Time to write essays for the NSF graduate research fellowship program. Actually I had planned to have them drafted by now, but oh well.

So what should I write about? Really, given the goals of the program, it's obvious and I shouldn't bother making this post, but it's probably positive to reflect on the last three years a bit more than that (and in something besides bullet point). So the obvious thing is to talk about the research project I've been working on and which got me a publication as first author in the proceedings of a well-known (in the field) international refereed conference before I was old enough to drink. At least, going by when the work that got presented was done. I've done stuff on the project since then.

I'll go on in the essay to talk about how this project has also introduced me to the central tension I want to exploit. In short, it uses statistical methods to recover a more informative set of symbols. This tension between symbolic and statistical representations, I think, lies at the heart of the nativist/non-nativist debate. Nativists tend represent linguistic regularities in terms of symbolic structures, which are readily interpretable (at least once you have the discrete math down) but hard to learn because the learner has to make several all-or-nothing decisions. Since a small number of bad decisions renders the whole system incapacitated (and are mathematically inevitable in the unconstrained case), the learner must have a lot of _native_ knowledge to prevent bad decisions (to produce the right constrained case). Non-nativists tend represent linguistic regularities in terms of, essentially, degrees of correlation between random variables. Correlations are (comparatively) easy to learn because they require only small amounts of commitment at every decision point, so bad decisions aren't as destructive and native knowledge is not as important. However, it's not easy to interpret correlations between random variables, especially when working in the high-dimensional data spaces that characterize statistical methods.

The thing I like about my project is that it holds that symbolic representations are essentially _what_ people learn (in the limit, as learning converges provided input generated by a stable grammar), and that such representations can be recovered through statistical techniques. It wasn't a complete success, but I did find that the statistical methods I looked at came up with a set of symbols that encoded linguistically-interesting distinctions (like phrase-final particle, phrase-medial particle, and particle lacking a locally-realized complement... ask me if you actually care :p).

The form of the project even reflected (or perhaps drove?) a focus on the relationship between symbolic and statistical methods. I took (project advisor) d.m.'s symbolic parsing course in the winter, and was taking (current project advisor) c.b.'s statistical natural language processing (stat. NLP) course when we were getting the project geared up. D.m. doesn't do stat. NLP, and I didn't know enough at the beginning of the project to realize that's what I was wanting to do. So I kept coming at d.m. with all these statistical and information theory things, and he wasn't very familiar with them either. It created an absolutely phenomenal environment of genuine collaboration where I'd work to think about these statistical techniques in a way that my symbolist advisor would find compelling. In the words of c.b., we "taught each other a bunch of highly advanced statistics" at the end of Spring quarter and over the course of the Summer to carry out the project.

This tension between representations is a good one to zero in on because it's also relevant to the engineers who might be reviewing my NSF app. Why do I think kids might be using statistical methods? Because statistical methods work without requiring a bunch of detailed (= expensive) prior knowledge. But the general form of symbolic representations are important because they can inform the architecture of the statistical induction system. So even if we aren't interested in modelling (i.e. eschewing special knowledge sources like dictionaries that kids don't have or learning mechanisms like straightforward backpropagation that are not particularly psychobiologically plausible), the relationship between symbols and statistics is important.

As I was writing that terribly self-inflating second paragraph at the top ("So what should..."), I was thinking about this perennial puzzle I have (aside from the one about how I am alright at some things and unforgivably abysmal at some certain other thing). How did I get here? On late nights towards the end of the quarter, when the stress feels absolutely unbearable, I tell myself "This is how you got here. Don't worry about the clock. You're in this situation now because you've done it before and done it well and knew you could do it again under more pressure. And you'll be better off and glad for it when you've done."

I hope that's true and I'm not just the opportunistic beneficiary of socioeconomic circumstance. Or genetics. In third grade (okay late primary) when I was (unsuccessfully) organizing chess tournaments, it genuinely did not occur to me that other kids might not value intellectual prowess as much as me. Maybe I just lucked out on the interest lottery?

Of course, there's no such thing as free will :p So I suppose it's just a matter of how I've experienced this dive into understanding what there is. Which experience of course is complex and changing and would turn this into a monster of a post.

A few weeks ago at ACL I met with the recent Ph.D graduate a.r. whose thesis I'm extending. He's working on turning that thesis into a paper for a major journal ("Computational Linguistics"), and had a meeting with his two (former (sort of)) advisors. One of them is my advisor c.b. and the other I've been working with closely on my project, and so they let me sit in on the discussion. It was amazing, and about precisely what computional linguists per se (i.e. not cognitive scientists) find compelling and interesting about the relation between symbolic and statistical representations. Srsly guys I got chills. Psycho Comp Ling where it at yo.

Sometimes I think I should be more social. When this happens, I make an effort to spend time with people. Sometimes it works out nicely and I have a really good relaxing time. Usually when this happens it is because I am spending time with somebody I know very well who I haven't spent much time with for a while, or because I am meeting somebody new and have not decided whether I want to bother spending time with this person. Sometimes I am reminded that I really am just abysmally bad at that some certain other thing. Most of the time I end up disengaging from whatever the event is and wishing I had done more work. Why do I like to do work on all this so much? This is probably an important question that I need to answer.

Hm. This has been a productive free-write session. I think I will modify some bits of this and incorporate it into my essays. Thanks guys ^_^


-=-raptur-=-