Python Gotcha, Nested Lambda to the Rescue

*Note: This has been sitting sitting around as a draft for quite awhile. I’m finally posting it because it came up recently; luckily this time I was prepared.

This was brought to my attention by Jason Yan of Disqus.

Consider the following code fragment. What’s is it’s output?

1
2
3
4

num_funcs = []
for i in xrange(0, 101):
	num_funcs.append(lambda x: x + i)
print num_funcs[10](20)

If you guessed 30, look again and consider the value of i during the print statement. It’s value will be 100.

What is num_funcs[10]? Clearly it is a lambda function. More importantly, it is a lambda function which has not yet been evaluated. Now consider again, the value i. What’s the the output? Hopefully you see that it is not 30, but rather 120. If you’re still unsure why that is true, let’s code the loop in such a way which will produce the expected output. Namely we would like num_funcs[n](20) to evaluate to n+20.

The trick here, is to force i to be the iteration value at the time of the append, rather than letting it persist as a variable until the print statement. To do this we simply close over the original anonymous function with another, thus establishing deeper scope and hence forcing i to be evaluated at the time of the append, like so.

1
2
3
4
5

num_funcs = []
for i in xrange(0, 101):
	func = (lambda j: (lambda x: x + i))(i)
	num_funcs.append(func)
print num_funcs[10](20)

Note that I’ve the argument to the outer lambda is called j, but we are passing i.

This is a little tricky, because it is so subtle. Don’t get burned.

Spam Everywhere

Update: I’m now using the Disqus comment system. They provide a nifty service; check them out.

I’ve not been writing very much lately so I haven’t been maintaining comments either. I just marked over 50 comments as spam!

What is the best way to handle all the spam!?

Visit to UC Berkeley

I am writing this post from my temporary desk in the RAD Lab at UC Berkeley in the Computer Science Division. I’ll be spending some time here learning about Hadoop, a framework for distributed computing. Luckily for me, my long-time friend, Michael Armbrust is a fourth-year CS PhD student here working on distributed applications. There are a lot of people here doing research in large-scale distributed and parallel computing, including Michael.

Since third grade, Michael and I would spend countless hours in front of our computers. We wrote little applications in QBasic then graduated to Visual Basic, all while a small business venture in the background. Our largest project was called ‘System Assistant’. The System Assistant was a plugin-based system tray bound application. It’s main function was to load DLLs of individual self-contained helper applications. I can’t even remember how many plugins we wrote for it, but it was pretty awesome.

After Michael picked me up from the North Berkeley BART station we went the the CS graduate student social hour which included wine and cheese, bread and even smoked salmon. After that we sat down for a bit to talk about setting some goals for my trip before going out on the town.

We were finally able sit down in front of a keyboard, like old times, yesterday after dragging ourselves out of bed for a 9am conference call. Eventually coding ensued, and about 14 hours later still sitting in the same conference room since that morning we had managed to encapsulate my existing Python code to run it using Hadoop. Around 11pm that night we had run a Hadoop job against 10,000 PDS files totaling ~30GB. Processing took just barely over 15 minutes. The bottleneck turned out to be loading the data into HDFS, which itself took about an hour.

I’ve got a really good foundation now and can spend next week tweaking and polishing everything.

Visiting The Adler Planetarium

I met Lucy Fortsen, Vice President for Research at Adler, while we were both attending the 214th meeting of the American Astronomical Society in Pasadena, CA back in June. Some of the Galaxy Zoo team members went out for a few drinks one night and we got to talking about how wonderful Chicago is (a fact I was reminded of as soon as I took my first step outside of Union Station). Lucy and I got to talking about Galaxy Zoo and our roles in the collaboration. One thing led to another and before I knew it I was planning a trip to visit Adler and other people involved in the project.

I spent most of my day yesterday July 20 doing two things: traveling, or visiting The Adler Planetarium. The train ride from the St. Louis area is five hours long. I spent most of those five hours working on projects for my part in the Galaxy Zoo collaboration. Eventually, at 10:00 am + 20-something minutes of lateness I arrived at Union Station in the heart of downtown Chicago and was off the the planetarium.

My original plan for the week was to meet with Nancy Dribins on Monday, Doug Roberts sometime over Thursday or Friday and meet with Lucy when time allowed. As it turns out we all met that Monday, which also happened to be the 40th anniversary of the Apollo 11 landing. Needless to say, there was a lot of excitement in the air that day, and cake.

The Apollo 11 Lunar Module Cake

I also played with the new ‘Moon Wall’ that Adler recently debuted. The Moon Wall is basically at giant display for high-res lunar images. The display simulates flying around the Moon and has a basic joystick control. I was pretty neat.

The Moon Wall at The Adler Planetarium

I also managed to find some time to walk around and see a couple shows. It had been several years since my last visit to Adler, so I had a lot of fun and couldn’t have visited on a better day. Here are a few other pictures I took.

Chicago Skyline with 'Man Enters the Cosmos' in the foreground

The Time Has Come

It’s official; SIUE has begun removing all the clocks on campus. [1, 2]

I am in my shared office in the basement of the science building. It’s about thirty minutes to midnight. I only know that because I have computers (and three screens).

Just a moment ago someone came in here looking for university owned time telling devices. The are none in here, but he is beginning to gather a collection.

I think this is one of those things that you won’t miss until it’s gone.

[1] SIUE clockwork’s origin
[2] Broken clocks gone in 60 seconds

Hello Blogosphere!

Yes, that was a programming joke.

I cannot say this is my first attempt at blogging. I have a blogger account with, I believe, one or two posts. I hope this time I can actually push content on a somewhat regular schedule (read: when I think I have something interesting to say and time to write about it). I am not quite sure where this road will take me. I am not sure what topics I will write about. I do expect, however, the number of categories to grow quickly and unwieldily.

One thing I would like to do here is publish reviews of books, movies and music. I admit that my library mainly consists of textbooks and technical material. I also have a handful of science titles along the lines of Feynman’s Six Easy Pieces. My taste in movies, much like my taste in music, is much more eclectic. I do not dare attempt to describe either of them now.

Other than that you can probably expect to hear me rave and rant about the things that currently occupy my time. Recent developments and current events surrounding the scientific community seem to consume much of my time these days. As I am a student of physics (gravitating towards astronomy) with a strong background in computational methods you can also expect to read about such things. Just a couple years ago I would probably write about Fortran and maybe…maybe C/C++ but those days have been eclipsed by Python.

That said, I hope you will join me. Please do comment, as having an audience it sure to keep me motivated and keep the posts coming – this time.

Hello world!

Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!