I primed the pump on the flight to PyCon by catching up on my reading. “Expert Python Programming” (Tarek Ziade) reminded me that I wanted to play with ipython shell and virtualenv (just learned today about the handy extension “virtualenvwrapper”), and reinforced and offered many great Best Practices.

Optimization Tutorial

I then got on my geek at “Faster Python Programs through Optimization” (Mike Müller of Python Academy), where we dove deeper into profiling and tips on improving speed or saving memory.

Some paraphrased guidelines to consider before you start optimizing (which were also reinforced in the “Real World Django” tutorial which I’ll chat about next):

• Make sure your program is really too slow - could be other factors like network traffic, database, etc.

• Don’t optimize as you go - might ultimately not need to spend that time.  Also working code is always important first.

• Only consider realistic use cases and user experience

We played with the profiling tools (profile, cProfile, time, pystone, heapy) and used them to compare various techniques:

• xrange and also Generators shaved off time by not having to allocate memory for large data sets.

• use built-in types as much as possible (including some newer collection classes)

• iterating and appending strings by first appending to lists, then using a join statement to create large strings (versus building strings via += and loops)

• One new one for me was converting lists to Sets before testing for membership of an item in the list, which is fast due to Set optimizations.

• The tutorial also covered pysco, processing and numpy modules, as well as caching techniques.

Real-world Django Tutorial

This very aptly named presentation by Jacob Kaplan-Moss and James Bennett was excellent for those of us who develop and deploy Django websites.  The full skinny (with link to slides) is here: http://jacobian.org/speaking/2009/real-world-django/

Some highlights for me included:

• Focus on tight Django Applications that promote reuse while also breaking a website into components. Benefits of also leveraging packaging up your own components.

• Gain flexibility by leveraging Django Managers, and they help encapsulate behavior behind an API.

• Can extend models via new (in Django 1.1) Proxy subclasses.

• Lots of discussion and recommendations for testing – from unit testing, through functional testing, and then browser-based functional testing. Yep, you need them all. I’d like to play more with Twill and Windmill.

• Automating deployment - including options like virtualenv (and virtualenvwrapper), Ian Bicking’s pip (“pip installs packages”), zc.buildout, and Fabric.  zc.buildout’s power was emphasized (with its recipes, etc) was a bit overshadowed by comments on lack of documentation. I’d like to give pip and Fabric a try.

• Apache + mod_wsgi is now a preferred platform for server Django sites (or at least much more consistent performance and memory-usage wise than Apache + mod_python).

• Definitely flip through the session slides!  These were just some highlights for me out of 189 slides of useful information.

Various tidbits for the next few days here at PyCon:

• Open Space sessions come highly recommended

• There’s a heavy testing thread throughout conference (10 sessions worth!)

• Friday 11am: Using Windmill

• Saturday 4:15p: Ian Bicking’s session (creator of PIP and virtualenv, among many other topics)

• Sunday 10:35a: Panel: Functional Testing Tools in Python

• … though it will ultimately be tough to pick and choose from all the great topics!

Time for some sleep… more tutorials tomorrow, then 3 days of conference, then 4 days of sprints!

ps: It’s been great to see familiar faces from the Zope and Plone communities, which is often where I “get my Python on”.  Lately I’m also doing a lot of Django, so enjoying all the synergy around Python here at PyCon2009!

Congratulations to Jon Ramer from The Interra Project, and Mark Okazaki from Neighborhood House for winning this year’s Bill Grace Leadership Legacy Award awards from the Center for Ethical Leadership.

Both the Interra Project and Center for Ethical Leadership are clients (and friends) of ours at Web Collective and we’re impressed with the great work they are doing.

We developed websites for them in Plone, including Boston Community Change, Puget Sound Community Change and Center for Ethical Leadership.

This award “celebrates Puget Sound leaders whose vision, commitment and unceasing efforts are significantly advancing social, environmental, and economic justice.”

Tickets are available for the 4th Annual Bill Grace Leadership Legacy Awards Dinner on March 5th, 2009.

The Interra Project “provides consumers with the incentives they need to shift their purchasing habits to support the health of their communities by shopping with locally-focused, environmentally and sustainably-minded businesses.”  I was part of the team that built the Boston Community Change website.  Now there is a  Puget Sound Community Change in our local community.  You can sign-up for a community change card for free and use when purchasing at local businesses, or if you own a business you can offer your services to those who have cards.

My experience with Neighborhood House is through a monthly Multicultural Committee meeting where I live (at New Rainier Vista) where they provide language translation and other great community services. Our community has native speakers from Somalia, Ethiopia, China, and Vietnam.

I enjoyed hanging with the local Python crowd yesterday in Seattle for Northwest Python Day 2009.

As usual, Python is popular in many realms.

Who attended?

We started with quick introductions - a nice mix of folks with some traveling from Portland OR, Vancouver BC and even one from Chicago and DC.  Many folks using Python – several announcing Python job openings.  People were from various organizations such as University of Washington, NOAA, ONENW, Web Collective, NPower, LexisNexis, Microsoft, Sun, and many interesting companies I didn’t catch the names of.

Quick Highlights

We started with a lightning talk with tips on moving your code toward Python 3.0 (running Python 2.6 with -3 option; using future, running 2to3).

Then saw a light-weight web framework called Werkzeug - I like its idea of decorating a Python view function with its URL mapping [e.g. @expose(‘/’) to connect a view with the root of the site].

We then heard about the ease of leveraging buildbot for testing.

NOAA started the presentations with their CAMEO Chemical modeling application, “a Pylons-based web app wrapped in a wxPython interface for desktop use.”  There were various complications making this work cross-platform on both IE and Safari, but overall successful.  Chris has high hopes for upcoming wxWebKit (which wasn’t quite mature enough at the time they were developing their app), and might consider pyQT or pyGTK for future projects.

University of Washington’s Beraber was interesting - a way to offer open source cloud computing (via a Python-based VM) by sharing your computer safely with others, and being able to run programs on many computers around the world.

After lunch, lightning talks resumed with Sphinx, an RST based system for writing documentation for your code (used for Python’s documentation).

We then saw NodeBox, “a Mac OS X application that lets you create 2D visuals (static, animated or interactive) using Python programming code and export them as a PDF or a QuickTime movie.”  I checked out their website – some cool plugins like modeling of flocks.  You could probably make some very cool desktop wallpapers with this too.

If you’d like to play with virtualization and open source, Derek Simkowiak is working on a program called “vmshell” that allows you to more easily manage virtual sandboxes.  Management of VMs was mentioned as something missing from many open source VM solutions.

Our first afternoon presentation talked about the benefits of high-level languages like Python and benefits over lower-level languages like C++ or Java.  Mark McWiggins presented good arguments for why organizations may want to consider Python over these other languages.

Sage, and its 5+ million lines of code, offers open source math modeling.  For those that need Mathematica, Magma, Maple, or Matlab power, Sage was impressive – from interacting with and showing complex math formulas in Python and Javascript, to live 2D/3D plotting, to importing the library into your own Python program and going to town.  One of its innovative features (from a web dev perspective) is writing a math function (in Python) which you want to interact with it on the web – instead of creating your own web form, you can decorate your function with @interact, which introspects the function parameters and automatically creates a web form for that function.

After having played a bit with Google App Engine, it was nice to hear a real-world experience about using this in a production project.  Web 2.0 apps can be a sweet spot for GAE, though there are differences with other traditional web development methods that may help determine if your app fits GAE or not.  I won an online O’Reilly book on this topic.

I had seen mention of Cython, but hadn’t investigated.  Cython is a way to compile your Python code in C code for major speed improvements.  It has some cool profiling features like an interactive web-based code display that uses light-to-dark color-coding to show which Python code lines are the slowest, and allows you to click on the line to see the actual C code that was generated.

The last presentation was by Sun, who are investing in Python (and other languages in addition to Java) due to their popularity by programmers.  They are also investing in Jython (adding more resources than before) to bring this up to latest versions of Python 2.x, and some work on the JVM to support languages other than Java.

Pycon

Registration just opened for PyCon 2009 (in March) in Chicago.

I plan on attending this year, hope to see you there!

Thanks!

Thank you Seattle Python and the University of Washington for hosting!

Santa brought me an Arduino this year - an open-hardware and open-software platform for connecting computers to the physical world – used by artists, designers, geeks, entrepreneurs who often find interesting things to create.

Now I’m finding renewal in a mix of art, physical computing, and the new concepts and ideas that come along with that.

What would you do with a platform where you can buy interesting off-the-shelf components (such as GPS, wired/wireless Internet connectivity, mini-cameras, color/light sensors, accelerometers) that you could put together then control via a small $35 micro-controller, or easily connect to your computer as input or output?

The possibilities are endless.  Try searching for “physical computing” or “arduino” on YouTube.com for some ideas, or see Arduino Playground.

Creating gizmos that interact with the physical world make keyboards and computer screens pretty darn boring.

Personal note: Renewal is always very energizing.  In regard to my software engineering interests, when I discovered Open Source in 2000, I learned everything I could about Linux and Open Source software, gave talks, ran Linux as my main desktop for many years.  When I discovered Zope and Plone and Python in 2002, I helped fuel a local Plone community, and based my whole business on Plone (RagingWeb.com and now at WebCollective.coop) and never looked back.  More recently I continue to find interesting projects in Plone and Django, while playing with Google App Engine, the Apple iPhone and the myriad of social applications that keep popping up.

What’s happening with Physical Computing in Seattle?

I missed an Arduino class in November, but discovered dorkbot seattle and attended my first meeting last night at 911media.org.  I found a vibrant every-seat-taken mini-auditorium full of people ready to hear the night’s line-up (see photo above).

Here was the [paraphrased] lineup:

First, all three presenters are established artists with interesting projects.  My brief takeaways don’t do justice to their work and knowledge. Please check out the links to their blogs for many cool projects and exhibits.

1. “Josh Kopel: “Out of Control” a quick review of the micro-controller and DIY electronics explosion that was 2008. 2008 might well be called the year of Arduino, as the little micro-controller from Italy invaded the DIY scene and showed up just about everywhere. Beyond just the Arduino, 2008 also saw a vast increase of interest in micro-controllers and unique interfaces for use in the arts.”

Some takeaways: I hadn’t realized Arduino was open hardware and all the variations that have arisen from that, such as the LilyPad Arduino that is a tiny wearable computer.  Josh was mentioning that the mix of an open, inexpensive, and easy to leverage platforms (Arduino is just one of many platforms out there) plus the fact that you can find and buy components in sizes of 1 (versus years ago when you had to buy large quantities of parts) has brought this technology to the masses.  Try searching for Arduino in Google Trends. One project where the husband hooked up a home-made gizmo to his wife that measured each time his unborn baby kicked with a “I kicked Mommy at 5:21pm” Twitter message got a pretty good laugh from the audience.

1. “Tina Aufiero: Computers, wireless cameras, electronics, and swans. Tina will talk about using computers, a wireless camera, and some electronics to marry the abstract concepts and representational forms in her works, which includes “project_swancam”. We know Tina as the Education Director at 911 Media Arts Center.”

Some takeaways: Swans are definitely interesting birds!  Tina demonstrated a mix of her art, growing knowledge and experimentation with technology such as video manipulation and wireless camera, and her love for swans - which she has made into many interesting exhibits, and used to support her activism.

1. “Hugo Solis Garcia: Juum, a framework for multimedia production and composition. In his talk, Hugo will talk about Juum, a framework for multimedia production and composition that he has been developing during the last year. The tool has evolved because of the artistic requirements and the art pieces have been influenced by the program.”

Some takeaways: Hugo was able to augment his talents as a musician by creating new composition techniques using visual programming tools and home-made hardware.  Too difficult to summarize, but my favorite piece was Tell-Tale Piano, which was a chair on top of a box that contained a used piano within. People could sit in the chair and experience dramatic music being created below their feet electronically via the piano which followed the theme of Edgar Allen Poe’s “Tell Tale Heart”.

What’s Next?

• Josh Kopel will likely be teaching another class in March 2009

• This year’s “People Doing Strange Things With Electricity” event is in June 2009, and is a showcase of projects by local artists and hobbyists.  See dorkbot seattle archive for previous events and projects.

I’ve started my first Arduino project, now with additional inspiration from last night’s meeting, which I plan to blog about as the pieces come together…

As I gleam more information through blogs and microblogs there comes a time when the fun wears off and overload sets in.

In 2009, I’ve decided to tame overload and reinstate fun with some simple rules:

For Twitter (and other microblogging services):

• Follow only folks that have a reasonable number of posts per day. Chronic Twitters (some individuals, but often sources like large blog sites or newspapers) force me to weed through a lot of cruft to come to the juicy information I can use.  No one has infinitely interesting things happening all day long.

• Twitter is not email - I find back-and-forth conversation via public twitter replies to be tedious unless one puts a bit of context in their reply.  If the reply is interesting in itself, I’ll spend the effort to flip to the initial Tweet and try to see what was initially said.

• For chronic Twitters that still do have some interesting morsels, I often stop following them on Twitter, but subscribe to their blog RSS and follow lower traffic there – RSS is much easier to skim through than Twitter.  Garden hose instead of fire hose.

• Don’t blindly follow! (good advice in many situations). When someone follows me, I check out their profile, recent Tweets, and number of Tweets per day before jumping in.  If a reasonable number of posts are interesting, and they’re not Tweeting every moment of their lives, I will follow them too.

• Keep it interesting! Sure, not every tweet someone makes is interesting, but many should be. Try out folks, but don’t be afraid to jump off without guilt.  I’m personally attracted to Tweets pointing me to resources, interesting sites, or interesting happenings – in other words, I rely on my social graph to provide interesting content, not noise.  I keep this in mind when posting Tweets for others.

For RSS

• Cut out large noisy RSS feeds even if occasionally interesting.  Feeds that produce too much traffic (albeit this traffic can safely be higher than microblogging since it’s easier to skim) even if a few morsels are interesting should be tossed because they prevent you from reading the really interesting ones.

We each have a limited amount of attention we can give - these simple rules will keep these services interesting and useful.

What other techniques do people use to handle the growing amount of information coming at them through microblogging (e.g. Twitter, Facebook), location-aware microblogging (e.g. BrightKite) and good old fashion blogging (via RSS)?

If you’re using Gmail, and want to manage it using GTD, I found a great solution.

I’m now using Remember the Milk as my primary GTD tool with the Firefox extension for Gmail. The extension lets you see all your action items on the right side of the Gmail page (grouped by day) and lets you add new ones and roll-over existing ones and make quick changes and reschedules.

Its most useful feature is that you can add action items automatically when you “star” (or tag) a Gmail message.  With many of my actionable items and reminders coming in through email, this is a killer feature for me.

For example…

…if a client sends me an email that I need to respond to later in the day, I just star it and it registers as an action item for today.  If I decide I really could answer it tomorrow, I can roll over the new action item, and click the “postpone” button or type in a due date of “tomorrow”.

Then, when tomorrow comes and it’s time to act on this, I roll over the action item and now see a little “envelope” icon which allows me to jump to the email, reply to it, unstar it, and the task disappears because it’s done.

There are some very nice OSX GTD apps (like OmniFocus, and Things) but after personally jumping from client to client, I decided I wanted to go with a web-based service that had an API in case I wanted to extend it with a new UI, or maybe integrate it with another web service.

I looked at many web-based tools, but liked Remember the Milk for its API, its web-based interface (not flashy, but useful), no syncing worries, the Gmail Firefox extension (my main interface for using it), and it even has a nice iPhone web interface (with a native iPhone app rumored).  For the iPhone interface, they ask you to join their Pro membership, but $25 for a year is well worth supporting this great service.

I also built CollectiveGTD for Plone, but have decided to take a different direction with this product.  Instead of it being a tool to manage all my general GTD needs, it will evolve into a set of tools for managing Plone content within a site.

A recurring conference theme for me centered around agility. Plone has benefited by moving from Zope2 to the more flexible Zope3 Component Architecture.  Zope technologies are being made available as middleware for use on other platforms (via Repoze and WSGI). Grok is a Zope3 framework for projects needing something more lightweight. There were several very useful sessions on Agile Project Management.

There are also continuous improvements making Plone more agile for end-users.  Plone 4.0 aims to make it even easier for end-users to manage their content (including better media support, unified widgets for easier page compositioning, “page centric”, kupu improvements around layout), as well as more through-the-web features done right!

Tuesday Arrival

On my Tuesday Amtrak ride to DC, I connected with some conference folks on Twitter, found my hotel and ended up at the Science Club for a few drinks and mini-reunions. Hotel Harrington wasn’t fancy, but clean, had wifi and was only two blocks from the conference and metro, so I was happy.

Wednesday

“10% Manifesto” was my first session. In Seattle, we’ve been holding a weekly Open Source Friday, which has a lot of parallels to discussions in this session.  We’re all practicing ways of putting that extra effort in to give back to the Plone community in which we all flourish, and also socialize and learn in the process.

I then caught up on the latest happenings with Ecommerce in Plone with the GetPaid project.  This is now being used for a variety of use cases, and its component architecture seems to make it relatively straight-forward to create modules for your own needs.

“When Software is a Service, Will Only Network Luddites Be Free” brings up an interesting question: What does a free [as in freedom and open] computing cloud look like? http://autonomo.us

Thursday

I started the day as a member of the “So you want to be a Plone consultant?” panel.  It’s a big discussion for a short amount of time, but we had a nice variety of panelists and good questions. Geir Baekholt’s discussion of Open Scope contracts peaked my curiosity.

One of the questions asked “How do project come into Web Collective?”  Clients often find us through our existing work with many progressive businesses and non-profits, our contributions of software and presentations in the open source community and Seattle Plone user group, and an interest in working with a company that practices a cooperative business model where all the members are owners of the company.

I was pleasantly surprised by a several tracks on Agile Project Management (“What makes a great development team” by Mike Robinson, and “High performance teams: What’s the secret sauce?” by Gerry Kirk on Friday).  Communication and project management often determine the success or failure of a software project, and tips and tricks in this area are always welcome.

Another interesting session was “Using Grok to walk like a duck” which covered several component architectures (from the limits of subclassing all the way through to the beauty of Adapters) and how Grok (and Zope3) support a dynamic version of Adaption.

After hearing about Deliverance 6 months earlier, “Using Deliverance to theme a website” was a nice way to catch up with the latest happenings here.  If you want to apply themes to multiple sites, this seems the way to go.

Friday

Experiencing some of the complexities and limitations of formlib, I wanted to check out Stephan Richter’s “z3c.form” session. z3c.form seems more refined, and can be used in in viewlets and portlets.

After talking with Jim Fulton at the Plone Summit at Google several months ago, I’ve been anxious to play with ExtJS, but haven’t had an opportunity.  Godefroid Chapelle showed us a rich browser solution using ExtJS and KSS.

“Repoze.bfg: A zope explosion” - Several months ago, we had the pleasure of bringing Chris McDonough to Seattle to talk about Repoze.  It was pretty new back then, and WSGI wasn’t something I had played with.  Since then, and after working with some other platforms such as Django, I was excited to catch up with the latest on this project.  The primary concept is to make Zope technologies available as middleware which could mix’n’match with other Python technologies via WSGI pipeline. In addition to the nice lightweight infrastructure, I think this highlights a longer term vision of being able to share best-of-breed Python technologies across various Python frameworks.

Saturday and Sunday Plone Sprints

Record turnout of people at the Plone sprints!

I was happy to collaborate for a couple of days with Aaron VanDerlip on Plone4Artists Calendar.  I had used this on a recent project, and wanted to help make improvements to what’s becoming a defacto solution for Plone calendaring, including an implementation of recurring events.  I’m relatively new to the project and have been impressed with the product’s architecture and its recent 2.0 alpha release.  I enjoyed lending a hand to help fix bugs and refining some functionality.

I also though the concept of a “Beginner Sprint” was ingenious and helpful for including new people in the community and getting them up and running with everything they need. Nice work Jon Stahl and Joel Burton (and other Plonistas that participated).

Monday, Heading Home

Then a perfect ending – I lucked out on a direct flight to Seattle (pays to get to the airport early) and ended up sitting right next to my friend and fellow Plone collaborator Andrew Burkhalter.

Scale and Performance - Being Awesome

After Web Collective just finished a Django website that needed to scale to 12,000 simultaneous peak users, Flickr architect Cal Henderson’s keynote was timely and very entertaining - particularly his commentary on “serious” frameworks and essentially a comic roasting of all of the major ones.

Since most of us are building “houses and not skyscrapers” there is some question as to whether Django should introduce all of Cal’s suggestions (who is a “scale” geek working with on a very large website), but how can you argue with his three major focus areas of scale, performance and “being awesome”.

Discussions of “scale” and “Django” remind me to play with Google App Engine soon. I also just discovered Django Debug Toolbar and other interesting things reading through Cal’s blog.

Learning from other Python Frameworks (Django and Zope)

With Zope (and Plone) being the platform I’ve used for some time now, I found Mark Ramm’s perspective on how to learn from all the smart folks in various Python (and other) frameworks a good 10,000 ft reminder.

He highlights that Zope3, Turbogears, and Pylons have learned some of the lessons of old Zope2 in that they are frameworks that strive to use community components that work with each other in multiple contexts.  They also are trying not to isolate themselves from the Python web community by building functionality specific to just one platform.

He wasn’t saying Django was far afloat in these areas, but rather that Django could make some of its innovations available more easily to other frameworks (e.g. Django’s ORM, and Django’s template engine), and that it should maybe use some of the Python components from other frameworks (e.g. Django middleware and WSGI middleware compatibility).

I look forward to poking around the rest of the DjangoCon 2008 videos, such as Guido van Rossum’s keynote and others on YouTube’s DjangoCon 2008 Session’s playlist.

This is a Plone product that implements an Open Source version of the Getting Things Done (GTD) methodology.

New in this release: This version fixes all known unicode issues, so that non-ascii characters can now be used throughout interface for editing, viewing, and KSS inline editing.

For download, please visit http://plone.org/products/collectivegtd-thoughts/releases/1.0b2

Change log:

• NEW: Fixed all known unicode issues which were causing context/project tagging to break when editing and when viewing, and were preventing KSS from updating the UI. Language-to-date features are still only supported in English.  Templates have not been internationalized.

  • Future Plan is to replace home-grown language-to-date functionality with the parsedate python module, which also supports multiple languages.

• NEW: Added new getAllActiveActions(outputFormat=”xml) method to spit out all ActionItems, Context tags and Project tags for use by a desktop application.

• INCOMPLETE: Started to build an iPhone interface. Partially complete. Further work on this postponed.

Cooperative Businesses are relatively new in the consulting and services field, as compared with more well known producer coops.

We’ve spent the last two years forming and running a Cooperative Business, and hope others consider doing the same.

The combination of organizing as a Cooperative, being part of Plone and other Open Source software communities, and partnering with clients who share our values, have led to a strong year of collaboration and growth.

Here is a brief list of incentives and highlights for others thinking about alternative business models that may include Cooperatives and Open Source software.

Web Collective, Inc recently celebrated its one-year anniversary as a Washington State Cooperative Business.