I was performing functional tests on my models that employed Attachment_Fu this morning and thought it would be worthwhile to share the code since it was a bit of a hassle pulling it together. Kudos to Mike Subelsky for his introduction to functional testing Attachment_Fu. It got me going in the right direction. What proved difficult once again was the multi-model controller. Once I got over that hump, I was on my way. As you can see from all the detail in the HTTP POST below, that was not an entirely easy task.

class ProductsControllerTest < Test::Unit::TestCase
...
def test_create_with_user
    num_products = Product.count
    imgdata = fixture_file_upload('/files/image.png', 'image/png')
    audiodata = fixture_file_upload('/files/sound.mp3', 'audio/mpeg')
    post :create, {:product => {
            :name => "Widget",
            :description => "A small tool-like item",
            :weight => "3",
            :price => "19.99",
            :language_id => "1"
                   },
            :image => {:uploaded_data => imgdata},
            :audio => {:uploaded_data => audiodata} ,
            :html => { :multipart => true }
          },
          {:user_id => users(:valid_active_user).id}
    assert_response :redirect
    assert_redirected_to :action => 'show'
    assert_equal num_products + 1, Product.count
  end
  ...
  end

Continuing my Rails on Windows thread, I’m going to spend a bit of time on something that’s brought me both some substantial gains and some minor woes lately, running the Attachment_Fu plugin on Windows. I’ll start off with some general Attachment_Fu information and then get into some of the quirks, which are, as expected, mostly specific to the Windows environment.

First, for those not in the know, Attachment_Fu is a Rails plugin that allows you to store binary data (e.g. images, video, documents) and associate it with other models in your Rails application. Metadata (content type, size, height, width) about the attachment is stored in a separate model. Attachment_Fu’s sweet spot is handling images. It can handle automatic image conversion and thumbnailing using a number of popular image processors such as ImageScience, RMagick, or minmagick. Although not provided, you can imagine that Attachment_Fu might be extended to handle other types of binary processing utilities such as PDF converters or audio/video transcoding software. The other very cool thing about Attachment_Fu is that it provides support for pluggable persistence mechanisms. Out of the box, it allows for storage on the file system, as binary information in a database or on Amazon’s S3 storage service.

There is an abundance of information already written about Attachment_Fu so to avoid re-inventing the wheel, I’ll provide what I found to be the best sources of information to start.

• Mike Clark’s tutorial is the gold standard introduction to using Attachment_Fu. The code is simplistic but rock solid. It covers using both the file system and S3 for storage and will get you up and running on Attachment_Fu in no time.
  

• This post on the Attachment_Fu message board provides a solution to associating the attachment model with another model (i.e. making it an attachment to something). The post provides both the controller and the view code for uploading the initial attachment and rendering it. Handling the attachment relationship in your MVC is going to be a fairly common requirement and most Attachment_Fu users will benefit from this post.
  

For my part, I’m going to provide some controller source code for updating the attachment when you have a relationship with another model (an extension of the second item above) since this is one area that wasn’t covered well anywhere else and might save you some time in your travels. In the code below, my main model is the product and the image is the model where a photo and thumbnail are stored using Attachment_Fu.

class ProductController < ApplicationController

  def update
    @product = Product.find(params[:id])
    # Load up product categories for the view
    @all_categories = Category.find(:all, :order=>"name_en")
    if @product.update_attributes(params[:product])
      if !params[:image][:uploaded_data].blank?
        # My product only has one image / thumbnail, I'll destroy 'each'
        # wait 3 # See quirk no.1 below
        @product.images.each {|img| img.destroy}
        @image = @product.images ||= Image.new
        @image = @product.images.build(params[:image])
        @image.save
      end
      flash[:notice] = 'Product was successfully updated.'
      redirect_to :action => 'show', :id => @product
    else
      render :action => 'edit'
    end
  end
 
end

The links above, in combination with my snippet, should get you through creating an attachment and handling CRUD for an attachment and its parent model from a single view. Now comes the Windows quirkiness. Not knowing to expect these Attachment_Fu quirks and then having to root out the cause of the behavior took up a lot of time. It turns out that most of I found that most of the quirks are documented in some way, shape, or form. I’ve pulled together a list of the quirks as well as some best practice workarounds.

• When running Attachment_Fu on Windows, the most commonly accounted problem is the “Size Is Not Included In List” validation error. This post goes into some details and speculation around the cause of the issue. It appears that no amount of fixing in the Ruby code is going to help here since it appears to be a Windows file system issue. The workaround is really simple, just add a wait x statement before your attachment processing and things will be golden. The x (which denotes seconds) time will vary based upon the size of the attachments you are processing. Bigger attachments require more of a wait. Also, be sure to comment this code out in production since this is a Windows only issue.

7/19/2007 Update - Rick suggested using RUBY_PLATFORM to determine if the wait should be invoked. I tested this and it worked as suggested

• When you invoke the destroy method on your attachment using Attachment_Fu on Windows, your models reference to the attachment will be deleted but the physical attachments themselves will not be deleted if you have persisted them to the file system. If you look at the Attachment_Fu source code or your log files, you’ll see that Attachment_Fu assumes that you are using a UNIX-based system and executes UNIX commands like rm to remove these files. These commands will obviously not work in a Windows environment, leaving you with a bunch of zombie files. This should not be a problem if you use a database or S3 persistence mechanism since these mechanisms are independent of the operating system.

7/19/2007 Update - Rick corrected me. He is indeed calling the OS safe FileUtils.rm in the file system backend. It still isn't working though - at least on my machine.

• My last Windows specific quirk is actually an Internet Explorer issue. If your attachments are images, you may have problems with uploading JPEG’s using the default Attachment_Fu plugin. From what I’ve been able to determine, if you upload a JPEG from IE with a file extension of .JPEG, IE will set the MIME type to image/pjpeg for a progressive JPEG. However, if the image extension is simply .jpg, IE will set the MIME type to image/jpg. This MIME type, however, is not included in the default list of content types accepted by Attachment_Fu. My suggestion is to add this type to the list in the source code until Rick can get around to modifying the source.

7/19/2007 Update - The MIME type was added to source. For reference, Rick suggested that this could have been done without changing the source simply by adding
    Technoweenie::AttachmentFu.content_types << 'image/jpg

The last quirk for my post should be meaningful to all of those using Capistrano, the Rails migration utility. Capistrano manages versions of the application for rollforward / rollback by creating symlinks to previous versions of an application and deploying the most recent version of your entire application tree from your version control system (e.g. Subversion). However, since it’s very unlikely that you are storing all of the attachments for your application under version control, the attachments will be unlinked and no longer available when you migrate a new version of your application to production. To get around this issue, the solution proposed here creates a separate physical directory for the attachments outside of your application’s directory and then updates a symlink from your application’s attachment directory to the separate physical directory every time you migrate.

I’ve had some really good experiences with some of the iTunes Original collections, which include a mix of pre-existing songs, original versions of hits and artist narrations. I’ve especially enjoyed the iTunes Originals with Rob Thomas. This weekend, I picked up my first iTunes Exclusive Live Sessions mix. The Live Sessions series at 5 or 6 songs per collection offers only about half the music of your average Original collection but, as the title indicates, it’s all live music.

Since I’ve downloaded the Five for Fighting Live Session from iTunes, I have not been able to get the music off of my mind. I’ve been listening to Five for Fighting since their first CD, which accompanied my wife and I on a memorable trip down the US West Coast. Even if you can’t associate the Five for Fighting name with a particular song, it’s fairly likely that you’ve heard their music since it gets a good amount of radio play and has found favor with a number of TV commercial producers.

Granted, you are not going to get any original music here but what you do get is Five for Fighting’s best material done live in a pure acoustic (piano and guitar) format. The album is tight and the recording quality is superb. Artist narration, storytelling, and interludes are edited out except for one story about the writing of the song Two Lights which really accentuates that piece. At about $5 for the collection, you really can’t go wrong with this one whether you are an old fan or someone simply looking to pick up some great music to listen to.

I’ve been putting a good deal of time recently into converting GeoGlue from .NET to Rails. One of the things that I’m looking to get into the alpha release is the dynamic creation of podcasts. This is really nothing special since a podcast is little more than a special case of an RSS feed that points at external media files (i.e. audio or video).

I plan on covering the audio/video entry in an upcoming post about the nuances of the Attachment_Fu plugin on Windows. In this post, I’m going to just lay out the code for the podcast creation, since this is nothing more that a simple rxml file. I’ve sprinkled in comments liberally but most of the code should be fairly self explanatory to those familiar with Ruby and RSS feeds.

xml.instruct! :xml, :version=>"1.0", :encoding=>"UTF-8"
xml.rss('version' => '2.0') do
  xml.channel do
    xml.title @podcast.name
    # Self-referencing link
    xml.link url_for(:only_path => false)
    # Important --> RFC-822 compliant datetime
    xml.pubDate(Time.now.strftime("%a, %d %b %Y %H:%M:%S %Z"))
    xml.language "en-us"
    xml.ttl "40"
    # User who caused the feed to be generated
    xml.generator User.find(:first, session[:user_id]).name
    xml.description @podcast.description
    # 'public_filename' is a method from the Attachment_Fu plugin
    xml.image do
        xml.url url_for(:controller => @podcast.images[0].public_filename, :only_path => false)
        xml.link url_for(:only_path => false)
        xml.title @podcast.name
        xml.width @podcast.images[0].width
        xml.height @podcast.images[0].height
    end
    @podcast.entries.each do |entry|
      xml.item do
        xml.title(entry.title)
        xml.link(url_for(:controller => entry.audios[0].public_filename, :only_path => false))
        # User who actually generated the media (i.e. audio)
        xml.author(entry.user.name)
        xml.category "Uncategorized"
        xml.guid(url_for(:controller => entry.audios[0].public_filename, :only_path => false))
        xml.description(entry.description)
        # Simplification, you should pull from updated_at/updated_on
        xml.pubDate(Time.now.strftime("%a, %d %b %Y %H:%M:%S %Z"))
        # The enclosure is very important!!
        # If you use Attachment_Fu, everything you need is included in the model
        xml.enclosure(:type=>entry.audios[0].content_type,
                :length=>entry.audios[0].size.to_s,
                :url=>url_for(:controller => entry.audios[0].public_filename, :only_path => false)
          )
      end
    end
  end
end

A couple of lessons learned from my experience. Firstly, Apple provides some good resources on generating podcasts. This is especially important since the iTunes crowd is a large and important contingent of the feed consuming world. There are iTunes-specific tags (and a schema) available. These tags are not mandatory (I didn’t use them here) but they will help you produce a richer feed for consumption within iTunes. Secondly, since the RXML file is just another view, make sure to turn off any default layouts that you might have applied to your other views. I’ve included a snippet below to demonstrate how to do this. Check your version of Rails, mileage may vary with exempt_from_layout based upon your release.

class ApplicationController < ActionController::Base

  # Pick a unique cookie name to distinguish our session data from others'
  session :session_key => '_trunk_session_id'
  layout 'default'
  exempt_from_layout :rxml
  ...

end

My final caveat is not to apply forms-based authentication to your podcast (RXML view). Either make the view public or, if you wish to protect it, do so using HTTP Basic authentication instead. If you’re using both forms-based and HTTP Basic authentication, you’ll probably need to sync the two by using a single LDAP repository. That’s fodder for a completely different post.

As soon as you’ve spent some time dealing with Rails, you’re bound to hear the fact quoted that the entire Core Rails Team does their work on Macs. There are likely several reasons for this: (1) these folks really like Macs (you can’t fault them for that); (2) they’re getting kickbacks to use Powerbooks (could be; not likely though); or (3) Rails is fun, and using Windows puts a bit of damper on that fun. I think the last answer is the most likely even though I’d like to think that Steve Jobs has some skin in the Rails game.

What you’ll also hear and experience when dealing with Rails is that it’s “opinionated software”, which it is. It just so turns out that the prevailing Rails opinions tend to align more closely with the UNIX-derivative camp (like Mac’s OS-X) than with the Windows camp. There’s a price to pay for working against the prevailing opinions and using a Windows environment to do your development. In most cases, the community that supports Rails has done a great job to make sure that this cost is very miniscule. However, once in a while, when you’re working with a Ruby Gem or Rails Plugin that is outside the core framework, you’ll hit the opinionated software wall head-on.

This is not meant to be a critique of Rails or the concept of opinionated software. Rather, the things that make Rails and some of these Gems and Plugins so special is that they leverage existing capabilities of the underlying operating system (that’s Rail’s DRY principle in action) such as the UNIX symlink command that powers Capistrano. These capabilities are difficult or impossible to replicate across operating systems; leaving the Windows-based developer with three choices: buy a Mac; install Linux, or hack your way through.

In the first of x installations of my experiences with Rails on Windows, I’ll touch on some of the learning points I had with Capistrano. As a refresher, or for the completely uninitiated, Capistrano is, in the simplest sense, a Rails deployment utility. It provides a collection of tasks that anyone with experience in deploying Web applications will immediately recognize as extremely useful. Tasks like automated deployments, checking the differences between your most recent source and the existing deployment, temporarily disabling an application and putting up a maintenance page, performing database migrations, and rolling back your application to previously deployed versions can each be performed using a single Capistrano command. Like many things Rails, the obvious utility of such functionality may lead you to wonder why a tool like this wasn’t invented much sooner and used universally.

Capistrano is quite overt about being opinionated software, going as far as to clearly document the assumptions it makes. Amongst these assumptions is that you are deploying to a POSIX-compliant UNIX shell (sorry, no Windows), you are using Subversion for source control, and that all your passwords (i.e. production server and Subversion) are synchronized. Once again, following the Rails convention, some things that Capistrano assumes are overridable. Other things, however, are not. Some of the learning points I touch on below are directly related to Windows; others are not.

• You’re going to need the full Subversion binaries. If you, like me, had gotten by using various Subversion clients (e.g. TortiseSVN and Subclipse), the gig is up. You’re going to need Subversion anyway if you ever plan to run EdgeRails
• Some installation instructions for Capistrano will specify that you should --include the termios Gem when installing Capistrano. Normally, termios removes the need to display and manually enter your password during the execution of Capistrano tasks. However, since the termios Ruby Gem is simply a wrapper around the POSIX termios command, this won’t fly with Windows. Solution: don’t include a termios dependency and get used to entering your password each time you invoke Capistrano from Windows.
• If your Capistrano install fails with a Zlib::BufError, don’t fret it. Try updating your gems (gem update –-system). This seems to be a fairly common occurrence with Windows. I’ve heard of folks having to update gems multiple times for this to take.
• Another must for Capistrano deployment, and one that escapes folks who have spent life in the Windows world, is the need to chmod files so that they have the appropriate permissions set. This is especially true for the Ruby and FCGI dispatch files (if you’re using FCGI). Ideally, you should create your Rails projects on the UNIX box you plan on deploying to, check it into Subversion, and then begin work on your Windows development machine from there. This helps to avoid a host of issues such as chmod problems and bad shebang lines that routinely plague Windows users.
• Select a hosting provider that has one or more sample Capistrano deployment files available or that have customized the standard Shovel file for their environment. You’ll still have to do some tweaking but this will help save a good deal of time. Suffice it to say that if your hosting provider doesn’t know how Capistrano works, turn and run… fast.
• If you maintain critical files outside of Subversion such as your database.yml or if you have multiple copies of the same file (e.g. different environment.rb files for staging and production deployments), the simple Ruby put command goes a long way. For example:

put(File.read('config/database.yml'),"#{deploy_to}/current/config/database.yml", :mode => 0444)

put(File.read('config/environment.staging.rb'),"#{deploy_to}/current/config/environment.rb", :mode => 0664)

There are plenty of purists out there that have invented all sorts of ways to get unversioned files onto your productions server if need be. I don’t see the need for such complexity, especially if only one or two people have been granted deployment rights with Capistrano.

I feel as if someone tacked a “show me your enterprise service bus” sign onto my back and I’ve been walking around blissfully unaware of this fact for months now. Client presentations, vendor presentations, casual conversations – everyone wants to show off their visuals of an ESB, SOA, and next generation architectures. Thank goodness there’s no fine print on my sign restricting me from asking tough (and not so tough) questions.

• So how do I avoid vendor lock in?
• Do we really need SOAP?
• Grid computing… show me some client references!
• JSR 168 portals… yawn.

To escalate the situation, I came up with my own next generation architecture diagram and talked it through with a bunch of my peers. People liked it at first because of all the nice icons. They really loved it when the answer to any of the hard questions was “let me show you how this works, do you have a Web browser handy?” I’ve included the diagram below for your enjoyment and jotted down some quick write-ups with the obligatory links so that you can see, understand, and convince yourself of the reality of these tools.

• Ruby on Rails – Although there are tons of free services and a number of high quality paid services that can be leveraged to enhance applications on the Web, it’s hard to go very far without having some dedicated computing power. Using Ruby on Rails and MySql will get you the maximum bang for your buck (that’s no bucks for those who are counting). While you’re riding the Rails, make sure to take advantage of Ruby gems and Rails plugins.
• Web Service APIs – Lots of folks talk about enterprise applications that invoke common APIs to store documents, images, or access business services. For most, it’s talk of a far off and distant future. Would you like to see how this works today? Check out box.net, flickr, and salesforce.com for file, image, and business Web Service APIs in action.
• Yahoo Pipes – The minibus within the bus, Yahoo pipes provide a visual environment for aggregating, manipulating, and mashing up data and producing value-added output. Good mashup implementation but the interactive visual editor gets most of the attention – rightfully so. Imagine your business users mashing up business data to solve problems in new and creative ways that your analysts and developers never imagined.
• Google Base – A loosely organized, metadata-driven, data store available through Google. Data is accessible via an HTTP API with either Atom or JSON feeds.
• Open ID – Rather than supported a single point of control system for authentication, like Microsoft’s Passport, OpenID is a decentralized system that relies upon distributed identity stores and, for the most part, ownership of a particular URI. The system is lightweight yet still manages to provide for the distribution of basic profile information in addition to straight authentication. With more and more sites adopting this service, adoption is likely to steadily increase over the next several years.
• Amazon Web Services – Despite the lack of any hard-and-fast SLAs on their services, developers are increasingly leveraging the AWS platform for production applications. Their Elastic Compute Cloud (EC2) provides the average developer access to a significant grid computing array. Their Simple Storage Service (S3) and Simple Queue Service (SQS) provide access to globally distributed storage and messaging services, respectively. All of this is based upon a very fair “pay as you go” model that requires you to only pay for what you use and scale up and down without the usual provisioning and financial burdens.
• Sugar CRM – For one, I like Sugar’s tagline “Commercial Open Source”. A PHP-based CRM alternative to products like Siebel and Salesforce.com, Sugar is gaining pretty significant traction in the marketplace and is proving to be the first lucrative open source business application. The software has a good look and feel to it and their distribution options will likely set the standard for all other open source business software. You can opt for off-site dedicated hosting (on-demand), fully configured appliance-based distribution, or host-it-yourself (with or without a support contract).
• Bit Torrent – Peer-to-peer file sharing technologies have yet to find their place in the enterprise. On the open Internet, though, such technologies are said to account for as much as 40% of global Internet traffic. As desktop search technologies mature, sharing of decentralized data is going to be the best way to get at all of the knowledge otherwise hidden within the enterprise.
• Google Gears – If you don’t have the time or inclination to build offline clients to support your disconnected users, how about just making your Web app “disconnectable”? Gone are the cross platform, DLL, and distribution issues. Your Web app can sense when it’s lost network connectivity and go into disconnected mode. A great idea that will likely only gain limited traction in the enterprise.
• Netvibes Universal Widget API (UWA) – JSR 168 (or is it 286 now) compliant portlets seem so passé. With widget-based start pages becoming the norm and Windows and Apple both integrating widgets as integral parts of the future desktops, “write once, run anywhere” were just a matter of time.
• Microsoft Virtual Earth – I’ve blogged about this before and even did a quick Webcast. Take the hottest Web 2.0 visualization technique (AJAX maps), add birds-eye views in 2D and realistic 3D virtual earth renderings that run in-browser for both IE and Firefox and you’ve got Virtual Earth. It simply must be experienced to be believed.
• Simile Timeline– An equally interesting visualization technique but one that’s got significantly less press is the Simile Timeline AJAX widget for bringing time-based information to life.

With the 2007 NASCIO IT recognition award submission process closed and the evaluation process in full swing, I’m anxiously awaiting the publication of the nominations from across the country. It’s always interesting to see what new and innovative practices are being applied in different state governments. With Web 2.0, blogging, wikis, multi-media, and social computing firmly established in the Internet at large (see Time Person of the Year 2006), it’s high time that this wave hits the government sector, which usually lags behind in such trends by a couple of years.

I’ve been catching up on blogs the last couple of days and took in a couple of interesting sites. Dave Fletcher, who I believe is the CIO or CTO of Utah, has re-emerged with a vengeance in his blogging and is to thank directly or indirectly for much of this information.

•         Kansas’s new state portal features a MyKansas page with drag-and-drop type widgets similar to what you’d find on the Web 2.0 style portals like Netvibes. The portal has some other interesting features but still belays the shallow integration with other state sites that characterizes most state portals. As is to be expected, it is likely to be a multi-decade initiative to provide deep multi-channel integration across the different state government agency service offerings.

•         The federal Department of Health and Human Services (HHS) looks to have really done their blogging presence right on their Pandemic Flu Leadership Blog. Well laid out, with expert contributions and a wealth of comments (albeit moderated), the site is a great example of opening up a dialog with the experts in this area and making participation and information accessible to the public at large.

•         The State of Delaware is using VoiceXML to provide telephone-accessible services to citizens across different agencies (here’s that multi-channel thing again). This directly addresses the fact that e-government doesn’t flow exclusively through the Web browser. Citizens requiring state assistance services are less likely to have access to a high-speed Internet connection, less likely to be comfortable with these services, and more likely to have some degree of physical or cognitive impairments; all of which make Web-browser based applications less than ideal. VoiceXML solutions can leverage XML technologies in existing Web-based applications and provide access to citizens through a very important alternative channel.

•         I’d be remiss if I didn’t mention some of the trends in Dave’s own state, Utah. Dave has a bunch of information in his blog. I appreciate especially the Google search function as I’ve heard anecdotally, on a number of occasions, tales of state employees bypassing their own state portals and going to Google directly when they really want to find something within their states.

In Pennsylvania, as I believe in many other states, IT consolidation is the order of the day. The recently re-issued executive order from Governor Rendell has a specific section covering IT Consolidation and Services (Section G) and has strengthened the IT review and governance processes significantly from the previous revision. BSCoE’s implementation of the Logidex product, which I’ve blogged about previously, will provide some support in the area of reuse, consolidation and measurement and will be one of the NASCIO submissions that is publicized in the next couple of months.

Every IT generation has its seminal tome that transcends time and connects the dots in a way that no book had before it. For the object oriented generation in the 1980s, it was the Gang of Four (GoF) book. For the application architecture generation in the 1990s, it was Fowler’s book on patterns (PoEAA). “RESTful Web Services” will be, in my opinion, that book for the 2000s Web services generation.

There is something absolutely special about this book that readers of GoF or PoEAA will immediately recognize and appreciate. The book covers a breadth of technologies and ideas yet it helps the reader see how they all connect. It uses short code samples (in Ruby, the choice of this generation) to illustrate rather than obfuscate the ideas. Most importantly, it makes the complex comprehensible and delivers epiphany-like experiences throughout the book.

There are too many highlights in this book to enumerate in this review. However, some of the coverage that I appreciated most included:

• The chapters on resource-oriented design, since there was practically no written information available on this topic prior to this book
• The chapter on resource-oriented best practices
• An overview of the service building blocks, including the different representational formats and WADL, which I wasn’t aware of
• The chapter comparing and contrasting RESTful services with the “Big” (e.g. SOAP) service overhead that is common in most enterprise environments

I would have liked to see this book touch on simple POX versus true REST and handle the resource-oriented security concerns in a bit more detail but you can only ask so much of any one book. I’m fairly confident that “RESTful Web Services”, like the seminal tomes that have gone before it, will become assumed reading