Widgetbox Goes Hollywood

Grant Lee
Lead Engineer

As part of Widgetbox's new staff series—we'll be bringing you occasional guest posts from members of the Widgetbox team.

To be honest, when I was asked to write a guest blog post for our company blog, I was having a little more than a hard time coming up with a good topic to write about. Looking back over the past few guest blog posts written by my colleagues, a few words come to mind: useful, insightful, informative, timely, relevant. So as I turn on my laptop to bang out a few hundred or so words for a blog post, you can imagine one word that pops into my mind: "intimidated".

Well, maybe that's exaggerating. But I will tell you that it is not an exaggeration to say I have nothing but the highest respect for my fellow co-workers for their individual talents, knowledge, and skill in their respective fields.

But I digress, I still have not stated the topic on which I will write this blog post yet, have I? What shall I, a hapless software engineer for a web startup, contribute to the company's blog for all to read... I wonder? I imagine that I could bore you with the details about developing rich web applications that are optimized for mobile devices, or better yet, an essay on how to deliver a high-impact, highly-engaging software demo. But alas, such an endeavor taken on by Yours Truly could only result in an epic failure... an amateurish attempt at best that would be completely devoid of any comprehensibility and credibility.

So instead, I have chosen a fun and lighthearted topic that I feel could be entertaining for both my fellow Widgetboxers and our loyal readers: Widgetbox's Which Hollywood Star Would Play "You" in Your Movie Biography? *insert applause and cheers* For the purposes of this blog post, I will cast an actor/actress to play the role of a Widgetbox employee for his/her own movie biography. A fun game, don't you think? Now, to keep the game duration/length manageable, I will limit my selections to members of our Engineering team. Besides being the people I am most familiar with, doing so also has the added benefits of keeping my selection quality high and minimizing the potential of my offending the subject in question (I hope!). So let's begin shall we? Let's start with our beloved...

Co-founder/CTO: Giles Goodwin Played by: Russell Crowe So, one's born in England the other an Aussie...but that's pretty close right? In terms of likeness, I think you can't get any closer, especially when they both wear a beard. And I'm pretty sure that CTO's of web startups were the Roman generals of the olden days.
Database Architect: Ike Walker Played by: Corey Feldman I don't know why but I've always thought that Corey had a strange similarity to Ike. Not just in likeness, but both are clever, witty, and have a great sense of humor. Ike would've nailed the part of "Mouth" (Goonies) had he been cast in it during his teenage years.
Lead Operations Engineer: David Hoatson Played by: Played by: Roger Moore This is a perfect match in my mind. It's not hard to see the similarities with these two Brits, dashing, handsome, a regular ladies man with a twist of international intrigue. And I'm pretty sure that they both like their martinis shaken, not stirred.
Software Engineer: Brady Allchin Played by: Michael Cera Now this one was easy. When you need a pasty white guy who's geeky-looking and maybe a little awkward, you don't need to look much further than Michael Cera. I think Cera would play the role brilliantly, though he might need to swap tube socks for sandals.
Software Engineer: David Lemieux Played by: Dan Aykroyd If you take a much younger Dan Aykroyd, say circa the original Ghostbusters, and a slightly older David, I think you pretty much have winner here. If it weren't for the Internet, I'm sure Davey would have invented a high voltage laser containment system designed to store pesky apparitions.
Director of Engineering: Christopher Coco Played by: Wesley Snipes So I already know I'm gonna get the "racist" card played on me for this, but I'm gonna say it anyways. 3 roles: Willy Mays Hayes (Major League), John Cutter (Passenger 57), and Flipper Purify (Jungle Fever)...'nuff said.
Software Engineer: Jeff Remer Played by: Scott Caan I probably actually thought this when Jeff first started working at Widgetbox: "He looks a lot like that guy from Varsity Blues". I can picture both on a really fast bike or speeding down the sidelines for the winning touchdown catch.
Senior Software Engineer: Nelz Carpentier Played by: Bruce Willis I'd admit the likeness is not 100% there, but I think Bruce could definitely sell it. Give him some piercings and some tattoos (maybe even dye his beard blue) and you've got it. Both have that tough-as-nails type personality and would definitely choose samurai swords over chain saws. Runner up: Vin Diesel Because Vin already has tattoos.
Director, User Experience: George Penston Played by: Jim Belushi With a little makeup and some glasses, Jim could pull of a great George "Mumbles" Penston... he might have a Chicago accent, but given his talents could probably twist it into a Philadelphia one... or better yet, he could just mumble!
Software Engineer: Omar Megdadi Played by: Adam Goldberg Omar's a tough one, I'm not sure why, maybe it's his hair? The role of Pvt. Stanley Mellish (Saving Private Ryan) is what prompted me to select Goldberg, something about the scene he did with the Nazi near the end of the movie... Runner up: Jonathan Silverman Because I can see Omar getting into a situation like in Weekend At Bernies.
UI Developer: Andy Yang Played by: Val Kilmer Asians will always get the short end of the stick in this game because there just aren't enough non-kung-fu Asian male actors to go around. But I can say that because I'm Asian. That said though, Val (in his Top Gun years) would've played an Asian brilliantly. Val definitely has what it takes: smooth style, good looks, and a winning smile. And if Hollywood ever needed an Asian to play Jim Morrison, Andy would fit the bill nicely.
Special Bonus!
Marketing Associate: Jillian Jones Played by: Robin Givens Jill is not in Engineering, but since she does manage our blog I'm including her in the game! So you'll have to take a much younger Givens here (maybe somewhere between Head of the Class and Boomerang), but you know that whole blowing the eyelash away before it gets in your eye thing? Yeah... she can play Jill.

As for myself... I'll leave it up to you! Hope you've enjoyed the blog post, I certainly had fun writing it :)

Want to make a spiffy Slideshow like the one above? Make your own customized slideshow widget at Widgetbox

—When Grant isn't in the running for funniest guest blog post contributor, he can be found in Oakland practicing martial arts, going on runs, or picking up donuts for work

Click to read more ...

Get Your Twitter Content @Anywhere

Jeff Remer
Software Engineer

As part of Widgetbox's new staff series—we'll be bringing you occasional guest posts from members of the Widgetbox team.

Twitter has never been stingy with their data. With enough expertise, you can do just about anything you can do on Twitter.com off the site using with their extensive REST and Search APIs. Developers have implemented Twitter-based authentication using Twitter as an OAUTH provider, with some websites providing their own version of Twitter status timelines based on the kinds of information they want to display. The sheer number of Twitter apps, clients, and mashups is a testament to the wealth of data available through relatively simple API calls and REST URLs. Full or at least partial Twitter API implementations are not uncommon, however they are not very lightweight.

Until recently, some of the most desirable types of Twitter API integrations have also been the most difficult, with even the most modest implementations requiring a significant portion of code dedicated to authenticating API calls. Allowing Twitter users to authenticate and tweet off-site have typically required relatively involved API implementations and multi-legged OAUTH requests. While Javascript OAUTH client libraries do exist, they simply mirror unnecessarily verbose server-side implementations. This development approach doesn't align very well with the wealth of data and functions that Twitter has made open from the beginning.

Recently Twitter addressed this challenge and unveiled @anywhere, a simple yet powerful Javascript client library. This lightweight set of tools allows developers to easily integrate Twitter functionality such as authentication and inline tweets. Like Facebook Connect, Connect with Twitter is now available by simply registering a new @anywhere application and including a script tag and a few lines a Javascript. Developers previously had the ability to implement @anywhere to allow users to log into a website using their Twitter credentials, but it was not nearly as concise or flexible.

Once connected, a website implementing @anywhere can offer a Twitter user more functionality including the ability to follow other Twitter users and tweet from inline Tweet Boxes—all without having to go to Twitter.com. Even if a user on a website isn't registered on Twitter, or if they choose not to login with Twitter Connect, developers still get a couple of other benefits by adding @anywhere.

As the number of Twitter users grows, referring to someone's online personality with their Twitter username has become increasingly pervasive. Typically websites do their own linking of Twitter usernames either by hand or through some simple string manipulations. @anywhere provides a method to automatically link Twitter usernames in a given portion of web page automatically. The Twitter @anywhere developers took this feature one step further and introduced Hovercards. By hovering over a linked Twitter Username, a web page visitor can see a preview of a Twitter user's profile and even follow the user without leaving the web site.

Currently @anywhere consists of these few core features: Follow Buttons, Tweet Boxes, Linkify/Hovercards, and Connect with Twitter. However, by implementing Twitter authentication in a simple Javascript library, Twitter is opening up their API even further and could eventually allow for an extensive set of functions based on combinations of their REST API.

Twitter recently announced Blackbird Pie, a tool that allows you to embed static tweets in a web page. Blackbird Pie solves the problem of transcription. In the developer community I think we can expect more features like Blackbird Pie to make their way into @anywhere. Twitter understands that a crucial key to user expansion will be allowing for a programmatic way to embed tweets in a web page and allow users to interact with Twitter content off-site.

Sneak peak: http://blog.twitter.com/2010/03/anywhere.html
Announcement: http://blog.twitter.com/2010/04/its-alive.html
API Docs: http://dev.twitter.com/anywhere
Blackbird Pie: http://media.twitter.com/blackbird-pie/
Make a Twitter Widget at Widgetbox: http://www.widgetbox.com/make/twitter

—When he isn't biking competitively or on his way to be sponsored by Naked Juice, Jeff is a front-end software engineer here at Widgetbox

Click to read more ...

Tips for Getting Google App Engine to Operate in a Maven Environment

Nelz Carpentier
Senior Software Engineer

As part of Widgetbox's new staff series—we'll be bringing you occasional guest posts from members of the Widgetbox team.

At Widgetbox, I sometimes get to play around with interesting technologies that are outside of our regular stack. A couple of weeks ago, I was asked to use Google App Engine’s Java environment (GAE/J) to prototype a resizing image proxy.

At first, I just developed the prototype in the default GAE/J Eclipse environment until I could deliver a functional POC. After finding the GAE/J capabilities more than adequate for what we wanted to do, I was challenged to bring the project into our standard IntelliJ + Maven development environment. For the rest of this post, I’ll share a couple of tips and tricks for getting your GAE/J project to operate in this environment.

Basic POM File

There’s some funny business and frustration around the Maven community’s adoption of GAE/J, but I’ll skip that part of the story for right now. What I found is that the maven-gae-plugin project is the best place to go to for help Mavenizing a GAE/J build.

I have to say that it’s not ‘use the archetype’ easy (their archetype failed for me), but with a bit of elbow-grease and rummaging through their documentation I was able to get a decent and functional POM file built. Here it is (with some of our proprietary information scrubbed to protect innocent servers):

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001
/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <groupId>com.widgetbox</groupId>
   <artifactId>image-proxy-webapp</artifactId>
   <version>1.0-SNAPSHOT</version>

   <name>Widgetbox :: Image-Proxy :: Webapp</name>
   <packaging>war</packaging>
   <properties>

     <gae.version>1.3.0</gae.version>
     <gae.app.name>qa-image-proxy</gae.app.name>
   </properties>

   <dependencies>
     <dependency>
       <groupId>javax.jdo</groupId>

       <artifactId>jdo2-api</artifactId>
       <version>2.3-eb</version>
       <exclusions>

         <exclusion>
           <groupId>javax.transaction</groupId>
           <artifactId>transaction-api</artifactId>

         </exclusion>
       </exclusions>
     </dependency>
     <dependency>

       <groupId>javax.transaction</groupId>
       <artifactId>jta</artifactId>
       <version>1.1</version>

     </dependency>
     <dependency>
       <groupId>com.google.appengine.orm</groupId>

       <artifactId>datanucleus-appengine</artifactId>
       <version>1.0.4.1</version>
     </dependency>

     <dependency>
       <groupId>org.datanucleus</groupId>
       <artifactId>datanucleus-core</artifactId>

       <version>1.1.5</version>
       <exclusions>
         <exclusion>

           <groupId>javax.transaction</groupId>
           <artifactId>transaction-api</artifactId>
         </exclusion>

       </exclusions>
     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>

       <artifactId>datanucleus-jpa</artifactId>
       <version>1.1.5</version>
       <scope>runtime</scope>

     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>

       <artifactId>geronimo-jpa_3.0_spec</artifactId>
       <version>1.1.1</version>
       <scope>runtime</scope>

     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>

       <artifactId>appengine-api-1.0-sdk</artifactId>
       <version>${gae.version}</version>
     </dependency>

   </dependencies>
   <build>
     <plugins>
       <plugin>

         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-compiler-plugin</artifactId>
         <version>2.0.2</version>

         <configuration>
           <source>1.6</source>
           <target>1.6</target>

         </configuration>
       </plugin>
       <plugin>
         <groupId>net.kindleit</groupId>

         <artifactId>maven-gae-plugin</artifactId>
         <version>0.5.3</version>
       </plugin>

       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-war-plugin</artifactId>

         <version>2.1-beta-1</version>
         <configuration>
           <filters>

             <filter>${project.build.directory}/version.properties</filter>
           </filters>
           <webResources>

             <resource>
               <directory>src/main/external</directory>
               <targetPath>WEB-INF</targetPath>

               <filtering>true</filtering>
             </resource>
           </webResources>

         </configuration>
       </plugin>
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>

         <artifactId>maven-antrun-plugin</artifactId>
         <version>1.3</version>
         <executions>

           <execution>
             <phase>compile</phase>
             <configuration>
               <tasks>

                 <echo file="${project.build.directory}/version.properties">
                     friendlyversion=${project.version}
                 </echo>
                 <replace file="${project.build.directory}/version.properties"

token="." value="-"/>
                 <replace file="${project.build.directory}/version.properties" 

token="SNAPSHOT" value="snapshot"/>

               </tasks>
             </configuration>
             <goals>
               <goal>run</goal>

             </goals>
           </execution>
         </executions>
       </plugin>
     </plugins>

   </build>
   <repositories>
     <repository>
       <id>maven-gae-plugin-repo</id>

       <name>maven-gae-plugin repository</name>
       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
     </repository>
   </repositories>

   <pluginRepositories>
     <pluginRepository>
       <id>maven-gae-plugin-repo</id>
       <name>maven-gae-plugin repository</name>

       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
     </pluginRepository>
   </pluginRepositories>
 </project>

(FYI, we’re not actively using any datastore functionality just yet, so if you are going to use this template please forgive me if those dependencies are a little bit wonky.)

Since Google hasn’t (yet) decided to publish their development environment in a Maven-friendly way, there’s a bit of dependency wonkiness involved in getting the maven-gae-plugin to work. I included the repository information required by the plugin (lines 121 – 134), but if you use a repository manager (like Nexus), you’ll want to remove those lines from the POM and add a proxy for the maven-gae-plugin’s repository.

To get the development environment working the plugin also requires access to the unzipped SDK as packaged by Google. The plugin tries to help you set this up (“gae:unpack”) but that failed for me. I was able to get stuff working by manually unzipping the SDK artifact downloaded directly from Google to the following directory:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0

Incremental Improvments

Initially, I had kept the appengine-web.xml within the WEB-INF directory, but I realized I could make our Release Manager’s life a bit easier if I added a bit of build-time substitution.

Here’s our appengine-web.xml:

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
	<application>${gae.app.name}</application>

	<version>${friendlyversion}</version>
	<system-properties>
		<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
	</system-properties>
</appengine-web-app>

And you’ll see that I put it into a new source directory called ‘external’:

At build time, I use the AntRun plugin (lines 97-118) to create a small file under the target directory that holds a ’sanitized’ version of the standard Maven version. (I.e. “1.0-SNAPSHOT” becomes GAE-friendly “1-0-snapshot”.) I then use the Maven filter functionality available in the WAR plugin (lines 80-96) to copy the appengine-web.xml into its proper directory with the version substituted in.

You’ll also notice in our appengine-web.xml that we substitute in our application name. By default this comes from the properties section of the pom.xml file (line 11). I did this because we’ve actually got 2 different applications up on GAE’s servers, the QA version and the Production version. By default we build using the QA server’s application name, but when our Release Manager is building to upload to Production, all that is needed is an additional command-line argument of “-Dgae.app.name=<prod-name>”.

Running, Debugging, and Deploying

The two most valuable targets that maven-gae-plugin provide are “gae:run” and “gae:debug”. These will assemble your code in the standard Maven webapp target directories and run your app. (Note: “gae:debug” didn’t actually work for me until the 0.5.3 version of the plugin.)

There is also a “gae:deploy” target that is supposed to invoke the Google-supplied shell script that will upload your application to the Google servers, but it failed for me several time. Since then, I’ve defaulted to using the shell script directly to deploy my app once it has been built:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0/bin/appcfg.sh \
    update \
    ./target/myApp-1.0-SNAPSHOT

Results

So, this is how we got up and running with GAE/J in our standard development environment. Hopefully this post ends up helping people out to reduce their bootstrap time when evaluating/investigating GAE/J for their own uses.

—When he isn't busy with Santa Con, No Pants Day, and a whole host of other cool SF events, Nelz is Widgetbox's Senior Software Engineer. To read more topics on Nelz's personal blog visit Nelz.net.

Click to read more ...

Expanding In the Cloud

David Hoatson
Operations Lead Engineer

As part of Widgetbox's new staff series—we'll be bringing you occasional guest posts from members of the Widgetbox team.

At Widgetbox, we serve nearly a billion impressions a month of widgets installed into millions of sites on the web. This creates some unique challenges for us running our operations. To meet these challenges, we use both "traditional" managed hosting and hosting in "the cloud". Widgetbox still relies heavily on our managed service provider, Contegix, and their world-class 24 x 7 support remains a key part of our business. But for cost and redundancy reasons it was imperative for us to expand our service partnerships into the Cloud. We have investigated multiple cloud vendors and used services from Amazon, Google and Rackspace. But most of our success in the cloud to date has been with Amazon. We serve tens of millions of requests today backed by servers in Amazon taking advantage of many of their services. (Contegix has a cloud service too, but we didn't want to be tied to one vendor.)

There were two important Amazon products announcements that were key to Widgetbox's expansion in the Cloud. Today they remain two of the main features that differentiate Amazon's services from the rest of the competition.

The first, in October 2008, was the Elastic Block Store ("EBS"), or their 'RAID in the sky' as I first saw it. From when we launched in 2007, we'd always been looking to get real-time replication of our data outside of our primary data center. We'd tested a few of the Cloud offerings, but the local drive performance was just not good enough for a relational database of the size and complexity we needed. So, the arrival of EBS with its increased I/O and throughput performance meant we could finally do redundant, real-time replication streams of our data inexpensively in the Cloud, and with no initial capital expenditure or commitment. Our business reporting and analytics are built from this Cloud version of our data without compromising our core user base. This also gives us the capability to fail over to this data set in the event of a outage to our primary data center. Finally, the ability to create very inexpensive, point-in-time snapshots from these EBS volumes has freed us from most of the frustrations of traditional relational database backups.

When Amazon finally announced Elastic Load Balancing ("ELB"), after months of anticipation, in May, 2008, we took our second, next big step into the Cloud. While the development community, and smaller sites, were content up to this point to use software solutions like HA Proxy (or even round robin DNS!), we felt we needed the redundancy and reliability that ELB offered. The product is built around the same infrastructure used by the retailing giant itself, so we were more than ready to jump on board. We rolled out the first of our applications using ELB-backed EC2 instances the following month, and have been slowly expanding since then. The virtual nature of ELB, having distributed between Amazon Availability Zones (e.g. data centers) and not reliant on a single piece of hardware, helps our operations team sleep at night.

Supporting Widgetbox's growth would have been almost unthinkable 3 years ago without the flexibility and cost-savings that Amazon's Cloud has given us. We started out using very simple services but have expanded to use Data, RAID, Load Balancing, NoSQL, and many more. Widgetbox will continue to grow into the cloud both at Amazon and with other vendors. The time and cost of scale are greatly reduced and we're able to experiment with new deployment topologies and environments faster than we ever could with co-location or managed hosting.

—When he isn't being boxing clever, running 1/2 marathons, and impressing Americans with his British accent, David is Widgetbox's Worldwide Operations Lead.

Click to read more ...

Our Mobile Site Builder Gets Even Better With Two New Features

Last month we were excited to introduce our new Mobile Site feature, and this month we are pleased to announce two brand new feature enhancements for your Mobile Site—Facebook Fan Pages and new sharing capabilities.

Facebook
In addition to Twitter, blog feeds, Google Maps, and more, you can now add a Facebook Fan Page to your mobile site. Simply enter your Facebook Fan Page URL and your content will be automatically imported. Best of all your mobile site content will dynamically reflect any new updates to your Fan Page.

Share to Twitter, Facebook, Email, and Text Message
Starting this month, users can easily share your mobile site with others via Twitter, Facebook, email, or even text message! The new share feature allows your mobile site to go viral as people share your site with others. As a Publisher Pro or Business Pro subscriber you can get even more viral exposure by customizing the About section of your site.

Make a mobile site and start using the new features »

Click to read more ...