Thursday, July 18, 2013

Worry Less. Do More. Be Fearless!

"we better have health insurance"

. . .

"peyton got hurt"

These are not the text messages you want to receive from your wife on Mother's Day, when you are four hours away from home.

Our daughter Peyton is four years old. She has a twin brother. They scare me daily. I assumed she broke her arm on the swings, her wrist on her bike or something like that. Nothing to worry too much about.

About a month earlier I quit my awesome job at Disney Interactive making mobile games to dive in and cofound a startup, RealCrowd. A long time friend and cofounder Andy was at my parents' house with me, and we were on our way to the bay area to meet with our other cofounders and live under one roof. We had been interviewed and accepted into the YCombinator (YC) Summer 2013 batch and decided to get a jump start by moving to silicon valley a few weeks before the program was scheduled to officially start.

I've been working from home the last few years after the last startup (Hive7) was acquired and consequently we are a very close family. The twins are a lot of work, and I try to support my wife Erika by being as flexible with my time as possible. I was already feeling the intense guilt of moving four hours away to Mountain View and relegating my family to second priority for a few months.

"Is this startup going to be worth it? Of course this happened the day I'm supposed to go to the bay area. Am I being too selfish?" These questions haunted me.

. . . and then the texts got worse.

"she fell out of the window"

A few weeks before this fateful Mother's Day we had moved into an affordable three story condominium in Stateline, Nevada (South Lake Tahoe area) with a great lake and mountain view, cutting our monthly rent in half and saving a bit on income taxes (there are no state income taxes in Nevada). I was cutting costs to help decrease my burn. Andy had planned to call that his second home while we hacked on RealCrowd, prior to being accepted into YC.

I visualized every window of our three story home, floor by floor. Adrenaline started pumping. My heart rate increased. I had to consciously suppress the panic building inside. I knew the worst possible scenario had transpired. She had fallen through the screen from the third story window in the dining room, onto the asphalt driveway below. I almost vomited. At this point I was oblivious to conversation people were trying to have with me.

I immediately tried to call my wife. No answer. Again. No answer. Again. No answer. Then it dawned on me she was probably on the phone with 911, and that's why she had been texting.

"can she move? is she conscious?" I replied. "keep her still. it'll be ok. i'm coming. love you." I added.


That was the most amazing "yes" I had ever received. It trumped our marriage proposal, every deal close, every VC commitment, every new customer.

I looked up from my phone and told my parents I had to leave, Peyton had fallen out of a window at home. I jumped in the driver's seat of Andy's car and dragged him along with me to the hospital. He tried to make conversation to get my mind off of analyzing all the possible outcomes. He's a great friend. Due to the nature of the event, Peyton was life flighted to a hospital in Reno, NV. Erika had to find a ride as she couldn't go in the helicopter. It took her about an hour, and me about three hours to get to Reno.

Peyton had suffered multiple cervical spine injuries. Her neck was broken!

Those first few days after the event are a blur in my memory. Full of extreme sadness, happiness, worry, guilt, lack of sleep, and everything in-between. Would she be paralyzed? Would she live? I researched statistics on these things. The stress of not knowing ate me alive. Peyton had a surgery to align her cervical spine and had a halo traction device applied. She woke up from anesthesia and the nurse did some tests.

Peyton could wiggle her fingers and toes and apply normal amounts of pressure!

She walked a few steps the next day. And then a few more. But we lived in a three story condo. There is no way Peyton could navigate the stairs. My sister took the initiative and managed a crew of our amazing friends and family, including Andy, to move us out of the home we had moved into just a few weeks earlier and into a storage unit. We'd be taking our clothes to my in-laws single story house in the Sacramento area, and my family would move there for the treatment period, near to a cervical spine specialist at UC Davis Medical Center.

For the next week I was not sure if I'd be co-founding this company after all. Peyton would wake up in the middle of the night in extreme anger pulling at the bars and screaming to take the halo off of her. She was frustrated by her lack of mobility. She didn't want to try to walk. She was depressed and angry most of the time. I couldn't leave her.

But, thankfully, things got better. By the end of the first week she had adapted to her new mobility limitations, daily cleaning of the screws in her head (ouch), and started asking for help when she needed it. She stopped waking up in the middle of the night every night. She would walk wherever she wanted to go. We took her to Chuck E Cheese's.

At the end of week two I knew Peyton would be able to deal with the halo, and my amazing wife encouraged me to not pass up the great opportunity that is RealCrowd and YC.

Peyton is still wearing the halo, and will be for a total of 12 weeks. I see her on weekends. My little girl is alive. But life is not normal. She has a very serious risk of infection (in fact she's on antibiotics now for one), a risk of losing traction, a risk of her skull being punctured by the screws. She can't get her vest wet. She can't run and jump and tumble. She has a twin brother that gets to do all these things. She gets frustrated every day. At the end of dealing with all of that we'll know if she healed or if we have to do more invasive surgery to fuse some bits together. I worry about her constantly.

Will anyone use our product? Can we close that deal? Will our business keep scaling? Will we meet our goals? Can we hire that person? What if ... ? These questions no longer keep me awake at night. I do not fear the outcome. I keep pressing on. We're building a business, not dodging death. Perspective is everything.

Worry Less. Do More. Be Fearless!

Oh, and yes, we did have health insurance.

Public service announcement: If you have a window that's near the floor, make sure nobody can pass through the screen. Many state/local building codes say openings on the 2nd story or higher that are less than 24" off the floor must not permit a 3" sphere from passing through (aka, you need bars on the windows or a way to restrict them from opening). Ironically this can conflict with fire codes, so, your mileage may vary. This applies to deck and stairway railings as well. If you are a landlord, check out your rental properties and make sure they're compliant too.

Monday, August 1, 2011

Install node.js in 10 Seconds or Less

I've been playing around a bit with node recently. One thing it's quite good at is quickly bringing up a developer workstation. You can just run a quick command line to have the server running rather than futzing with config files, virtual directories, software installations, and such. When your team uses different development platforms (Linux, Mac, Windows) or your development environment is different from production, a bit of scripting makes it much easier to insure everyone is on a level playing field.

I found myself having to remember too many commands when setting up node for my application on two different OSX laptops. I wanted to be able to quickly get node up and running for a non-privileged user account and with node modules installed locally for the application. This is ideal when you're working with multiple projects that may use different versions of node or different versions of modules.

The key ingredient is the great version management tool called nvm (Node Version Manager). It lets you easily download, compile, and associate a version of node with a particular terminal session. Combined with npm (Node Package Manager), making a setup script for an application is nice-n-easy. I whipped up this little script for a recent prototype project. Copy and modify as desired.


#A script to setup the node environment using nvm and npm.
#This is intended to be run on developer workstations. 

#Check for well known prereqs that might be missing
hash git 2>&- || { echo >&2 "I require 'git'."; exit 1; }
hash make 2>&- || { echo >&2 "I require 'make'."; exit 1; }
hash python 2>&- || { echo >&2 "I require 'python'."; exit 1; }

#Clean up old stuff.
rm -rf ~/.nvm
rm -rf node_modules

#Download, source, and update nvm's package list
git clone git:// ~/.nvm
. ~/.nvm/
nvm sync

#Install latest stable version of node
#Change 'stable' to a version number to install a specific node
#See `nvm help` for more info
nvm install stable
nvm use stable

#Install all the modules we desire
node_modules="mime qs hashlib connect ejs express hiredis policyfile redis uglify-js multi-node generic-pool"
for m in $node_modules;
  do npm install $m;

  1. Copy and paste this script into an file
  2. chmod a+x
  3. ./

Make sure to run the script in the directory where you want the node_modules to be installed. That's it! Happy node hacking.

Also, this script is... destructive. If you don't want to download and rebuild node every time you might want to remove the `rm -rf ~/.nvm`.

Thursday, July 28, 2011

How to Recruit Great Engineering Talent

Raiders of the Last Nerd made the front page of Hacker News yesterday and went a bit viral around the geek ecosystem. For good reason. Fueled by accelerating technology innovation, VC money, and even sometimes profit, the job market for experienced engineering talent has been heating up all over the place. In recent months I have received an email or phone call from at least one recruiter per day and I haven't even worked for Google or Facebook or one of those other Golden Names. Last month, out of sheer frustration with the lack of quality, I wrote an open letter to recruiters titled Dearest Recruiter. I'd like to expand on that a bit.

The recruiting industry is broken. I'm not talking about in-company recruiters here, but those outside agencies like the one Mr. Carvajal runs. There is a horde of non-technical, outgoing, sales people trying to court highly analytical, mildly autistic, geeks. After fielding calls and emails from recruiters for the last 12 years I've grown a pretty thick skin and have become very defensive. When I speak to a recruiter I assume everything they say is an attempted manipulation. I know that both myself and the company for which they are recruiting are getting ripped off. I will often just hang up on them. They remind me of the slime I had to talk to every night during dinner before the Do Not Call Registry stopped most cold call telemarketers in their tracks.

But on the other side of the coin, many geeks don't know their value, or don't know how to assert it, and the recruiters take advantage of that. Let's say you are recruited through one of the bigger tech recruiting firms such as TEKSystems or RHI. While you're on contract they'll probably take at least a 50% cut. Expect it to be much more if you're inexperienced and don't negotiate.

When I started out in the industry I was much less jaded. In 1999, after dropping out of my second semester of college, I was referred by a family friend and took my first programming job at a ski boat manufacturing company. School was way behind the curve on technology and utterly boring for me. Like most geeks in my generation, I'd been messing with computers since I was five years old and programming somewhere shortly thereafter (it's a fuzzy memory now). I grew up in a small agricultural town and had no idea what wages should be or how to find out. This job paid a whopping $8/hr when minimum wage was $4.25. By 2001 I was making a stellar $13/hr. My friends though I was rich as I was living above the poverty line. But I got bored at that job so I found this cool web site called where I could post my resume. This was my first experience with recruiters.

They were all very nice people, these recruiters. They saw some fresh meat in me. A sucker. They pitched me to their client as this awesome young rock star. I was paid $35/hr (~$70k/yr). I later found out the company was billed more than 2x that amount. Was I really worth more than double that? And why didn't they tell me? My next position was also through a (different) contracting company. I was paid $45/hr (~$90k/yr) having 3 years of professional experience. I found out well after the fact that they billed about $100/hr. That really got me thinking about recruiters.

If you're a company and have a contractual relationship with a recruiting firm for direct hires, they'll probably take something like a 25% cut of the first year's salary of whoever they refer to your company. Of course it's in the contract. We had one of these at Hive7. On top of that, as a manager, I've only had mild success with talent acquired through recruiting firms. The best hires I've made have always been through my own network via referrals. Ouch!

Transparency is recommended. Us geeks are information and knowledge addicts. We learn and digest everything we possibly can. We embrace transparency. Just witness the popularity of the Open Source Software movement and how much value we place on being a part of it.

Look. We know how the recruiting industry works. This antiquated people-trade reminds me of another slimy role that is currently being blown to pieces. If you tried to buy a new car 10 years ago you'd have to go to a dealership, haggle with someone, get ripped off anyway, and walk away feeling dirty. Nowadays when I buy a car I do so online. I send emails to as many dealerships as I would drive to and ask for quotes. They provide quotes, and many provide a copy of the actual invoice they received when purchasing the car from the manufacturer. Of course they also receive some kick backs on the back end for volume and other special programs, but you can still walk away feeling like you weren't completely ripped off. The salesperson makes a hundred bucks for a few minutes work and you get a car.

I'm sure we'll see this level of transparency and marginalization of recruiters in the next 10 years. Just like the car salesman, technical recruiters are becoming largely irrelevant. Online social tools like Facebook, LinkedIn, Twitter, Google+ are expanding the available talent network one has access to. Up-and-coming geek-oriented job sites like Stack Overflow Careers are putting the tools into the hands of the hiring manager or HR department. All that's missing is a good aggregator of all these that can send me positions that would actually be interesting to me.

I don't know much about the tech scene in NYC, but I have been contacted by recruiters there. They haven't had any more to offer than people over here in the San Francisco bay area. They're all the same slimy salespeople trying to convince you of their golden opportunity. I was going to use a used car salesman analogy here again, but I think something more fitting would be Viagra spammers. It is apparent that recruiters blindly send emails to hundreds of people hoping there will be some sort of a response.

Most of the job reqs that come my way via recruiters are for Developer Lead or Senior Engineer type positions. Salaries are usually in the $100k-$150k range. There is the occasional Director level position, but salaries are roughly the same. After reading an article like Raiders of the Last Nerd, you'd think that we were back in the .com boom with companies throwing sports cars and huge signing bonuses around. But that's just not true. The cases are much more isolated.

In the interview Dave Carvajal stated that “We came from a place over the last two years where people were going to start-ups for below market [rates], People aren’t necessarily going to do that now.” This is partly true. But it really depends. It shows some of the lack-of-understanding of how the geek mind works. Maybe in the financial sector it's all about the money, but to a great hacker it's more about the project.

The best engineers out there do not work for the money. Sure, they'll calculate how much their upside is at various acquisition prices, but they don't really care. Things like solving interesting problems, using new technology, making a visible impact, and working with a fun team are much more compelling for those that would work for a startup.

For startups it's about the equity. We're in the middle of a startup boom. Services like AngelList are making capital more easily accessible to good entrepreneurs, and incubators like YCombinator are teaching young entrepreneurs the ropes. These startups are willing to offer large amounts of equity to early employees in exchange for sub-market pay rates. Any decent founder out there deeply believes her company is going to succeed and thus believes her equity is worth significantly more than any amount of cash. That, and, they don't have much cash to throw around.  There are a lot of very small funding rounds of < $1MM happening. That doesn't give a company without significant revenue much buying power if they want that to last a year.

Do you want to be successful in recruiting someone? Try this on for size.

Show me MY money. On first contact, tell me how much money I will make. The exact dollar figure. If  you would be willing to pay $300k/yr for a role for the right person, just put that amount in the job. Us engineers know that there is a very high variance between the least productive and most productive members of our kind. Why not try paying for that?

Tell me about the project. The project is just as important as the cash, if not more so. Tell me what it is. Don't tell me in weird abstract terms you don't understand. If you can't tell me what it is, don't bother talking to me.

Show me YOUR money. As a recruiter, how much money are you going to make on the deal? In our heads we're already doing the math and assume you're ripping us off. You might as well tell us, and break it down by the hour.

Don't ask how much I make. We know that asking how much a person makes right now is really just gathering negotiation leverage. It's also wasting our time. You'll take that, then go back to the company, then get a new number, and blah blah blah.

Tell me the company you're recruiting for. The abstract job req is useless. Just say the name of the company. I will want to research them. It's more about the company fit than the particular job they're hiring for. Almost never do people do the same thing they were hired for in a company.

Let me talk. Many recruiters all-too-often steam roll conversations. This is super annoying. That's not how you sell.

Don't call me. Oh yeah, and avoid calling me at all costs. Email or LinkedIn or text or something is much preferred. Phones are horrible. I hate talking on them. I'm not alone.

Don't pretend you are technical. You bring up some recent tech news or talk about how you used to make web sites for fun or how awesome "that C language" is. We know you are just manipulating us to try to get us to open up to you. Give it up. Admit you're just a Pimp trying to pick up a new Ho.

Have something compelling. When you're recruiting experienced talent out of a comfortable position you had better have something great. Either that's a truckload of cash (think multipliers on current salary), a first-employee position, a responsibility level-up, or a really freaking cool project. A combination of this stuff is preferred. Mention it upfront and don't dance around the important facts. We do not want to interview without knowing this stuff.

It's a strange world we are living in right now. Jobless rates are insanely high, and here I am complaining about too many people wanting me to interview for jobs. It makes me feel extremely lucky to be in this industry.

Monday, June 27, 2011

Dearest Recruiter

Update:I expanded on this a bit over here:

I'm taking some time out of my lunch break today, when I'd much rather be indulging in some geek porn over at Hacker News, to submit a plea on behalf of all the in-demand geeks out there.

You probably found my resume, my consulting company, my blog, my Linked In, my Facebook profile, the MySpace profile I haven't taken the time to delete, my Stack Exchange profile, and some personal blogs about my kids via Google. Did you actually read any of it? No? I am an entrepreneurial generalist that prefers to balance business, product, and technology. I do not want a heads-down, order-following, engineering role.

Yes, I joke about golden handcuffs, but I am happy in my current position. I work from home for the most part. I am working on autonomous startup-like projects that provide great satisfaction. I have some cool benefits and awesome perks. Oh yeah, and I'm paid well.

But, I digress. I do enjoy money. I am human, and a good American consumer. I like to buy stuff. My wife likes to buy stuff. We like vacationing. It's fun. So I'll make you an offer. I will consider your position if you can provide compensation like Sergey Aleynikov. Let's round it up to $500k/yr total compensation, and the work better be self rewarding as well. Yes, I'm serious.

Do I feel bad for asking for such a huge sum? No, I don't. Am I entitled to it? Probably not. But I'm happy in my position and a few thousand dollars extra will not change my mind. I am not interested in a market-competitive salary. I am not interested in being the 25th employee at the next great Groupon clone. I understand how company capitalization and ESOP's work.

Now, I might be interested in being the first employee at a startup or a technical cofounder, but it had better be really darn interesting. I've got my own ideas and stack of half finished prototypes to productize. :) On that note, consulting arrangements at > $200/hr will also be considered.

Stop contacting me with inappropriate jobs that have nothing to do with my experience. Stop attempting to relocate me to Indiana. Stop asking what it would take for me to move to a new position. Stop wasting my time. Please. Stop.

Best Regards,

The In-Demand Geek

Monday, April 19, 2010

ValidateInput Attribute Doesn't Work in ASP.NET 4.0

Today I decided to upgrade some of our new projects (top secret, shhh) to Visual Studio 2010, ASP.NET 4.0, and ASP.NET MVC 2.0. There are about a million new features that look quite useful in all of these new releases. We have some fairly complex projects so I was excepting a few speed bumps, but, not this one.

It seems with every new release Microsoft adds annoying (isn't security always?) features to protect us from ourselves. Way back when they added the idea of Request Validation. If ASP.NET thinks a user is posting something "bad" to the server (i.e. things that lead to XSS attacks), the request is denied. This is cool except when you want to, say, allow the user to input HTML or have a web service that takes XML as a parameter in a form, or use a ":" in your URL. In ASP.NET web forms you work around this feature by turning it off at the page level or globally in your web.config through the validateRequest option. In MVC you use the ValidateInput attribute on your action.

This is really really important:

In .NET 2.0-3.5 the runtime only validated requests sent to .aspx pages, but that has now changed and any request will be validated, even it is sent to a custom handler, or an MVC application.

ASP.NET MVC implements its own request validation, which is also on by default. To turn it off you simply slap a ValidateInput(false) attribute on your controller action. This is fine and dandy, except with the latest ASP.NET 4.0 changes, it no longer works and an exception is thrown. So you might see an error like:

A potentially dangerous Request.Form value was detected from the client or A potentially dangerous Request.Path value was detected from the client

The workaround is pretty easy. Just follow the instructions on the ASP.NET 4.0 Breaking Changes page. Stick this XML in your web.config to revert to the behavior as it were in ASP.NET 2.0-3.5. This will put request validation back in the hands of the ASP.NET MVC engine and your ValidateInput attribute will start working again.

Tuesday, March 9, 2010

Mr. Sprite Sheet, Meet Ms. MovieClip

In Youtopia we wanted to have animations. We also wanted to have thousands of buildings on the screen at once. Anyone who has done a lot of Flash development will tell you that these two things are not compatible. You simply cannot create that many movie clip instances and have them playing. But, all is not lost! With a sprite sheet animation system like the one in PushButton Engine (PBE) you can have your cake and eat it too!

Sprite sheets are as old as video games that have pre-rendered art. The basic idea is you draw a bunch of frames of an animation (or multiple animations) and put them on a single image. You can see some examples online. The software knows how to read and draw a frame of the desired animation from that one big graphic, and that's what you see on the screen. Easy!

However, sprite sheets also have their down sides. First, they aren't made to scale. We're talking bitmap graphics here. Your resized image is only going to look as good as your resizing code can make it look (which usually is not very good). Second, animations with a lot of frames can create very large file sizes. A five second animation at 30 frames per second means you end up with all 150 frames of the animation on a single image.

This is where movie clips come in. "But wait," you say with bewilderment, "didn't you tell me earlier we couldn't use movie clips." Why yes I did, so let me explain. Flash was designed from its inception to create small file sizes for downloads. It was also designed to support resizing. This is done using vector graphics which can be animated in a movie clip. We should take advantage of this feature.

One of the bits of code I contributed to PBE was the SWFSpriteSheetComponent. It takes an instance of a MovieClip and converts each frame into a bitmap. It then exposes these bitmaps to the PBE rendering engine as if they were part of a sprite sheet. Viola! You get the best of both worlds. A small download size, the option to render your animation at any scale, and super awesome performance. Using this technique and well drawn vector graphics we were able to save over 5X on the download size of Youtopia as compared to sprite sheets and get the same performance!

NOTE: If you're not familiar with PBE you might want to skim through the docs before diving into my code below. I'd highly recommend the video talks. The composite entity architecture can throw you for a loop if you're not used to seeing it.

Using the SWFSpriteSheetComponent is really easy and I've created an example application to show it off. The application spawns a sleeping Ben Garney every second and makes him move. There are some animated zzz's to indicate that, regardless of the smile, he is in fact sleeping. Ben is the lead developer on PBE, so he's getting picked on. ;) Click here to see the app. You can also download the source and follow along at home.

The process starts by creating and exporting a MovieClip in your fla with a flash.display.MovieClip base class. Give it a class name that you're not going to forget. In this example we call it "z_fx". Then publish the swf and put it somewhere that your PBE application can find it. The example has a "res" folder where I put the effects.swf. The CS3 format effects.fla (created for me by Jesse Tudela here at Hive7) is in the res folder in the download if you want to check it out. Now on to the code!

For easy deployment we embed the effects.swf and the garney.png file using PBE's ResourceBundle like so:

import com.pblabs.engine.resource.ResourceBundle;

public class Resources extends ResourceBundle
public static const EFFECTS_PATH:String = "../res/effects.swf";
public static const GARNEY_PATH:String = "../res/garney.png";

public var effects:Class;

public var garney:Class;

Now on to our "main" method. We start off by creating a SceneView, which is the target where PBE will draw stuff. Then we call startup, load our embedded resources, and create the scene. This is all standard PBE initialization. A ThinkingComponent is added to the scene entity in order to spawn Garney instances based on the game's virtual time. And finally, we register our garney entity factory with the TemplateManager and spawn a Garney!

// The SceneView is where PBE will draw to
var sv:SceneView = new SceneView();

// Start the logger, processmanager, etc

// Embed my resources
PBE.addResources(new Resources());

// Create a basic scene through code
var scene:IEntity = PBE.initializeScene(sv);

// ThinkingComponent is an efficient Timer based on virtualTime rather than real time
scene.addComponent(new ThinkingComponent(), "spawnThinker");

// Register the callback for my "garney" template that will be used to instantiate garneys

// Spawn a garney then kick off the timer. They keep coming, eeek!

The setupTemplate method is where all the interesting stuff happens. In here we choose all the components that make up our entity and determine how they relate to each other.

private function setupTemplate():void
var e:IEntity = PBE.allocateEntity();

// Spatial component knows where to put the garney
var spatial:SimpleSpatialComponent = new SimpleSpatialComponent();
spatial.spatialManager = PBE.spatialManager;

// Rendering component knows how to draw the garney
var render:SpriteRenderer = new SpriteRenderer();
render.fileName = Resources.GARNEY_PATH;
render.positionProperty = new PropertyReference("@spatial.position");
render.scene = PBE.scene;

// Here's the SWFSpriteSheet magic!
var fxSheet:SWFSpriteSheetComponent = new SWFSpriteSheetComponent();
fxSheet.swf = PBE.resourceManager.load(Resources.EFFECTS_PATH, SWFResource) as SWFResource;
fxSheet.clipName = "z_fx";

// Fx Rendering component knows how to draw the z's from the spritesheet
var fxRender:SpriteSheetRenderer = new SpriteSheetRenderer();
fxRender.positionProperty = new PropertyReference("@spatial.position");
fxRender.positionOffset = new Point(30, 10);
fxRender.scene = PBE.scene;

// Need an animation controller to assign and animate the sprite sheet on the renderer
var animator:AnimationController = new AnimationController();
animator.spriteSheetReference = new PropertyReference("@fxRender.spriteSheet");
animator.currentFrameReference = new PropertyReference("@fxRender.spriteIndex");

var idle:AnimationControllerInfo = new AnimationControllerInfo();
idle.loop = true;
idle.spriteSheet = fxSheet;
idle.frameRate = 30; // In PBE your animation framerate can be independent of your stage framerate

animator.animations["idle"] = idle;
animator.defaultAnimation = "idle";

// Garneys self destruct after a random amount of time
var suicide:ThinkingComponent = new ThinkingComponent();

// Add all the components to the entity
e.addComponent(spatial, "spatial");
e.addComponent(render, "render");
e.addComponent(fxSheet, "fxSheet");
e.addComponent(fxRender, "fxRender");
e.addComponent(animator, "animator");
e.addComponent(suicide, "suicide");

return e;

The "garney" template is made up of six distinct components. Each of these components perfmorm a small piece of highly specialized work. I created two components for rendering, one for the garney sprite and one for the animated z's. The z's use a SpriteSheetRenderer and a SWFSpriteSheetComponent. Both renderers are positioned based on the position property on the spatial component. The AnimationController is a very powerful class that lets you do things like automatically change out the animation being rendered based on an event firing. But, that's probably a post for another day. In this case it just plays the z's animation on the fxRender component.

All of this gets tied together in the spawn method, which creates a new garney entity based on the template, assigns it some random values for position and velocity, makes sure it draws the most recently spawned entity on top, picks a random time for the entity to commit suicide, and schedules the next spawn.

private function spawn():void
// Create a garney!
var garney:IEntity = PBE.templateManager.instantiateEntity("garney");

// Randomly position a garney!
var spatial:SimpleSpatialComponent = garney.lookupComponentByName("spatial") as SimpleSpatialComponent;
spatial.position = new Point(Math.random() * 800, Math.random() * 600);
spatial.velocity = new Point(Math.random() * 50 * (Math.random() < .5 ? -1 : 1), Math.random() * 50 * (Math.random() < .5 ? -1 : 1));

// Choose when this garney commits suicide, up to 20,000 virtual MS from now
var suicide:ThinkingComponent = garney.lookupComponentByName("suicide") as ThinkingComponent;
suicide.think(garney.destroy, Math.random() * 20000);

// Set the zIndex so our components render consistently in spawn-order
var render:DisplayObjectRenderer = garney.lookupComponentByName("render") as DisplayObjectRenderer;
render.zIndex = _zIndex;

var fxRender:DisplayObjectRenderer = garney.lookupComponentByName("fxRender") as DisplayObjectRenderer;
fxRender.zIndex = _zIndex;

// Grab the global spawn thinking component and schedule a think
var thinker:ThinkingComponent = PBE.lookupComponentByName("SceneDB", "spawnThinker") as ThinkingComponent;
thinker.think(spawn, 1000);

That's all there is to it! There are some caveats, though. Make sure you keep your source MovieClips really simple. Just like it is CPU intensive to play a complex MovieClip, it is CPU intensive to render each frame to a bitmap. In addition, nested clips with separate timelines and as3 code in the clip that is not based on the timeline will not be executed. This works really well for simple frame based animations, but is not designed for complex interactive clips with tweening. YMMV.

Let me know if you have any questions or if you're interested in any other PBE related topics. Also, PBE 1.0 is out! Download it from the project site. This example includes the 1.0 release swc.

Sunday, November 22, 2009

Angry Players Make Sunday More Interesting

Youtopia has been growing quickly the last couple of weeks. It's fun to watch and the team is really excited about it. Of course, with the growth comes a lot of performance tuning with our code. Today we hit an issue I wasn't expecting at all. . .

We've been running Windows 2008, IIS7, and ASP.NET 3.5 in production for a while now, but haven't had to do much of any performance tuning. It just works, and is fast. Which is awesome!

But today, Youtopia was running slowly and requests were hanging so I investigated. The databases were performing normally and not having any locking issues. The network looked good. The memcached cluster was healthy. The queueing service looked great. The ASP.NET performance counters even looked good at first glance.

None of the diagnostic performance monitors I'd used in the past (such as Requests in Application Queue) showed the issue, but requests were absolutely being queued -- or otherwise not processed immediately. There were also plenty of free worker and IOCP threads. The only thing that clued me in was the Pipeline Instance Count and Requests Executing counters were exactly the same (96) on all the servers. So I started investigating from there.

It turns out that due to the way IIS7 ASP.NET integrated mode threading model functions there is a (configurable) request limit of 12 per CPU. We hit this limit in Youtopia today because we hold open requests for asynchronous Comet-like communications and there were over 288 people online simultaneously. Our three eight core web servers each had 96 (8*12) people connected to them and weren't really serving any other requests. We aren't running into any thread configuration limits as the long running requests are asynchronous and not using ASP.NET worker threads.

Here are a few great links that came out of my research.

With ASP.NET 3.5 SP1 it boils down to a simple configuration file change. Use something like this in the aspnet.config file (in x64 it's at C:\Windows\Microsoft.NET\Framework64\v2.0.50727\aspnet.config). This is the default. Adjust maxConcurrentRequestsPerCPU to suit your needs.

<applicationPool maxConcurrentRequestsPerCPU="12" maxConcurrentThreadsPerCPU="0" requestQueueLimit="5000"/>

In addition, the application pool needs to be configured to allow more requests. By default it only allows 1000 concurrent requests. This is done under the Advanced Settings for the application pool in the IIS 7 manager. Set Queue Length to 5000 to match this system level configuration.

About the Author

Wow, you made it to the bottom! That means we're destined to be life long friends. Follow Me on Twitter.

I am an entrepreneur and hacker. I'm a Cofounder at RealCrowd. Most recently I was CTO at Hive7, a social gaming startup that sold to Playdom and then Disney. These are my stories.

You can find far too much information about me on linkedin: No, I'm not interested in an amazing Paradox DBA role in the Antarctic with an excellent culture!