several hours?  can you go into more detail of what sorts of tests?  i would think 20 minutes for everything should be plenty.  asking someone to give up a whole day for a chance at a job seems a bit much.  that's like sitting for the ARE.

maybe if you offer to pay them a day's wage for taking said tests?  that's only something like $60 anyway, right?  wouldn't cost any more than a good dinner out for you and the missus anyway (lots of assumptions there, i know)

Let's call it 2 hours and let's assume I'll buy them a nice lunch in the office where we can get to know each other a bit more.  I'm curious what others say about paying for the 2 hours.  A good number of firms have their preferred  candidates back for a second interview, often having them meet other key team members, etc. and no one expects to be paid for that...do archinectors think this a different animal because the applicants are doing something other than talking?

Two possible tasks...this person will be attending site meetings and producing meeting minutes for distribution, so one task may be to write a short (one decent paragraph summarizing what we talked about at the delicious lunch we just had) to gauge their written communication skills.  They'll be building both study and presentation models, so I may have them build a simple 3D shape to see how long it takes them, if they make that object precisely.

I took a Revit test at a major SE firm - 20 minutes to generate connection families for K brace in a hospital project, look it up in AISC manual, set it up with full parameters, install Gusset assemblies, set up detail sheets, print and verbally explain what I did to 5 members of the firm. Some firms have a high degree of rigor and most people just aren't up to it - these tests provide some insight - typically the most efficient persons are the ones that have it.  

2 hours and a nice lunch sounds a lot better than several hours while being whipped and tortured in a sweat shop with generally low light levels and occasional spots with too much glare.

i added some color to the potential scenarios above...

No personal offense bklyntotfc, but this is kind of obnoxious in my opinion as are all of the "expert" job postings. I appreciate you are at least asking for opinions though. . .

If you're in the position to be hiring someone then you have a responsibility beyond paying a wage and having a cog in a seat that makes more money for you, then they will be getting paid. We all know this to be the case but where is the employer-employee exchange beyond the marginal dollars per hour?

If you can't see someones potential through a conversation, ie their desire to learn, their dedication and passion towards the profession then there is no skill test that is going to help you out. What happened to mentorship in this profession? Who cares how long it takes them to build a 3D model, how will a young person every get good if they don't get a chance to learn from someone with more experience, when will they build their skills?

I think there are few other posts on here that are touching on this topic as well. It seems everyone is looking for a ready-made. I just want to know where the architect factory is that's producing interns that already have experience?

Find someone you connect with personally and teach them. There's a skill test for you to take on as an employer.

I'd hire the best candidate - as selected by interview, recommendations and portfolio - on a temporary basis and give them a few weeks or so to demonstrate their performance. The trial basis would be at paid at the same level as the permanent position, which would be awarded upon successful completion of the trial period. Your investment is small, both in time and in money, and the applicant has an(other) incentive to perform well and in real real world conditions.

curtkram....don't worry, we don't let them know about the 80 hour workweek and unheated studio until after they're hired.  You should see the look on their face the first time I show them their workspace in the unfinished cellar ;).

don't be a goat.....no offense taken.  I agree that my job is to mentor, and am not looking for an 'expert', I'm looking for someone who's a generalist, and who has a basic skill set.  But I do think it's reasonable to expect that they have a basic grasp those skills that will make them successful in my office.  But I don't think I should have to be teaching them basic punctuation and spelling skills, or how to cut a piece of chipboard in a straight line.

miles jaffe.....I have thought about doing a trial period.  Do people think it would be more fair to (in the worst case), to agree w/ the applicant that they'd be hired for a 3 week term, but at the end of that term fire them, or to ask them to invest 2 hours and get a free lunch one day?  I felt like it would be more disruptive for a job seeker to have a gig for 3 weeks and lose it.

I don't want to give the impression that I burn through a dozen people to find the perfect fit.  Over the years I've hired about 25 people.  10 of them were great.  10 were good.  And 5 did not work out, which wasn't fun for me or them.  BTW, 'great' and 'good' doesn't mean that they didn't need training and mentoring, and didn't have weak spots, it means that they had a foundation that I could work with to take them to the next levels.

I'd just love to find a way to reduce the 20% bad outcomes to 10% or less.

don't be a goat, there is a thread on this forum where kids are talking about how they lie to get a job.  if that's the climate this guy is operating in, he won't be able to get a very good idea of what a person can do just by what they say.  he wants to hire the best job candidate, not the best liar.

i agree whole-heartedly that employers have a responsibility to teach their staff, and that older staff has a responsibility to teach younger staff as much as they can too.  it seems very common in our society today, not just in architecture but everywhere, that people think someone else will do all the teaching.  that's an incredibly stupid way to think the world works.  that's the core of why i don't like baby boomers too, though of course i like the baby boomers that aren't horrible idiots (there just aren't enough of those).

What we do is to hire on a temporary or probationary basis for a couple weeks, with full pay, but no benefits, time-off accrued, etc. It really makes it clear whether or not you want to hire someone. 

I figure out everything for myself - software, architecture - There isn't anything we can't figure out - it's about problem solving - architects and industrial designers are problem solvers - Nobody should have to think for anyone else -

I sort of had a probationary period (at least that is what the employee handbook said). Mine was 6 weeks with full pay, and then all my benefits started.

That said, a couple hour task test with a lunch would be ok with me if I were looking for a job. Seems pretty reasonable.  

In my experience, what is called above a "probationary period" is the best way to go. Skill tests tend to provide unreliable predictions of contribution. Use the interview(s) to determine the character, personality and general capabilities of the candidates in which you have a serious interest. Bring the best candidate on board, but explain that the first 90-days will be an opportunity for both the new employee and the firm to assess the other in detail. Use that time to assess the new hire's assertions about his/her skills. Be sure to inform the selected candidate that during this assessment period the agreed upon pay rate will be in effect but standard employee benefits are withheld; also make sure the candidate understands that either party can terminate the relationship during that period with at least a 2-day notice.

Typical policy language for this sort of thing is as follows:

Introductory Period

During the first ninety (90) days of employment, each new employee will be in an "introductory status".  During this period, the individual will have an opportunity to examine the company and to evaluate the conditions of employment.  Concurrently, the company will have an opportunity to evaluate the individual's performance and ability to fulfill the initial terms of employment.

During the introductory period, employment may be terminated by either the individual or the firm without further obligation or without prejudice.  If the firm elects to terminate the employee, a minimum of two days notice will be given.  Conversely, the firm expects like notice in the event the employee elects to terminate the relationship.

At the end of the introductory period, the individual's performance will be reviewed with a Principal of the firm.  It is important to note that this is a "performance" review - not a salary review.  If, during this review, both the firm and the employee are satisfied with the relationship, the individual's status becomes that of a "permanent employee".  If either party is not satisfied at this review, "introductory status" may be continued for an agreed upon time period or employment may be terminated.

. . .who cares about spelling skills and punctuation? It use to be a known fact that good artists/architects often suffer from mild dyslexia, should be an indicator you might have some talent on your hands :)

Jeez, I haven't seen this thread about the lying of skills, that's extremely unfortunate, I wonder what prompted that trend though? Is it not in response to an overly technocratic hiring process?  Since there is no more, "Hey kid I like you, I'll give you a shot" vibe out there, maybe this is the new foot in the door tactic . . . if that's the case we have bit of that old chicken-egg dilemma on our hands.

"good artists/architects often suffer from mild dyslexia"

In my experience good architects often suffer from brain damage. And starvation.

We're talking about brand new interns here! Some folks are just slow starters or need mentorship and investment to become the superstars every employer is looking for (without having to pay them commensurately, of course). It pisses me off that many experienced architects seem to have forgotten that mentorship is part of their job and duty to the profession.

If employers need somebody to jump right in without any adjustment period and mentorship, then they should be hiring designers with several years of productive office experience, and be paying them appropriately. Unsurprisingly, finding somebody with less than 2 years experience who fits all your needs is going to be incredibly difficult - and they'll likely be employed already and command higher pay, anyway.

i have no problem admitting that I'd very likely fail such a workplace test. It's impossible to know what every employer is looking for, how they like things done, and how quickly, etc. I'm a professional, not a robot or a wizard. At least not yet. I agree that a trial of 6 weeks should tell you what you need to know.

I don't see anything in your possible test examples that couldn't be ascertained from interviews, resume, cover letter, portfolio, references, previous employers, etc. Perhaps writing up meeting minutes, but again, you have samples of their written word with portfolio, cover letter and resume. Not to mention you could contact their references and ask about their ability to write concisely and coherently. 

I agree with picking the best candidate and giving them a trial/introductory period of 30/60/90 days, whatever you feel might be necessary. 

You all seem out of touch with what is happening in the world besides architecture.  My just-graduated-from college daughter recently was applying for entry level jobs (not in architecture) and the hoops she had to go thru at all of the places, for a variety of types of positions, greatly exceeded the sample tests discussed here.  Some of the positions required a series of written tests, then submitting writing samples, then preparing lesson plans for a mock group of executives, then a day of trial work.   All for entry level work.   If you were applying to Google you would go through a 3 to 5 day interview process where you would be given multiple tests and assignments to judge your capabilities.

I think the testing is a great idea.  You are looking for how a person follows instructions, how they think, how they approach a problem, etc.  You want to see if they take advice, will embrace training, can think independently, stuff like that.  Inc Magazine just had an article on interviewing where a technique was described:  you ask the person to explain some topic to you.  They are to make no assumptions as to your knowledge about the subject, and they can have as long as they want to prepare their presentation of 5 to 10 minutes.  I think this would be an excellent way to show how organized someone is, what their thought processes would be, how they communicate, etc.

Sometime you are not looking for specific skills, but in a small office, it is really important that all members of the team mesh together and can work fluidly.

Archie: what you say is correct for corporate America. But, those companies all have large and professional HR departments to plan and execute these complex hiring programs.

However, the OP is a 2-4 person firm - he probably has neither the skill nor the resources needed to make a 'skills test' program effective. Firms like his are better served by the use of probationary periods, which should achieve the same result.

just because your kid is going through it doesn't make it right.  those companies should probably relax their testing a bit too.  i doubt they have solid metrics saying it really makes a difference, especially since in the real world a business will have to train their new hire to work with the standards they've implemented anyway.  it may be what they're testing is who likes to be hazed and abused rather than who can actually fill out TPS reports or sort mail or whatever entry level people in an unspecified field do.  i also think the only reason the companies she applied to do that is because the labor market is so weak right now.  if we fixed the economy that crap would have to stop, at least in part because it's not effective.

google is the exception.  they get to do what the want.  welcome your benevolent overlords.

those "entry level" engineering positions at google are usually pushing six figures - it's  very different league of applicant.

I don't know what the OP is thinking about paying, but I'm guessing not enough to justify the barrage of tests in the interview.  another vote for probationary period..

I once interviewed at a firm in Richmond, VA.  I had just graduated from grad school.  They gave me an auto-CAD test.  At first I thought this practice was BS but now I understand the importance of it.  See I can evolve!

I had a bad experience working in a small firm, where the principals misled me during the interview.  The team culture also turned out not to be a good fit.  I left after 3 months. 
 

So for me in the future, once a potential employer express their serious interest, I would proposed to work for free for at least a full day with their team and in their office to really find out what I’m getting into.   I’m “interviewing” the office as much as the other way around.   90 day trial period is too long.  20 minute tests are not enough. 
 

I haven’t had the opportunity to test this proposal since I was fortunate enough to be welcomed back by my old firm so no need for testing there.  But as an potential employee, I’m perfectly willing to put in free labor to ascertain the culture and mentoring capabilities of a firm I’m interested in.  Question is: would an employer be willing to expose the inner workings of their office to truly find a good fit both ways?
 

 CrazyHouseCat, offering to work for free devalues you, and a day is not enough to scope out the situation for either side, Either side can decline to continue after a multi-week trial, and you get fairly paid for your effort.

^ +1  -  what Miles said.

+2 what miles said.

@ will galloway apparently interns are getting milked as well. . .

Just a question to any mid-career or senior architects out there. Did you go through a skills test when you got your first job? OP did you?

My first interview was with a well known NY architect. As I took out my portfolio he said, "Put that away, I have a portfolio." We talked about architecture, my career goals and why I wanted a job. Can you believe it, why do you want a job?! ( the answer is not to make money but boy I bet people would stumble on that one today)  I was amazed that I got hired based on just having a conversation . . .  followed by a sense of - oh damn, I really better step my game up this guy just put a ton of confidence in my ability. He put his chips down, as he should and so did I for the following four years working for him. For the employers out there; you want to know how to get the best out of an employee? Inspire them! They can only be as good of a worker as you are a leader. Give them the confidence that they don't yet have, give them some room to mess up, you did. Maybe we could use a senior managers/employers thread on here to share some stories about how bad they goofed when they started out . . . or last week.

There is a growing amount of research and opinion within the fields of cognitive psychology/neuroscience that suggest rules, regulations, testing, incentives and all the devices we use to strive for certainty are failing. We are killing intuitions and moral skills in the process and the outcome is mediocre at best. 

For any interns out there, don't participate in any skills tests, if you do you're a goat, a sheep, or some other passive ruminant . . . just showing up and chewing grass because someone told you to chew grass.

(this not all directed at the OP, just raising some criticism towards this topic in general)

Problem with multi-week trial is: a lot of us look for our next job while having a job. 

A day is not enough to really understand an office, I admit.  However, you'll get a much better feel for a place from actually sitting down amonst other employees and perhaps grabing lunch with your future peers and not just the boss, compared to being isolated in a conference room and hearing the one sided story from employer who is out to charm you. 

As to devaluing... I guess it depend on each person's point of view.  To me, I'm buying valuable intell, and it's worth a day's work.

Of course, nothing is full proof, there will always be element of risk and luck on both sides.  I may consider this a completely idiotic proposal in a few years.  But in my somewhat-youthful-ignorance, I don't see how else one can get a reliable feel for the culture of a work place without screwing up one's current job.  I actually rather prefer the 3-5 day jumping through hoops interview thing going on in other sectors as mentioned in other posts.

don't be a goat, here's an interview question for you then.

what good are you for your employer?

i know you can learn stuff if they take the time to teach you.  that's what you went to school for, and you paid your professors then.  why would an employer be your professor for free, and pay you as well?

if your employer is thinking "i need someone to help with drafting and taking notes," they are going to want a candidate that can help with drafting and taking notes.  they should pay you, and they can pass the cost on to their clients by billing for your time.  they should also teach you to be a better architect, and maybe even give you a position with more responsibility as time goes on (or encourage you to quit if they can't develop a better position for you).  if you can't spell or do any drafting, why does the employer want you around?  if people are lying about their skills, that might be why he feels a test is necessary instead of just talking about it in an interview.

to answer your question, i once got a job after one of those tests.  it was probably only about 20 minutes or something though, and i did not get  a lunch.  the test was only for drafting in autocad, and this was while i was still an undergrad student.  the probationary period is more common; i think i've had 2 of those.  as i recall in the first one i got a raise after the 60 or 90 days or whatever, so they basically stunted my paycheck while we were evaluating each other.

An employer passes the cost of staff onto the client with a multiplier attached and they cash in on the difference. That's how you get the big bucks for being the boss, for taking on risk and responsibility. Are you under the delusion that this happens at a one to one ratio?
You are hiring an employee because you can't do all the work on your own, they are going to be the ones who make you more money than you otherwise could alone.
They also learn the new tricks like how to  "do the Vray"  or any other new skills that older employers don't possess themselves but are cashing in on when it comes time to billing the client. 

Just keep in mind that some people can be bad test takers, nervous, squirrely, etc, especially if someone is standing behind their back, (the typical pirate's parrot).
 

But a regular trial work experience, seems to be more appropriated, giving the candidate a chance to be productive and perhaps just what you are looking for.
 

so the answer to "what good are you for your employer?" is "do the vray?"

no, although that seems to be the most prized skill nowadays. The answer is an employee expands the capacity of the employer which equates to more dollars. Why would you ever hire someone if you didn't need help?

that's the point of all this isn't it?  the employer needs to somehow be assured that you have the ability to contribute to their firm in a way that can help out where they need it.  the OP was asking whether a skills test is appropriate.  my opinion was that several hours was too much, but personally i think it's ok.  the general consensus here is that said skills can be tested better during a probationary period.  the crazyhousecat suggested that is a difficult position for someone looking to change jobs.  imho, these are all great suggestions.

a conversation during an interview is important, but that may not be enough for the hiring agent to know whether you can perform the tasks they will need you to perform.  then again, maybe it is enough.  as far as i can tell, your position is "no skills to start but teach me because you like me."  other than that, i'm not sure i know what you/we are talking about anymore.

@the OP. I think that's why they design the "probation period" either 3 or 6 months will be more than enough if the candidate are the right one for the job. Every office is different and requires different skills. I think you will need more than just a few tasks is not enough.

I'm just challenging the assurance part. People aren't machines and they shouldn't be treated as such. If anything it makes sense to me that a senior or highly technical position would have to pass a skills test not a first timer. Like if Arup was to hire a facade specialist and they wanted to see how someone handled the software needed to operate at a very high level of competence, then I get it. Very coveted position by a very established practice. They earned the right to skill testing. I'm just posing the alternate of a less empirical approach and trying to express that its empowering to someone starting out for their employer to have some faith in them and to take a chance, to invest in their potential for growth and development over a longer period of time rather than an evaluation on where they are today. 

If you want assurance then hire an equal or someone with experience and pay them accordingly, which I believe someone else here mentioned as well. My points are more specific to the nature of this being someone just starting out.

I'm curious as to your thoughts about what might be causing the skills lying that I posed in a post above? Could you post the link to that tread as well?

3-5-years-of-experience a.k.a. "The Skills Lying Tread"

here is the thread where people suggested lying.  they are referring to experience lying rather than skills lying. . .

http://archinect.com/forum/thread/67374688/3-5-years-of-experience

i'm pretty sure it's caused by people who want a job but cannot find one.  in our industry, it appears a low paying entry level job is a very coveted position, and unfortunately very well qualified and competent young people are competing against people with technical skills as well as experience.  there are a lot of people who deserve an opportunity but won't be able to find one.

Lots of interesting comments.  A few reactions:

I've never had applicants out and out lie about their skills, but have had people say (and I believe that they honestly believed) that they were very proficient at 'x', when they were not.  I don't think it's unreasonable to see where their skills actually lie, whether in a series of tasks, or in a initial review period, or maybe even both.

don't be a goat...assessing someone's current skills/abilities today does not exclude showing faith in them and investing in their potential.  I'm proposing to do both.

And trying to assess someone is not the same thing as trying to hire someone with project manager skills for the price of an entry level draftsman.  In my office those lines are a little blurred, so that I usually have a younger staff person take project notes - usually a project manager's job.  But that means that they get to attend client meetings, to go on site visits, etc, where someone in a larger office is probably spending that time drafting.  But if I hire them and subsequently find that they can't spell, have poor grammar, and are not able to concisely summarize a five minute conversation to it's essential points, then it's a problem.

The idea is to try to identify a few basic traits that are essential to working successfully in my office: can they communicate effectively in writing...can they follow multi step instructions without problem...are they literate in basic math...do they think holistically.

I've given the example of having them do meeting minutes of whatever our lunchtime discussion was.  Here's another idea I've been playing with:

Start an applicant with an existing, simple CAD drawing of a space - plan, RCP, sections, all laid out in one file.  Give them verbal instructions (they'll be asked to write them down, not to try to remember everything in their head) on how changes need to be made to the drawings by adjusting the square footage of one room.  After the adjustments are made, they are to bring the drawings into illustrator and and set them up at a certain architectural scale.

In theory, what am I hoping to see:

1. Can they follow the instructions.

2. If they have problems with the instructions, will they plow forward, or ask a question?

3. Can they do basic math (area calculation and scaling of drawing in illustrator)?

4. Do they adjust all of the drawings or only the plan?

It's not rocket science, and I'm definitely not going to determine if they are an Autocad genius.

To those archinectors who are not horrified by this idea, anyone want to propose any 20 minute tasks that would be effective in highlighting one aspect or another of architectural/professional aptitude?

I bring this up again - do you want somebody who knows the ins and outs of your particular work flow (which seems odd as you mention Illustrator) or do you want a critical thinker who might be able, given the opportunity, to exceed your expectations?

I started a new job last week (intern) and I was honest about my experience and approach. Even though I have very little autocad experience (and have never opened Revit), my new employers took a chance on me. I happen to have my head screwed on straight and I know that my value is not in whether or not I'm a CAD wiz.

your testing approach might be perfectly good if you're looking for a CAD tech (and it's okay if that's what you're looking for - maybe you don't need an intern at all). But if I was given a test for a sub-$40k job, I'd walk out because I know my value to you as well as myself. Good luck.

my first job out of school was given by conversation as well.  no portfolio.  worked out great.  second job involved portfolio but mostly was interview.  i have never taken a test to join an office.  could be interesting to go through kengo kuma's system just for fun (day long design charette as i hear it), but the corporate models sound painful.

i think younger architects (meaning all graduates from school, including interns in NA system) think they know more than they actually do.  Its natural enough.  And fairly obvious too.  lieing on the other hand is not coolio.

For my first and current job: After being called back three times for interviews and being narrowed down to two candidates I was asked to compete in a design competition with the other candidate. Over a weekend design a lobby space for a 'hip' office and produce a plan, rcp, and rendering. Frankly it pissed me off and I almost declined, but I did it and got the job on a 3 month temp basis. The 3 month temp basis is basically a way they can save a little money and not pay you benefits. There was never another critique at 3 months I just started getting benefits. They ended up hiring the 2nd place person 4 months later, turns out it pissed her off too, but she still did it. In all I probably spent 50 hours on this one job prospect. Obviously I'm still bitter about it. 

To try to add something constructive to the conversation, if you do decide to test this person in some way please give them some warning of the time involved and general sense of the task before they arrive so they can plan accordingly. They may have kids, other jobs, responsibilities. 

Or better yet just narrow down the search by resume's and hire the person you get along with the best. It shouldn't be this difficult. 

how did that work out heavy?  did you find out you just love the people and environment?  did you discover it was a sweat shop and every morning you wake up it's a little worse than the one before?  are you constantly afraid of layoffs because they pride themselves on higher turnover than mcdonalds?

Getting someone to do a design competition against another candidate sounds like something out of "the apprentice" with Donald trump. At least Trump offer high incentives.

Macdonalds don't require you to work unpaid over time.

In Japan a lot of office have an "open desk" system where you can have an extended office visit (somewhat as a starter internship), they pay a measly per diem; I frankly find it aweful unless you're a student with means.

My first job i had no experience (nor coursework) and no knowledge of CAD at all; but the economy was good and they had 2 applicants for 3 positions.  I got the job, worked my butt off to get familiar with everything and it worked out for everyone.  Since grad school post '08, employers seem far more skittish in taking on lower level labor; hardly anyone trusted my resume or portfolio.  Some insisted on more and more CAD drawing to see (you'd think 5~7 yrs of exp in firms would be enough for an entry level gig).  I eventually got jobs in places that trusted me and/or trusted my references, which brings me to why not interview references vigorously.

My current firm has schools they like to hire from, because they have consistently produced students of quality and personality that fit well; and we maintain regular contact with faculty there that we trust.  When looking at other applicants we ask the references in what capacity they know the applicant and then ask the hard questions of performance under stress, etc.

I also enjoyed my "probationary" period where I got to work as a contractor - kept me off of benefits but gave me flexibility tax wise and allowed me flexibility to quit and/or look for other work.  I can see why it's hard to quit a job and switch (much like tenured academics?), but it's fair for entry level without a proven work record.

@curtkram - I should have listened to my initial instinct. 

@bklyntotfc -  If you're interviewing someone with two years experience couldn't you call their previous employer? If it's a recent grad you're interested in could you find a small competition that you can work together on and get to know them that way? It might be more time invested for both of you but at least if it didn't work out it's something a recent grad could use to help them, you could even give them a good reference. 

you should atleast offer to pay for the 7 hrs, incase the person doesn't get the job.

I think that this test would do nothing to eliminate questions you already have about a particular candidate.   I have made the mistake of not calling references ONCE.  Any of these questions that you believe your test will identify can be easily extracted from a prior employer. 

You are also making the mistake it seems to be looking for someone with particular skills instead of looking for a good architect. And why are you afraid of fireing someone if they don't work out? That's what good leaders need to know how to do as well.

maestro - the idea is that my hypothetical tasks would identify or highlight whether a candidate was (or was not) going to be a "good architect"...or more accurately a good professional....Can you communicate, can you focus, can you anticipate what's next, etc.  These skills will make you a good architect, a good accountant, a good whatever.

And no, I'm not saying a candidate should be able to read my mind, or won't need plenty of on the job training, but with those traits, the training will stick.

A 2 hour test would not be the only gateway they'd have to pass through.  Their portfolio and cover letter are the 1st gate, their first interview is the 2nd, speaking with their references the 3rd.  For references, I've found that for those w/out much work experience - and we're talking about an entry level position - that a good rec from a professor doesn't always predict job performance.  And working for a large firm - which I've done too - is very different than working in a 2-4 person office, so again a good reference from their supervisor at SOM is also not so good a predictor of good performance in my office.

It's perfectly fair to ask whether it's possible to come up with a set of tasks that will consistently identify something that the first 3 gates missed.  The only way to really know is to do it for an extended time until there's a large enough pool of people to say definitively one way or the other.

As unpleasant as it is, I'm not afraid to fire someone, but it takes a significant amount of time to have to go through the process of vetting resumes, interviewing, etc. to find a replacement for the fired person, and extremely disruptive to have to cover that person's work between their firing and the hiring of their replacement.

I have been following this thread for the past few days. It has been interesting reading the dialogue so far. Anyways, for Bklyntotfc, I recently stumbled on this transcript of an interview conducted with BIG's Bjarke Ingels in 2011 . He was asked what led to his success so quickly. His response might shed some light for you regarding new hires (bold text is my emphasis):

Bjarke Ingels: (Bjarke draws a pie graph) So I would say about this much (5%) luck. I would say this much (5%) awesome clients. And the rest would be awesome talented, hard-working BIGsters. I think one thing that I've been fortunate with, and I really don't say this to be politically correct. I now have seven partners and half of them were my students at some point a long time ago. Practically all of my project leaders - actually that's not totally true - at least eight of the current superheroes within the office that win competitions and crank out awesome stuff, have all been interns or students of mine at some point. And the fact that over the last 11 years we have maintained relationships with people. Like some of my partners I have been work working with for 11 years. That means that we don't have to start over all the time, we don't have to hire people off the streets that are complete psychos trained in ultra-competitive environments and impossible to work with and really hard asses. You've probably worked in competitive office environments and you know the environments can be pretty psycho out there. In that sense the accumulated effort has made it sort of into a snowball so that I have more than 100 colleagues but most of them I know him intimately. I can have telephone conversations or text them instant messages and we will know what each other are talking about so it means that more than the amount of projects or the amount of buildings that we've been able to build its the fact that we have been able to accumulate a culture of collaboration. Also when you look at the techniques and talents and procedures and sensibilities that float within the organism is sort of a presence, an embodied potential of knowledge that is carried within the organization and is spreading around. So like after two months in the office it turns people into people that are incredibly good - they learn these skills that have nothing to do with me - it's simply just stuff that's there. And the leave with afterwards. So we've been fortunate in the sense that that 80% of the competitions they were doing right now have been interns that have left, graduated, and come back, so we don't have to take the chance of hiring somebody that has a great portfolio but turns out to be insane once they are inside the office. But we actually know people that start the first day and we have already worked with them. So I think more than anything it's this accumulation of culture within the office that ensures even though we're all getting more busy and traveling more and more, I still think that the work we do as an office gets better and better every time and that is simply because we're not just making projects and building buildings, we're really building an architecture office.

http://www.clog-online.com/news/clog-big-storefront-transcript/