Interviewing @ eBay Part V - Dir. and VP of Engineering

My base line for interviewing senior engineering management  is the following, non-scientific, completely made up, definition:

Job of a manager is to develop & allocate resources and manage execution of projects to satisfy time, budget and quality constraints and to minimize risks and promote efficiency (repeatability of the whole process). Leadership (as in the ability to inspire and influence change) is desirable, and indeed necessary the higher one goes in the management chain.

Based on this, there are five types of questions I’d normally ask,

·        General/ice breaker: One or two questions based on resume or general questions such as Why eBay? Why now

·        Technical Management : Ability to manage large teams, projects, timelines, budgets and plans

·        Leadership: ability to conceive , evangelize and cause positive change

·        Personal: I mean to assess personal integrity, awareness, reflection

·        Field Specific: Needless to say, a director of DB engineering gets specific DB questions, director of commerce must know details of order management and payment processing VP of personalization get collaborative filtering and VP of applications get “how you build a large web app” question.

 Here are the current (and expanding) bank of question I’d normally draw from, please email me or comment if you have other suggestions...

 -        General

o   First 90 days at eBay?

o   Why eBay, Why Now?
o   Talk about the most defining event in your professional life. 

o   What are the 2,3 interesting and promising trend you see?

o   What is leadership to you? What is management to you?

There are safe and conservative answers for these questions, however answers that reflect
measured risk taking and authenticity are always preferable. 

-         

 -        Tech Management

o   How would you structure an engineering org?

o   How do you measure the progress and success of a project?

o   How do you decide the allocation of engineering resources in multiple locations?

o   How do you manage promotion process?

o   How do you allocate bonus budget among your team members?

o   How do you see your relationship with product and architecture function?

o   How do you manage your hiring process? Who would you hire?

o   A new technology for part of your stack is emerging (e.g. a new presentation technology or a better and open sourced database, new JVM or cache …) would you replace your existing technology stack with the new one, why or why not?

o   How do make sure knowledge sharing is effective among your team members?

o   How do you ensure the quality of your delivery?

o   What is the most important job of a technology manager (pick one!), why?

o   How do you monitor the day to day tasks and assignment of your team?

o   What is your view about innovation? How do you practically manage an “innovative” team?

o   How do you deal with “NIH” – Not Invented Here – issue?

o   What is your talent development philosophy?

o   How do you chose and prepare your successor?

o   Would you direct your team to execute on a course of action you do not support?

o   How do you increase productivity? How do you measure it?
o   How do you empower your team to do "the right thing"? even when there is no time or budget for it?

    

-        Technology

o   How do you manage development life cycle? What development life cycle do you use?

o   What is the current technology stack you are using? What are the benefits? What are the drawbacks? Why it was chosen? (please don’t say “I don’t know, it was chosen before I join”)

o   How do you plan for migration from an old, stable large system to a newer version of the same system?

o   How do you feel about redundant work? is there an occasion that It may be useful?

o   When do you use open source? What are the challenges? When do you use vendors?

o   What is Agile to you? What are the benefits and challenges?

o   What 2,3 question you would like to ask about eBay technology?

o   Explain the CAP theorem.

o   What are the measures/steps you take when your system is in operation?

      The Person

o   How do you stay current of state of technology? What part of stack are you interested in the most?

o   How have you improved over the years as a manager?

o   What is your proudest moment as a manager?

o   Tell me about your biggest mistake, how did you realize it was a mistake? What did you do afterwards?

o   What is one criticism that your subordinates make about you?      

o   How can twitter be used to improve eBay?

o   How do you improve eBay? Now, I just told you to do something else – that you feel is not right - , how do you react?

o   What is your dream company to work for? Imagine now that you have an offer from that company, what should eBay do so that you work for eBay instead?

-         

  -        Leadership

o   How do you deal with a “failing” project? Or a project in crisis?

o   Your plan requires the cooperation of another team, but that team has its own priorities and plans, how do you convince them to allocate time and resource to your project?

o   How do you influence and convince a group of people over whom you have no authority? Give an example

o    How do you mentor/coach your team member?

o   You receive a call at 2am telling you that the entire search (or checkout) is down, what do you do?

o   Two senior technical leaders (or teams) escalate a technical difference to you (MemCache v.s NoSQL or doing it now vs. doing it in the future …) how do you settle the matter?

Interviewing @ eBay Part III - Software Architecture Interview

I don’t know of any job title/role in technology that is more controversial, and evoke more emotional reaction, than that of an “architect”. Engineer, engineering manager, product manager, accountant, business developer etc. all have almost the same definition/responsibilities from company to company,  architects role though vary widely: in some firms one cannot do anything without an architect permission and in some others the role is completely eliminated.

You should first know that architecture is a role with a wide definition (TOGAF alone defines five types of architect - enterprise, business, data, application, IT). EBay architects play a combination of tech lead, internal evangelist, tech management and product management, and role is often the agent of change for eBay technical direction, tech stack, technology choices, process and methodologies …

Interviewing and selecting an architect is especially challenging. In addition to core skills of a software engineer (yes if you are interviewing for an architect position, you should be comfortable coding – no Java guru, but be able to code), the main attributes I am looking for are:

-        Integrity: Change in technology often brings about change in organization and power structure, people currently in power know this and may not be enthusiastic about  it, architect should have the integrity and courage to call for change when  it is not popular.

-        Leadership: integrity and courage is necessary but not sufficient, in this role you should have leadership i.e. the ability to influence, inspire and induce change in direction (often major changes) in a way that people want to make the change, not forced to (you will have no formal power anyway)

-        Clarity : last but not least, architects MUST bring clarity to situations where goals are unclear, definition of problem is fuzzy, needs are uncertain, data is incomplete, assumptions are inaccurate, yet delivery is urgent and pressure is high …bringing clarity to all aspects of such situations are often the most important function of an architect at eBay.

So for interview, expect some of the core software engineering questions, with much more emphasis on modeling and problem solving plus few of the followings:

• -        When you are asked to “architect” a system – say photo album app – what does that mean to you? What tasks do you perform? What would be your deliverables? How would you interact with engineers?
• -        How do you ensure the delivered system conforms to your architecture?
• -        Model and Design eBay
• -        From the time you type in www.ebay.com , to when you see eBay home page, explain what happens under the hood, at all layers
• -        How does Ajax-style interaction impact a traditional/classical page-oriented architecture? What are the changes it would force to the classic architecture.
• -        How would proliferation of Mobile application impact the classical web based architecture?
• -        Explain Map/Reduce in simple but reasonably accurate term, in a way a marketing person can appreciate it.
• -        Describe challenges and best practices in developing a distributed system – such as SOA based system.
• -        Describe the qualities of a well-designed API or service interface .
• -        Describe your favorite application development framework or design, explain its benefits and shortcomings  (e.g. Spring or Struts, or your own framework)
• -        Compare and contrast SQL and NoSQL DBs, when do you use each?
• -        How do you store a social graph like LinkedIn or Facebook?
• -        How do you decide to buy or build a piece of technology?
• -        eBay, as other online merchants and markets, has a policy against sale of fire arms, how do you design a system to enforce this policy?
• -        How do you design an application – such as a cart or check out flow - in a way that product and UI folks can experiment with and optimized different aspect of it?
• -        At any given time, eBay support a set of widely used browsers, for the rest, it display a warning message and asks users to upgrade to another browser. How so you design this system?
• In a large and distributed system, how do you ensure data-consistency for critical functions such as  authentication/login 
• Discuss a few significant technology trends, why do you think they are important? How would you anticipate their impact on current architecture/system?
• What would you do in your first month of working for eBay

If some of the questions sound vague, it is because they are! (btw, they are a lot clearer than what you'd face with in reality). Remember that you need ask questions, seek and bring clarity to the problem definition before you jump into the solution.

Again if you are interviewing for a particular specialty such as Security, I18N, Messaging, Operations etc. you should expect particular question in those areas (I will post a list of question for my security and identity architecture interview later), but for system and application architecture, be prepared for at least 3 or 4 questions from the list above. 

Interviewing @ eBay, Part I - The basics

When someone interviews with eBay, s/he is given an interview schedule with the name and title of all interviewers, the natural expectation (at least mine) is that s/he searches for the name of all those people as part of the pre-interview preparation. I view this as minimum due diligence that a candidate should do 11 years into the 21^st century. So I hope whoever interview with me at eBay finds and reads this post (if you do, please let me know)

Now that you found this, I will give you a leg up over other candidates:  in the series of four posts, I tell you what questions I would be asking in my interviews for four positions:

• -        Software engineer
• -        Product Managers
• -        Software Architects
• -        Engineering Managers (Sr. Managers, Director, Sr. Director and VPs)

Before we start with specific position, let me first cover the common questions and aspects for all interviews.

I look for the following “necessary” – but not sufficient - qualities that make a candidate productive. In a nutshell, person should be smart, know his field, willing to work hard, willing to compromise and get things done and get along with people under a range of circumstances.

Smart: I am not talking about genius, or someone that can solve puzzles in 10 seconds, but some one that is generally sharp, can think on his/her feet and is solve problems. One of the clearest indication of it is whether someone listens to question, asks follow up questions to clarify what is being asked and then clearly and directly answers that question and then stop. No rambling, no answering other questions and no circular, perpendicular or random answers!

Knowledgeable: Candidate must have proficient level of knowledge in his/her domain, this is separate from being smart, each field requires certain level of experience and formal education – I expand on this with specific question in each of the fields above.

Work ethics: Regardless of how smart and knowledgeable one may be, s/he has to be focused and will to work hard. Real engineering tasks are 10-20% about great ideas, and 80% about grunt work, boring details, dealing with plumbing,  debug, re-build, fine tune etc. If you are not willing to do that, you won’t be successful.

Pragmatic: You must be willing to compromise, change course, give up credit, change your familiar and favorite terminology etc. to get things done. All the smarts, knowledge, hard work often is wasted if you cannot get it done and out at the end. I ask what you are willing and what you are not willing to compromise on for a given project and why, what you would do if you feel a wrong decision was made…

Culture fit: The last of the “necessary qualities” is the ability to get along with others under all sorts of circumstances: uncertain and insufficient data, deadline pressures, failures, inter personal and inter group rivalries … under all those conditions, you should be able to maintain your relationships and get along with others. One of the greatest indicators of whether someone can do it by the way, is sense of humor.

Next post: my list of questions for Software Engineering positions.

Decision Making Biases

We all make decisions, and I suspect we feel we make most of those decision objectively... or do we?

Here is a great article from HBR about decision making biases (free access till July 4, 2011), the best part is the survey link at the start of the article. Once you answer the survey, it compares your answers to other respondents' answers and grade your decision making biases from low risk to high risk for the following biases:

Pattern Recognition Bias - The "Oh, I have seen this before. Here is what are going to do ...."
Action Orientation Bias - The need and bias for action in part of decision maker
Stability Bias - The need to do as you have always done it
Social Harmony - The "group think", or "here is what everybody else think/say we should do ..."
Process Orientation Bias - Speaks for itself
Self-Interest  Bias - simply put, thinking what is good for you is good for the firm or whoever suggest an option, always makes it with the best interest of firm in mind.

The one, the jumps at me in particular is the "Social Harmony" bias, specially harmony with upper management. This is one bias I have seen so predominantly is most all decision making. Specially when upper managements do not formally separate discussion and decision and simply voice a strong opinion in a debate. Expressing dissent after that becomes so risky that even if you don't have the bias, you are likely to adopt one temporarily !

LSE economist answers: "Why are terrorists often engineers?

IEEE Spectrum has an interesting podcast here title Why are terrorists often engineers.

Basically two hypothesis:

- Engineers are smart and high potential people and when/if they don't find ample employment opportunities they become frustrated
- Engineers have orientations toward hierarchy and order that is also a common theme with fundamentalism.

Also interesting (according to the LSE researchers) engineers are more religious and right leaning compare to others - one wouldn't guess that living in the Bay Area.

Special Forces, Regular Army and Start ups

From time to time I advise start ups on mostly technical matters, inevitably they ask me for referrals to hire engineers or help with editing their job postings.

Typically they email me a very detailed and specific job description for a, say, Data Architect or Ajax messaging programmer or Java Engineer complete with an obscure list of libraries and technologies that the candidate expected to be master in (probably those technologies were suggested by their VC to them in their last meeting)

Start ups should be built like special forces, with a few brave and smart generalists with a lot of heart and "get it done" attitude, not by deep specialists in any certain area - I am talking about the first few engineers here - a statistical modeler with no programming skills or a Data Architect with focus on data replication makes a fine contractor for a short time to advise your core engineering team, but should not be one the first 5 engineers you hire.

Meetings kill Loose Coupling

There is a brilliant, yet underutilized, sociological observation called the Conway’s Law. It was stated by Mel Conway in his 1968 paper. It states that:

Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.

The key phrase is “communication structure”, that is roughly, but not always, equal to organization structure.
You may have not heard of or read his paper, but I am sure you have seen the effect of his observation:

If you have two teams called Application and Kernel, then chances are you end up with two deployment units called applications and kernel (jars or folders or zip files or any other form of bundling code), on the other hand if you engineering organization has two teams called Books and Games, then you end up with a book and a game application, if there is no other team/group, it is unlikely that you can see a kernel or core subsystem that encapsulate the common construct between the two.

Now, we all know about the principal of loose coupling, and how it enables flexibility, efficiency, increased productivity etc. One very effective way of creating loose coupling between two system (let’s call then consumer and producer) is to make sure that members of the producer team do not meet with members of consumer team! You may convey requirements to them, but forbid meetings. If there is not communication, it is hard to build hard coupling. It is a bit strange and counter-intuitive, but we have used this with success in building key infrastructure services.

I have found out that a good early predictor of the level of coupling (or quality of interfaces in general) of a system is the list of invitees to their early design meetings. Whenever one of few engineers from a future consumer is part of the meeting, there is a good indication that they systems will be somehow coupled (either thru the types or domain values of parameters passed or the way errors are handled, the marshalling of the result, or even the terminology used) – This of course is a generalization, the team somehow has to collect requirements, or share result s and get early feedback, this is all OK, but if your goal is to create “loosely coupled” system, you should make sure your communication structure,  are loosely coupled as well.
Take a service that verifies a credit card number to ensure validity and that a provided name indeed matches credit card issuer record. This service may have a method/operation in its interface:

Result VerifyCreditCard(UserId id)

It assumes that somehow the service can obtain a credit card number from a supplied UserID. This is a very tightly coupled interface that shows the service provider has had too much knowledge about its consumer.
Here is a less tightly coupled (better) example:

Result VerifyCreditCard(CreditCard card, BuyerName buyerName)

This is not too bad, but the choice of “BuyerName” indicates that the provider has the knowledge that consumer (the one she probably met with) happens to deal with principals that are buyer.
Consider a loosely coupled version that one would probably write if there is no additional knowledge of potential consumers

Result VerifyCreditCard(CreditCard card, Name name)

Communication channels are very important interface design, this means people in each design meetings should be selected carefully, if system A should not be dependent on system B, the best way to ensure it, is to reduce communication between developers of system A and system B.