I am often ask what is CommerceOS, equally as often I answer based on what a successful end state for CommerceOS looks like: It is a set of capabilities exposed largely as RESTful APIs that allows anyone to rapidly and cost effectively develop applications that facilitate commerce between or among real or legal entities.
This means if you want to develop an application for a consignment store to sell its merchandize online (on or off eBay) CommerceOs makes it easy for you. If you work for a large multi-national corporation and want to list your products on eBay and on your site, then let local, regional and international buyers purchase your product, CommerceOS is your APIs. If you want to write a pop-up app on iPhone and Android that showcases adventure riding gears and products by mashing up content and pictures with commerce and community, CommerceOS should make your job easier and cheaper. And countless other scenarios like these, you got the picture.
This answer, although accurate from utility and use point of view, does not describe CommerceOS from technical and architectural point of view. A casual observer may not appreciate the technical and organizational complexities that should be overcome to achieve CommerceOS vision. In this series of posts I try to explain CommerceOS from technical aspect.
The rest of this post dive a bit deeper into motivations and problem definitions, in the second part I will focus on how we have approached the problem and architectural details of CommerceOS. In the last part I write a bit about the tough yet important job of measurement and metrics (both tracking operations and goals).
First a bit of a context. Back in 2009-2010 timeframe (where CommerceOS transformation started) eBay had a global engineering workforce in the range of thousands (as it does today). The collective job of this engineering team was to work on an essentially web-based architecture (like so many other web era companies). This architecture assumed (perhaps more implicitly) that
- Buyer interacted with marketplace largely via a web browser, running on a desktop/laptop.
- Sellers were small(er) and interacted either via web UI to APIs designed to replicate Web UI.
- API code base and the business logic powering web UI were largely two independent code basis
- Data and table were shared among all applications and services
- Both data and code for each subsystem and functionality (e.g. Checkout UI or Payment services) was hard to isolate
- Operation technology was fixed capacity, manually provisioned, non-optimized.
The platform was stable and scalable (albit with cost curve implied but then current state of technology) with its own set of processes that made it possible for global teams to get their job done. However, in 2009-2010, it was very clear that we are a few years into development of trends that would re-shaped (or ram through depending on your point of view) traditional web-based architecture and require a different cost and productivity trajectories:
1- The ever present trend for efficiency and cost control driven by the need to expand or at least preserve margins or reallocate resource. This is natural trend for any market and segment as they become more mature and scale.
2- The mobile and connected device revolution
3- The advances in cloud computing
4- The shift of eBay merchant mix from smaller, more casual, lower volume sellers/merchant to larger, more architectural and operationally formal and higher volume sellers.
(I will talk about how technologies known as Big Data changed and impacted eBay architecture in a different post)
It was very clear that the forces of the four factors above require a different architectural approach from what, then, current web-oriented architecture was. We needed to bring down the cost of both infrastructure and operation as well as increase the productivity of our developers to address #1 above.
To address the multi-screen revolution (#2), we needed more than just a set of APIs (which eBay has had since 2003) - MP needed all screens (included desktop) to consume one set of APIs not two sets of code-bases one power the "main" web UI and the other to power all API/devices. This is a much harder problem than simply developing APIs. Additionally, we needed all APIs, regardless of which one of nearly 100 teams across the globe develop it, to look like a coherent portfolio of APIs (given the laws of thermodynamics, you can guess how tough this is).
To take advantage of promises of cloud computing (elasticity of capacity, related cost saving etc.) we needed our business logic and data to be encapsulated in well-defined and isolated modules, with clear dependencies that lend themselves to packaging suitable for cloud environment. Refactoring code to isolate systems and services is tough enough, but isolating data is an order of magnitude more challenging.
And finally, the larger, more sophisticated our merchants and partners became, the more we needed to use formal integration methodologies and feed-based interactions built on de-facto industry standard models of Order, Product, Inventory, Cart, Returns and less on auction-centered models such as Item, transaction etc. This represented a deep change in entity models and re-architecture of code and, perhaps more importantly, migration from old models to new models.
Given where we were and where we wanted to evolve MP architecture, we came up with a definition of a vision for CommerceOS as the basis for alignment and execution, knowing full well that no one paragraph accurately covers all aspect of this initiative. Having said that, here is the definition we came up with:
CommerceOS is eBay MP initiative to transform its architecture in such a way that large majority of its platform capabilities and business processes are exposed as a set of RESTful APIs in the cloud to all marketplace ecosystem participants, uniformly, securely, effectively and with high quality.
This may sound a bit abstract, but let me parse it:
The primary goal is to "transform architecture" - this means both technology stack (for services and applications) and processes not just technology (it never works that way).
Our services are RESTful, it is a de-facto standard and we developed a few internal spec to uniformly covers aspects such as use of non-http verbs, security, tracking, internationalization, filtering, constraints etc.
The deployment target for services is "Cloud". This is not a feel good or casual term, it strictly mean that a service must be produced in a way that it can be packaged fully isolated from other services with well known dependency, clear version number. This allow us to deal with a unit of capability as a building block and use technologies such as Docker effectively (we had a few de-tour here with OSGi but course corrected).
"All participants" indicates that the consumers will be applications written not only for buyers and small sellers who interact with marketplace via a browser, but everyone else on all devices including large sellers who never interact via ebay.com, CS and their internal tools, partners such as logistical service providers as well as internal eBay staff.
Then there is a set of non-functional requirements, each key to how APIs are built, they include
Uniform: basically one code base supporting all consumer, with no "primary" web consumer.
Secure: self-explanatory, platform MUST be secure with well defined identity, AM and auditing. Secure also include availability.
Efficient: APIs must be efficient to consume, i.e. application developer must spent min amount of time and effort to consume it (docs, samples and sandboxes), APIs also should be cost effective to operate (see cloud requirement)
High Quality: An umbrella term covering functional quality (no-bugs) as well as performance.
CommerceOS started with this vision definition and context, in the next post, I will describe the architecture and structure of CommerceOS, both as a technology platform and from organization/process point of view.