What is CommerceOS - Part IV - Foundations and Shared Services

Part I of this series of posts, described the what CommerceOS, Part II explained the structure and overview, in part III we focused on the standards that make a Microservice portfolio work. In this post I will focus on shared services. Let's start with the natural question

What is a Shared Service:

Of course most services are written to be re-usable and to be "shared" among multiple applications and services. But naturally the scope of use varies for each service. At the lower levels of any dependency graphs, in any service portfolio, there are services with much wider scope of use, a lot more services depend on them and their function is less domain specific and is more generic and "platformy" functions.

Although there are no fast, easy and mathematical rules to define a "shared service" - we have defined a few criteria:

•  Shared service can not depend on any business services (basic sanity of dependency graph)
•  Shared services are deployed and are accessible to all other services - they are part of the services runtime.
•  Shared service have more stable interfaces and slower release cycle - this is normally the case since the functionality is not business specific, so it does not change with business requirements.
• Shared service is, conceptually, something that is useful/applicable to any company (not just eBay marketplace), so at least in theory, they can be either open sourced or made available to different business/company.

The following services are CommerceOS list of shared services (at the time of this writing)

1. Identity and Access Management: Provisioning Identity for all Apps and Services. Plus an OpenID Connect based authentication protocol and a SAML based JWT based security token services. 
2. Billing: Multi-Tenant billing and invoicing based on billing event stream. 
3. Messaging: Few internal message buses, primary message bus is an internally developed light-weight and highly scale-able Business Event Stream (BES) - it is a publish-subscribe only messaging system. Transitioning to AMQP based messaging system to support wider set of messaging semantics - some use of Kafka.
4. Logging: Distributed logging, uses TIBCO message buses, stores in HBase and integrated with Hadoop. 
5. Monitoring: A real time version of Map-Reduce (Red Lemur) - similar to Apache Storm. 
6. Tracking: of API calls and user activity events. Custom event API, types and structure, over TIBCO buses, 
7. Config and Metadata:  Custom key/value pair service transitioning to Zookeeper based config management 
8. Content Management: A template language for defining content and its variations, translation workflow and a run time distributed content repository backed by MongoDB and managed by ZooKeeper
9. Discovery and Registry: A build time registry tool based on Google Discovery Doc (GDD) and backed by a Casandra cluster.  
10. Routing and Inter-mediation: WSO2 based ESB, kept simple. Only provides routing and location transparency. Most composition and transformation of lower services are done by higher level application services, intermediary layer is kept "dumb" as much as possible. 
11. Crypto Services: HSM based hashing and encryption service, based on each tenant security policy.
12. Object Storage: HBase-based storage for images (or any BLOB)
13. Caching: Memecahed based API/Protocol implemented by Couchbase  

Shared services are multi-tenant, and can be consumed either as a service (majority of cases) or as a separate private instance (a few cases). Shared services comply with the rest of CommerceOS standards. The access to shared services are controlled by tokens provisioned by CommerceOS STS (secure token service) based on the provisioning rules for each application or service.