• Automate Data Processing (whether its Business Item Ingestion, Cloud Platform Automation)  always need UI-driven (API-driven) Templates and Template-driven dynamic Workflows (preferably using MEAN Stack)
  • Users can define dynamic Wrokflows with any number of steps just by leveraging dynamic json documents inside a Pipeline Context .
  • So think lean and map the Json database of Business Rules with Angular Js Form elements / React-enable Cards
  • Let the UI drive the Business and define dynamic templates . Gone are the days when every single change in Business Requirement or Process configuration required a change in database which in turn unleash a dependency mayhem in top layers (break free from infamous ORM , memory-hungry Cache, Enterprise Anti-patterns, Facades , DTOs what not !)
  • Great !  No longer Bottom-Up … think Top-Down ! Empower Business Owners craft new Rules , create personalized multi-tenanted Dashboards and customized Pipelines !


Welcome to the highly parallelized asynchronous Node.js World !

  • Connect NodeJs to MongoDB, Elastic Search , Kafka and delegate all heavy-weight processing to Job Engines (Erlang / Go)
  • Remember we started with a single Workflow NoSQL Document structure where we keep pumping multi-level status messages !
  • Pump all the status messages via Socket.io to UI , refresh Dashboards , send Alerts !
  • So your Business Manager came up with bunch of new Policies – No problem offer him an Admin UI to tie the Policies to your Workflow either as embedded documents or linked documents !
  • Develop your code in the same speed of Business Decisions – Try early ! Fail early ! Experiment multiple User Experiences without changing your Node.js backend and Mongo datastore ! Favor decoupled , detached layers communicating via JSON !

Key Features :

  1. Capture all the business requirements as configurable Rules
    1. example :  specify all the rules (UI fields, data types, validation logic) inside Json scripts.
    2. Introducing new feature is as simple as adding a new rule { fieldName : “delivery_option”, fieldType : “String” , constraint : [“ACK_RCV”,”ACK_DLV”],  groupName : “MSGQ” }
  2. Create the UI dynamically configurable rules
    1. Convert the rules into Json models
    2. Leverage ng-models of AngularJs to create the models dynamically from configurable Json scripts
  3. Let the user pick the required fields and required sections in UI.
    1. Render the UI fields bound to ng-models
    2. Let user skip / select the model-bound UI sections
  4. UI spits a Json model on the fly
    1. based on the user selection , generate the final Json payload
  5. The model contains sub-structures which are  processed , analyzed , enriched in parallel and finally persisted in document database.
    1. Nodejs offers fast asynchronous , non-blocking, parallel execution of operations
    2. Let the Json payload be synchronized with the data in DB.
    3. Leverage all fast async modules like socket.io , elasticsearch-client , msg queue for real-time status updates, auditing, logging and monitoring.
  6. The entire stack ( UI > Server > DB ) speaks one language ~ javascript and exchanges one format of data ~ json.
    1. MongoDB is cloud-ready, scalable DB allowing fast sharding , in-memory lookups and concurrent reads.
    2. Relief from rigid schema and strict relationships.
    3. Using NoSQL DB makes it easy to implement parent child relationship , hierarchies, dependency graph and composition relationship
  • So think in NoSQL , represent Business Entities as documents ,  analyze texts using Elastic Search and build fantastic time-series stats in Kibana, send alerts to Slack channels and keep your customers Happy 🙂