Monday, April 2, 2018

The Story of Hardships and Triumphs - Breaking and Rebuilding Onerent's Platform


Summary: Onerent Engineering made it! We've crafted the new and enhanced platform that supports every customer, client and staff's need. Mission accomplished!

Have you tried building a platform from scratch that requires a heavy interaction with a CRM? How was the first release of the MVP?

In Onerent Inc., we've been busy crafting the new hybrid system, to have a revolutionized platform in order to enhance the user experience (both internal/external). As a startup company, innovation is what we always keep an eye on. While the objective is clear, trying to combine two different systems to glue an ultimate solution -- is not a straight and flat road. It wasn't easy but we managed to work-it-out and we know it's still a long way to go for bug fixing and feature request.

As we share our experience to the world, we hope that it gives everyone a better understanding about system architecture in general. We've spent so many hours of planning and weekend hackathons just to fail fast and apply what we've learn along the way. We specifically allocated time to test our hypothesis and ideas, thus, had caused us a long WIP (work in-progress) hours compared to what we normally consumed and accept in the engineering department. Discoveries and learning is so valuable that we never feel bad about the huge time investment in R&D (research and development).

Weekend Hackathon, a crucial planning session..
The mission impossible
Our main objective as we start our mission on rebuilding the Onerent Platform is to drive efficiency and apply innovation to all operational processes. The legacy system has served its purpose and this year, it's all about transformation.

What do we know about this mission?
  • We will use Salesforces as our CRM
  • We will use Nodejs, GraphQL, React/Redux, PostgreSQL and Heroku Connect to our mainstack
  • We will use AWS for our infrastructure

What do we care about on this mission?
  • Scalability
  • Efficiency
  • Usability
  • Product-Centric
  • User-Centric
  • High-Availability
  • Data-Driven

The warlords and champions
In a critical mission, you don't want people messing up the objective. Selecting the people to work on the project/component/tasks needs to be knoweldgeable about what he's doing or at least know how to ask if he's lost.

For our case, there were 5 major teams involved (1) Mainstack Engineering (2) Salesforce Engineering (3) DataOps  (4) Marketing (5) Business Operations. Each giving out there best, performing the work they will brag to the world.

How does these department play the role in the crafting process?
  • Business Operations - Where used cases are opened and gathered.
  • Marketing - Where UX/UI is studied and designed (includes context and optimization)
  • DataOps - Where infrastructure are planned and estimated
  • Salesforce Engineering - Where internal processes used cases are being translated to work pipelines
  • Mainstack Engineering - Where external processes used cases are being translated to product feature/component
With the teams that we have, we're aware that we've pretty much covered everything. And ofcourse, our Business Executives and Advisors that gives us the "strategic plan" for execution.

The baseline (magic number)
With the objectives set and teams being rounded up, setting the baseline as to "when we can say that the mission was a success or a complete failure" is crucial. Most startup companies die for not doing good with time projection (..timing is important) and they've only realized it on the 11th hour which gives them no time to alter the course (too late reaction).

We don't want to commit the same mistakes as the others, so we keep things on a tight deadline but realistic. With the commitment of everyone who are involved in this mission, by a rough estimate -- we were given 6months to craft the improved platform. Shocking? While 6months is kinda challenging knowing the scope of the project, no one ever doubted what we could do (we are just very excited to see what we can offer to our customers/clients).

How was the 6months consumed (outside development)?
  • Hiring more people for Salesforce Administration/Development
  • Added 2 more heads to work on Mainstack
  • On-boarding the 2nd hire for UX/UI (Marketing)
  • On-boarding our QA Engineer
  • Scouting for data-scientist
Within the time period, we are all aware that the holiday season was fast approaching and people are looking forward for the vacation. We don't have other chance if we won't make it to the 6months period, it was a "do or die" scenario we need to face.

The humps and bumps, things are now on fire
First weeks of development shows a great progress for everyone in different team. Things are rolled out smoothly, ideas being discussed, different solutions being evaluated, different technology being tested and meetings has been covering most of the things we needed.

Well, that was on the first few weeks...

Days past and slowly, the momentum has been shifted. Neither of us realize if we got everything covered and if we will be able to come-up with the "platform" everyone -- from our customers, clients and staff has been waiting for. Though, everyone is still aiming for the deliverables.

Some big blocks of learning we've faced, along the way:
  • Challenges on Marketing
    • Migration of Web Content (Improving Context as we migrate to the new platform)
    • Retaining SEO standing on the new platform
    • UX/UI Improvement (time-bound delivery)
    • Branding and Style Guides
  • Challenges on Mainstack Engineering
    • Technical debt on Salesforce architecture and process workflows
    • Technical debt on Heroku-Connect
    • Technical debt on the Payment System (Workflow)
    • Technical debt on the legacy system architecture
  • Challenges on Salesforce Engineering
    • Technical debt on the business model and process workflow
    • Technical debt on best Salesforce practices and standard approach
    • Insufficient workforce (lack of manpower)
  • Challenges on DataOps
    • Data Sanitation
    • Data Engineering (source of truth)
    • Data Migration (tooling)
    • Salesforce (architecture) Triggers and Required Fields
    • Heroku Connect (Usage and Best Practices)
  • Challenges on QA
    • Time-Bound Deadlines
    • Technical Debt on System Architecture
    • Tooling and Automation

For a team that's well oriented with the work that needs to be done, no blocker can ever hinders what needs to be delivered. Everyone who'd been part of this digital transformation has made a statement, "Challenge Accepted!" loud and clear.

Controlling the quality, delivering the fix
This is very common to every organization, though, no one ever had the chance to perfect it. Here in Onerent, we believe that there's no perfect system. As a system is only good as the business processes and models -- it keeps changing for it to be innovated and improved. Quality is king for us and we only care on the things that brings value stream to the table.

We have a list of what should not break and what people can play and hack around. We do not restrict everyone to break things to improve it, rather we tell everyone to "challenge" the system for it to be improved.

Things that should not break:
  • Payment System and Database
  • Mainstack System and Database
  • Salesforce
  • Heroku Connect (Production Env)
  • DNS (FQDN)
  • Mainstack Platform

Things people can play and hack around:
  • Everything inside Staging Infrastructure (Application/Database)
  • Microservices
  • Data Analytics (Models and Toolsets)
  • Wordpress Landing Pages
  • Monitoring (SRE)
We are very proactive on Site/Service Reliability and Scalability. Thus, these make us set all this guiding rules in the operations.

Claiming the throne! Rewards and Shoutouts!
In reality, we didn't made it to the 6months period. However, we've managed to nail it down on the 8th month from the time the project started. On the additional two months, we've included the "Transfer of Knowledge" about Salesforce to our Operation's people as we don't want our frontliners to face issues navigating the new dashboard.

Architectural Beauty, Onerent's Stack
Here's how our stack looks like from the application layer. Grouped by component, the 1st on the diagram is the Frontend Stack, composed of Nodejs, React and Apollo. The 2nd one is the Backend Stack, composed of Nodejs, Redis and Graphql. The last on the diagram is the Data Stack, composed of Postgresql, Heroku Connect and Salesforce.

Overall Transformation, An Improved Onerent
The changes we've performed is not only limited to the Mainstack and Salesforce Dashboard, we've also included a total makeover of the Onerent website, from our Landing Pages to Blog Pages. Bringing more on-boarding and user interaction in the table.

In about 24hours from now (04-02-2018 7:16PM Asia/Manila), Onerent's makeover will be available to public.

Thanks to everyone's effort for making this project a success! Onerent is king!
My next blog will probably about how we celebrated this massive success! In Onerent, we work hard and party harder..

We launched 2 days delayed to address some hiccups on our implementation. Today, it's live!