Recently just built and deployed an enterprise (on-prem) offering for my startup https://commando.io.
The single biggest problem by far was ripping/replacing 3rd party SaaS services that we use/pay for with native code. While building a SaaS it is wonderful to use 3rd parties to save time and complexity. Examples include Stripe (payments), Rollbar (error tracking), Intercom (support/engagement), Iron.io (queue and async tasks), Gauthify (two factor auth)... Howerver, when you go to on-prem often times the servers don't have a public interface, so your entire application has to work offline.
2nd tip. While it may seem like a good idea to create a separate branch in your repo for on-prem (i.e. enterprise branch), this is bad idea. It ends up being much better to just if/else in the master branch all over the place:
if(enterprise) {
// do something
}
If/else in the master branch is what GitHub Enterprise does as well (at least was told so).
This is great advice. Sometimes services can be removed from the on-prem version (billing), but some (queues, auth) need to find an on-prem compatible replacement. Most of the time, there are reasonable alternatives. But there are a lot of "on-prem" environments that run in AWS, so don't be afraid to start with an AWS-only "on-prem" version.
I completely agree with your advice about using "if (enterprise)" statements instead of a separate branch or repo for the enterprise version. Once you make that enterprise branch, it's tempting to let it sit too long and merging the upstream changes is one of the least-fun, most-dreaded tasks. By using the "if (enterprise)" approach, your enterprise version stays up to date. but it doesn't mean you need to constantly deploy the latest version to all of your customers.
Good advice, definitely want to keep everything in a single branch.
While building the enterprise version of Kumu [1], I found it much easier to maintain if I used the environment to toggle configuration flags rather than scattering if(enterprise) calls throughout the codebase:
BILLING_ENABLED = APP_ENV !== 'enterprise'
if (BILLING_ENABLED) {
// do something
}
I recently made an on-premises version of https://reviewable.io. I'd like to second most everything said in the article: use a flag, remove SaaS dependencies where possible, otherwise convert to use standardized services (SMTP) or apps that they can deploy internally as well (Sentry). It took me rather longer than a week but the ROI was still excellent.
One difference is that I decided to stick with a raw Docker container rather than going with Replicated (who are really nice people, don't get me wrong). In my case, everything runs on a single horizontally-scaled server so it's pretty easy to get running "manually". Also, all the potential customers I asked said that they were comfortable managing their own Docker environment, and some would actually prefer to get Docker images since they'd be easier for them to manage than "virtual appliance" VMs.
Another difference was that I use Firebase for the app's datastore and Google doesn't (and likely never will) offer an on-premises version. This was a show-stopper for nearly all customers, but it turned out that offering at-rest encryption of sensitive data with a key that never leaves their premises was sufficient to get through many of the security reviews. This obviously won't be acceptable to everyone but it can be useful to know that even if you have a fundamental dependency on an external third-party service there may be a way to keep using it in your "on-premises" product.
all the potential customers I asked said that they were comfortable managing their own Docker environment
Can we trade customers? Last year I've had to install our software in a CentOS 5.5 system. With no root access. Can't say compiling Python 2.7 and all that crap to run on a user account was much fun, but hey, at least they're billable hours.
Large, and especially public, companies seem to be the worst. SMBs were happy to receive a VM or OpenVZ image.
No, I actually meant "a single kind of server process that you just run more instances of to scale" but couldn't find a concise yet accurate way of expressing it. This differs from the setup described in the article since they have four different kinds of servers that need to be coordinated, interconnected, etc. -- a more complex setup that likely benefits from Replicated.
Here at Coveralls.io we can't recommend Replicated enough. We were able to migrate our internally-developed, Github Enterprise style packaged VM delivery approach over to their system in a couple weeks.
But I'd say the real value is in not needing to support an additional deployment platform -- their Docker based approach is like Heroku for on-prem, it abstracts the infrastructure details and just works.
Container folks currently advise to not put the database into a container unless you really really really know the internals of both. Even Kelsey Hightower stressed this in a talk - "if you want to keep your job, keep databases out of containers". ReadMe had no prior Docker experience, and they started by putting Mongo (which is complex to manage as is) into a container. This would make me uneasy.
Does Replicated assist with this database/persistence layer? Perhaps by helping you to spin up the DB as a separate VM on the clients' side?
Replicated founder here. Great question. The main concern most people have when considering putting a database in a container is around data loss or corruption. (i.e. What happens if the instance disappears and the data was stored on an ephemeral disk?) We have a couple of features to help with this:
Replicated allows for the end customer to choose if they want to run the containerized version of the database, or provide their own instance of it (not in a container) and simply provide a connection string so the application can use that instance. Generally this is exposed as a checkbox on the settings page, and doesn't require a lot of configuration.
If the end customer decides to install with the containerized version of the database, they can choose to run the database container on a specific instance (or instances) and control the host path of the volume mount(s). The idea here is that they will store the data on an EBS volume, network attached storage etc.
We also have a feature that allows for snapshots and restores to help in the scenario where the database was store on the local instance and there was a hardware failure or corruption.
These features work out of the box, without a lot of effort from the software vendor to configure.
After reading your docs and install scripts, I think another option for us is to run the database directly on the client's host. That is, as part of our installation guide, we'd say Step1:Install Replicated, Step2:Install DB, Step3:Run setup. This might be easier in a trusted environment where containers can be started on the host network. This approach has potential issues with replication, encryption, upgradeability, etc, but those are normal infrastructure challenges and have nothing to do with Replicated.
I can't find any details on Replicated pricing, is it reasonable? It seems like a product which would be nigh impossible to replace once installed at a customer site (making the customer do anything risks cancellations).
It's nice to see some momentum building behind the idea that multi-tenant SaaS doesn't have to be your only deployment. And congrats to the Replicated folks on the recent raise. It's great to see people validating these ideas/offerings. Based on mainstream tech press, you sometimes feel crazy working on something perceived as "anti-cloud".
Nice post from Readme.io, and I also want to shout out to Ben @ Replicated. He reached out to me when we launched our cloud HR SaaS offering, and talked about the possibilities of offering an enterprise on premise version using Replicated.
I haven't moved forward with that yet, but if I do, Replicated will be the team we work with. Ben spent a lot of time on a web hookup teaching me about Docker containers, and even suggesting how we might improve our development environments and processes (without using Replicated anywhere in the solution).
Founder of https://jidoteki.com here - we provide the ability to go from SaaS to On-Premises in less than one week[1] as well, without the need to jump into Docker.
I think ReadMe's approach is pretty good, and I applaud them for making the move.
Our aim is a bit different from others, as we focus exclusively on "actual" on-premises (local virtual machines), as opposed to "on someone else's premises" (AWS).
Thanks for writing about your experiences. Its a weak spot for most of us, and it was a refreshing kick in the butt, to get ourselves in gear about this topic which has been on our mind for a long time.
Its great to see more startup support in this weak area. I really hope replicated/gravitational succeed, and perhaps more articles can be written to help the rest of us reduce the procrastination or fear of taking the steps toward supporting enterprise.
Another shootout to replicated, the service mentioned to go from SaaS to on-premise. We are using it for our own offering at Enthought, and can't recommend it enough either. Good support, reactive to customer feedback. I wish more companies were like that
In case it helps other people using replicated and python together, we put together a small python client to replicated https://github.com/enthought/python-replicated. We are using it internally to test each PRs against a programmatically created replicated release
Are there any plans yet on how to use Replicated on existing on-premise cluster installations, like DC/OS or Kubernetes?
After having a look at the installation shell script, Replicated itself seems to run in containers and therefore should be deployable on DC/OS or the likes if I understand correctly...
Hi toblig. Absolutely. We do have support for installing Replicated on an existing Kubernetes cluster. For these installs, we provide a couple of k8s yaml files to create the deployment, service, etc on the cluster. It's still a beta feature, but we like the idea of supporting other schedulers.
Hi marcc, thanks for your feedback! Is there any documentation on what are the building blocks to be able to run Replicated on platforms like DC/OS and Kubernetes?
I'm especially interested in DC/OS, because my company is building a SaaS in it, and ideally we'd want a general solution on how to ship and maintain an on-premise deployment.
The single biggest problem by far was ripping/replacing 3rd party SaaS services that we use/pay for with native code. While building a SaaS it is wonderful to use 3rd parties to save time and complexity. Examples include Stripe (payments), Rollbar (error tracking), Intercom (support/engagement), Iron.io (queue and async tasks), Gauthify (two factor auth)... Howerver, when you go to on-prem often times the servers don't have a public interface, so your entire application has to work offline.
2nd tip. While it may seem like a good idea to create a separate branch in your repo for on-prem (i.e. enterprise branch), this is bad idea. It ends up being much better to just if/else in the master branch all over the place:
If/else in the master branch is what GitHub Enterprise does as well (at least was told so).