I’ve long been aware of the practice of using feature flags for things like releasing features more gradually or temporarily turning off a feature that’s causing issues. But until I started working with the Flipper team, I really had no understanding of just how much they can enable better workflows and release processes.
To be fair, there are plenty of potential downsides with feature flags if they aren’t used carefully, but considered use can significantly improve how we build and release software.
While they can be used in many different ways as temporary flags, permanent flags, circuit breakers, or a variety of other scenarios, it all boils down to de-risking feature launches by enabling smaller iterative releases that can be gradually enabled in order to make releases much less intimidating.
For us, that’s exciting enough that we even named the Flipper blog “the Friday Deploy.” And it’s not just about being funny. Last Friday afternoon, we released a sweeping change to Flipper Cloud’s plans and subscriptions by adding free accounts for everyone. After initially toggling it on, we found a couple of minor bugs, so we quickly turned it back off and fixed the bugs. An hour later, we turned it back on and never looked back.
By strategically applying feature flags, what would have been a terrifyingly large change with widespread implications turned into an almost whimsical case of turning it on because the vast majority of the code had been in place in production for weeks.
We view feature flags as a way to make our development and releases processes run more smoothly and be significantly less risky. We’ve pulled together all of the ways that feature flags can make releases much less anxiety-inducing.
Avoid Long-running Branches
Putting new functionality behind a feature flag makes it much less risky to build out new functionality in smaller pieces while ensuring that any new code won’t affect anything until you’re ready to turn it on.
In fact, this is precisely how we added a free plan to Flipper Cloud. We built out the structure for entitlements, added the logic related to free plans, and then implemented the limits and restrictions for the free plan one-by-one. Through the whole process, we were releasing working code over the course of multiple smaller pull requests until we were ready to flip the switch.
Roll Out Incrementally
Avoid performance surprises by slowly rolling out new features to a percentage of users, organizations, specific countries, or whatever segments you define. By exposing new features to smaller groups of your customer base, you’re more able to find and correct performance or usability issues before they affect all of your customers.
This is also particularly useful when rolling out cache changes. Changing a caching mechanism to improve hit rate is great, but starting with a completely cold cache is not. Using incremental roll out, you can slowly "warm" cache changes.
John used this approach at GitHub when they moved memcached off the git file servers to a dedicated cluster. Over the course of a few days, he ramped up mirroring keys to the new cluster from the app using a feature flag. The result was little to no performance impact on github.com despite moving terabytes of cache.
Temporarily Turn Features Off
Web applications have grown increasingly complex. Often times, one portion of an application might be struggling. Maybe it’s search or another feature that can be disabled without bringing down the entire application.
With feature flags, you can disable the service that’s failing while leaving the rest of the application up in order to minimize the impact to end users. Feature flags make shedding load like this a simple click.
Quickly Enable a Backup Provider
As applications grow, it’s often useful to have backup providers for critical services like email. If your application’s ESP encounters issues and temporarily stops delivering email, you can use feature flags to switch over to a backup provider until you’re comfortable that your primary provider is fully back up and running.
if Flipper.enabled?(:postmark, user)
# send via postmark
elsif Flipper.enabled?(:sendgrid, user)
# send via sendgrid
else
# no provider... what should we do
end
# postmark will be used
Flipper.enable :postmark
# turn postmark off and sendgrid on
Flipper.disable :postmark
Flipper.enable :sendgrid
Internal Production Testing
While testing in production shouldn’t be the primary testing approach, enabling only internal team members to use a new feature once they’re ready means that if something does go wrong in production, paying customers won’t be affected.
John mentioned this was a popular use internally at GitHub. They'd typically start by enabling the feature for the team working on it. Then, they'd expand to all of the GitHub staff and eventually roll it out to all GitHub users.
Flipper.register(:employees) do |actor|
actor.respond_to?(:employee?) && actor.employee?
end
# let employees in
Flipper.enable :new_feature, :employees
Provide Early Access for Important Customers
Once internal teams are comfortable with the quality of a new release, it’s easier to gradually expand access to trusted customers in order to get real feedback sooner. By shipping new features to production behind a feature flag, you can enable the feature for specific users or groups of users based on the criteria you select.
John and Steve have been doing this at Box Out forever. In fact, they currently have a new feature that is in beta testing and Flipper feature flags are controlling access. Just a few days ago John enabled the feature for an org from his truck while waiting in line to pick his kids up from school.
Let Customers Opt-in to Changes
Once a feature is ready for wider release, you can provide customers with the ability to opt-in to the new feature on their own timeline. That way, you minimize disruption by letting your users decide when they’re ready to become familiar with a drastic interface change and let your most eager users try it as soon as it’s ready.
Steve has used this method at Box Out Sports a few times – once for a complete redesign, which took months from start to finish. He used a feature flag to change the view paths used to render. At his own pace, he worked through all the views all the while deploying with no risk to customers. When everything was complete, many pull requests and deploys later, he allowed customers to opt in to the new design.
Reduce Risk of Launches
When your feature has already been in production for weeks behind a feature flag, there's no need to wait for last-minute deploys. Toggle it on with a single click. Then if something goes wrong, you can turn it off just as easily. So instead of large releases to multiple systems, you can have the code ready to go and handle releases or rollbacks without having to redeploy.
John can remember one particular GitHub Universe where eight Flipper feature flags were enabled in chat consecutively as Chris Wanstrath announced them on stage. Can you imagine timing deploys for that?
Control Permissions and Entitlements
Whether limiting access to premium features or giving teammates and contributors access to administrative features, feature flags can handle permissions and entitlements for users and groups.
John actually uses a flag named :administrator for small projects with an admin area that only a few people can access. Using a flag to enable access means it can be done at runtime, without needing to have another database column and a form to update or code change + deploy.
class User
def admin?
Flipper.enabled?(:administrator, self)
end
end
# make someone an admin (or do it from the Cloud UI)
Flipper.enable :administrator, user
Block Bad Actors
Flipper's feature checks are always blazing fast, so you can quickly and easily block bad actors that are abusing your app and rest assured your app is unaffected. Recognize someone using too much bandwidth or accidentally making too many requests? Add them to the list of throttled users, and their access an be progressively restricted.
def call(env)
user = user_from_env(env)
return throttled_response if Flipper.enabled?(:throttled, user)
# continue processing request...
end
# Throttle a user
Flipper.enable(:throttled, user)
Perform Maintenance
Planned or not, feature flags let you limit downtime by disabling only affected features while leaving the rest of the application fully accessible. Search service offline? Turn it off temporarily and redirect folks to an explanatory page. Then you can work on a fix without worrying about ripple effects or larger downtime with more critical services.
SpeakerDeck actually uses this pattern for its search functionality.
class SearchController < ApplicationController
def index
unless Flipper.enabled?(:search, current_user)
respond_with_error :disabled
return
end
# continue processing search request...
end
end
Run Experiments
Preview how a change affects real users in production by enabling an experimental feature for a percentage of users and then measure the results. Is it slower? Are customers still able to perform their tasks as efficiently as they could before? By learning and experimenting in smaller batches, it’s easier to improve and make the right calls before releasing new functionality.
Sunset Old Features
Sunset features the same way you roll them out. Once a feature is disabled, you can untangle and remove it from the code base without risk. Or you can disable a feature for everyone but keep it around for a handful of customers that still need it.
Synchronize Releases Across Services
Have multiple systems involved when releasing new features? Instead of synchronizing multiple releases, deploy the code to all of the systems separately, and then enable the feature from one place so all of the systems can cut over or roll back at the same time.
It’s About Less Stress
Running web applications has become increasingly complex, and managing change can be challenging both from a process and customer satisfaction standpoint. Feature flags make it less painful to make cumulative smaller bets instead of building extensive new functionality that introduces a lot of change and complexity in a single release.