Photo by Ray Hennessy on Unsplash

Big changes coming in Horde 0.8.0

By: Derek Kraan / 2020-09-03

This week I released the latest version of Horde, 0.8.0. This version has been a long time coming, with the last minor release (0.7.1) having been released 10 and a half months ago. 0.8.0 brings with it some fairly large changes, so let’s dive right in!

If you don’t know what Horde is, the README does a good job of explaining what it is!

edit: 0.8.1 has just been released with the new function Registry.delete_meta/2 that will appear in Elixir 1.11.0

members: :auto

Thanks to contributor Bernard Duggan, Horde now supports automatic membership, making sure it always reflects the nodes that are connected in your cluster.

Before, it was necessary to call Horde.Cluster.set_members/2 when a new node was added or removed from the cluster. Now, if you provide the option members: :auto to Horde.DynamicSupervisor or Horde.Registry on startup, it will monitor cluster changes for you and automatically adjust membership accordingly.

members: :auto makes the blunt assumption that you are running a process with the same name on every node. If you start Horde.Registry.start_link(_, _, name: :my_horde_registry) on every node in your cluster, then members: :auto will work for you. If you only start :my_horde_registry on a subset of nodes in your cluster, or have some other elaborate configuration, then members: auto will not be suitable for your needs and you’ll need to continue calling Horde.Cluster.set_members/2 manually.

Read more about this in the Pull Request.

Horde.DynamicSupervisor netsplit behaviour

Starting in 0.8.0, the behaviour of Horde.DynamicSupervisor has fundamentally changed. Previously, Horde.DynamicSupervisor was secretly tracking an id for each process, and deduping child processes based on these ids across the cluster. The problem with this approach is that in the most common set up, Horde.Registry was also deduping processes in the cluster. This caused a condition where if two processes, A and B, were in conflict, Horde.DynamicSupervisor could potentially choose process A, shutting down process B, while Horde.Registry could choose process B, shutting down process A. The result was that processes could disappear from the cluster. That’s probably not what you want.

Fundamentally, it can not work to have two separate mechanisms for enforcing process uniqueness in a cluster. In fact, Elixir.DynamicSupervisor does not have any mechanism for enforcing uniqueness, and it discards child_spec.id even if you set it.

So starting in 0.8.0, Horde.DynamicSupervisor will not check uniqueness of its child processes across the cluster. If you need that, you need to use Horde.Registry. In practice, this means that if you do not use Horde.Registry, and your cluster experiences a network partition, when the partition heals, you will end up with multiple versions of each child process. This tracks a little better with how Elixir.DynamicSupervisor and Elixir.Registry are used in practice. Even though Elixir.DynamicSupervisor doesn’t have to consider the consequences of a netsplit, it turns out that emulating its behaviour more closely is better in such a scenario.

Please do consider what this change means for your application, as this is a fundamental change in the way that Horde.DynamicSupervisor works.

If you use Horde.DynamicSupervisor + Horde.Registry, then this change likely fixes a bug in your system (that you may or may not have been aware of).

If you use Horde.DynamicSupervisor on its own, then you will likely need to add Horde.Registry in order to continue enjoying the same functionality as before.

Read more about this in the Pull Request.

Process rebalancing

Thanks to contributor Alex Pilafian, this often-requested feature is now a reality in Horde. Previously, Horde would only move processes between nodes when a node was removed from the cluster or had crashed, or the cluster lost quorum. Now, you can set the option process_redistribution: :active to have Horde automatically rebalance processes in the cluster when nodes are added. The default value is :passive, which preserves the previous behaviour.

There are performance implications in using this feature, especially if you use rolling deploys it has the potential to cause a lot of process churn, so carefully consider the pros and cons when turning this on. Especially for workloads where processes are short-lived, this option could end up doing more harm than good.

Read more about this in the Pull Request.

Conclusion

Horde 0.8.0 brings some big changes to the table. With this release, we are setting a good foundation for reaching 1.0.0 in the future.

Finally, I would like to take this opportunity to shamelessly remind readers that the team that brought you Horde and other related open source libraries offers consulting services and is currently available for new projects starting immediately. Our experienced team of 3 developers based in the Netherlands is specialized in Elixir and can help you accelerate development and solve your trickiest problems. Contact us using the form below and we’ll get back to you as soon as possible.

Drop us a line

Get the ball rolling on your new project, fill out the form below and we'll be in touch quickly.

Recent Posts

Big changes coming in Horde 0.8.0

By: Derek Kraan / 2020-09-03

Highlander, there can be only one

By: Derek Kraan / 2020-04-23

Where do I put startup code in Elixir?

By: Derek Kraan / 2019-12-06

Walkman - isolate your tests from the world

By: Derek Kraan / 2019-07-22

Introducing MerkleMap: improving Horde's performance

By: Derek Kraan / 2019-05-20

What's new in Horde v0.5.0

By: Derek Kraan / 2019-05-06