Hachyderm<p>Hello, hachyderm! we've been working hard on building up our ansible runbooks and improving hachyderm's overall resilience. Recently, we've been focusing on is database resilience.</p><p>We're getting close to retiring our original database server (finally!) and preparing to move to a fully ansible-managed set of databases servers, primary and replica on new hardware. We'll send another announcement when we do the cut over. The team has done excellent work to make this highly automated, quick, and painless! :blobfoxscience: </p><p>Done:</p><p>✅ author ansible roles for managing postgresql, pgbackrest (backups), pgbouncer, and primary/replica failover<br>✅ decide to continue with pgbouncer and *not* use pgcat<br>✅ rotate database passwords <br>✅ order new replica database hardware<br>✅ order new future primary database hardware</p><p>To do soon:</p><p>🟨 rebuild replica database with ansible scripts<br>🟨 prepare primary database with ansible scripts<br>🟨 start replicating to new database replica<br>🟨 cut over to new database server 🎉 </p><p>We're also planning on open-sourcing our ansible roles in the coming weeks - just a little housekeeping & tidying up before we do!</p><p><a href="https://hachyderm.io/tags/hachyderm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>hachyderm</span></a> <a href="https://hachyderm.io/tags/devops" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>devops</span></a> <a href="https://hachyderm.io/tags/sre" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>sre</span></a> <a href="https://hachyderm.io/tags/ansible" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ansible</span></a> <a href="https://hachyderm.io/tags/postgresql" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>postgresql</span></a></p>