Mistakes were made¹, and after recent maintenance, Jordan asked Infrastructure Team if there is a chance GNOME Nightly (and its younger sibling GNOME OS) could be moved somewhere with faster network and more storage. After short discussion, we decided to move it to AWS (thanks to promotional credits for open source projects). The new server has three times more space, uses SSD instead of HDD, and should offer up to 10 Gbps of network bandwidth.

It quickly became apparent we underestimated GNOME Nightly popularity by a large margin. On the second day, the new server transmitted almost 500 GB of data. As we wanted to limit potential egress costs, we reached out to CDN77, and they quickly agreed to help us. CDN77 team aren’t stranger to the open source world, helping other projects like KDE, Gentoo and Manjaro.

With the new server fully dedicated to GNOME Nightly, and turbo fast CDN in front, I can easily saturate my 1Gbit connection at home while downloading the infamous org.gnome.Sdk.Debug//master. Since the switch on January 22nd, CDN77 has served for us over 40 million requests, which sums up to 11 TB of traffic. While it is well-known nobody notices infrastructure until it’s broken, here’s hoping it just became more invisible.

Thanks to AWS and CDN77 for making it possible!

¹ Slow IO caused by spinning disks aside, the implementation of repository pruning in Flatpak before the 1.11.1 release was prone to remove actually required ostree objects if a new build was being published at the same time.