October 28, 2021

The story of an 18-hour launch

Everything that could go wrong did

A few days ago, my team at CubaPod and I decided to launch our podcaster dashboard to support podcast creators on our platform. This service had been in the making during the month of August but, to be honest, it was developed in just a week. We were just waiting until we hit that 100-podcast milestone on the platform.

Days went by, and we decided it was time to launch. No more waiting.

I was anxious. We had the pressure of settling on a launch date; August was almost through, and I had announced the month before that we would be launching an important product before September.

On the evening of August 27 at 11 pm, I began the migration process. “This will be easy,” I said to myself, “I just have to migrate CubaPod to a new server, install some new services, and launch.” I couldn’t have been more wrong.

The instructions on the launch guide were precise, the fixtures were ready, all the branches in the source code were up-to-date. I only had to follow the steps. Enter Murphy: everything that could go wrong did. I spent the entire night and the entire following day launching and stabilizing the application.

We were down for 18 hours! That’s how long it took!

During that time and battling exhaustion, I had to code 36 commits, 6 of which were bug fixes. For each bundle of commits, I had to rebuild all the Docker images on the server, restart the processes, and conduct manual tests. No, we don’t have CI/CD yet; we’ve been developing fast, devoting almost no time to writing tests (which is not good practice).

I needed three cold showers and five cups of coffee just to stay awake. My team kept telling me to go to sleep, but I couldn’t do it. The users were waiting, and the downtime was getting too long.

It was dramatic, but nobody knows that. The users don’t know that. Neither does the press. But now you do. You are a member of a select group of people who got to know the real story.

Those 18 hours of intense work resulted in a better release of our product. That wouldn’t have been possible without determination and resilience. Now, the product is online and fully operational. A week later, 24 podcasters have registered and are using their new dashboard. A lot of happy people have thanked us, and we’ve gotten three mentions in the press (it could be more, for all I know) from media outlets talking about our product. Not bad!

On August 24, at 5:02 pm, we published the official launch announcement and publicized it on every one of our social media channels. Everything turned out fine in the end, and I spent the entire next day sleeping.

Lessons learned from this launch

  • Do not deploy at night. If you’ve spent too many years working at night (as I have), your body is already exhausted and won’t hold out much longer. It’s bad for your physical and mental health. Just don’t do it. Deploy during the day, when you’re working at full capacity with a well-rested mind.
  • Your users won’t always know how to do things, even after you’ve explained it to them in your launch announcement. Newsflash: nobody reads those. You have to force your users to do what they need to do, herding them through your user interface.
  • If your product is good enough, the media will talk about it. But you have to publish at least one official announcement where you explain it. Even if your users won’t read it, journalists will.
  • Originally published on Mar 27, 2019 in Medium
  • It’s fine to wait before launching. Unchecked eagerness can harm your work and turn your success into a failure.
  • Expect the worst and prepare for it. Murphy is just waiting to make a mess out of your hard work.

This story is part of my personal newsletter, where I tell you about products, startups, and projects that I’m into, you’ll get personal direct emails related to ideas, concerns, and thoughts. From failure to success, I tell you the REAL story behind the scenes. Join me, subscribe.

Originally published on Sep 3, 2020 in Medium