“It’s a good problem to have.”

“…but not when you’re having it.”

We were offered this advice by many different people when discussing the scaling issues we may run into when creating such a distributed web application.

The idea is that it would be a good problem because it would surely indicate that we were popular, thriving, and thus successful. But it isn’t a good problem for all of you that are users of Disqus. It isn’t a good problem for us at Disqus, as we’re building our service based on trust, performance, and reliability.

We had some downtime this morning, starting at 7am PST, for a little under 30 minutes.
We are sorry. I am personally sorry. Being a popular service is not a valid excuse for being unavailable when other websites depend on you.

So, what are we doing about it? Lots. We have spent a lot of time recently optimizing and tweaking. Disqus is in transition-mode to upgrading our web and database servers (plus a few other misc. infrastructure points).

Bonus apology: This week, email us with the subject “Downtime makes me sad” and we’ll send over some cool Disqus swag. Be sure to indicate your blog/website and share a sentence or two about your experiences with Disqus.

Thanks all — it’s not a good problem to have and we’ll do our best to make sure it doesn’t become a problem you have to deal with.

Daniel on April 23rd 2008 in disqus

  • Glitches and growing pains will be inevitable. So what if my comments are unavailable for 30 mins. That's OK.

    The world won't stop spinning and I won't get innundated with email complaints. Although you might want to move Fred Wilson onto a dedicated server :-)

    At least, you come out with a prompt status update and an apology including the word 'Sorry' (in bold).

    Those guys from Twitter should take note.

    Onwards and upwards !
  • you guys are awesome. keep up the great work and great product.
  • Thanks for the update and the swag. No big deal about the downtime.

    =]
  • Downtime is almost always harder on you than it is on your customers. Take it seriously, but don't take it too hard; hopefully you're learn something and improve from each one.

    It's going to happen again in some form or another; the best you can do is be prepared (backups, redundancy, etc..) and make sure you're always designing with reliability and scalability in mind.

    Keep up the good work!
  • Sounds great guys, keep up the good work. No need for an apology, although the cool Disqus swag might not be too shabby =D
  • Free stuff is good. I'm a dev, so I understand downtime, but apologies do mean a lot. So thanks :)
  • Having problem is one aspect. The other aspect is acknowledging the problem and admitting it. All other problems just get washed off once solving the problem becomes your priority. Great going...We have the faith...Downtime is manageable; Reputation is not...You have managed your reputation...Downtime will automatically be managed. That's how the innovations about scalability and reliability will be taken care of.
  • i just want shwag even ythough I am happy!
  • you guys are awesome. keep up the great work and great product.
  • Thanks for the update and the swag. No big deal about the downtime.

    =]
  • Yeah, Downtime is no real problem, if you inform the users and try to fix it as soon as possible.
  • i just want shwag even ythough I am happy!
  • The one thing I've learned from several way-too-long downtime issues is to have a separately-hosted site for status updates. And make sure everyone knows that site.

    The first major downtime we had at my old old company, everything was hosted in the same facility, including email, and there was ZERO way to get the word out. Except to call people... which is what we had to do.

    I learned that as long as you're really good about communicating... then people give you a break.
  • Yeah, Downtime is no real problem, if you inform the users and try to fix it as soon as possible.
  • When a company runs into technical problems, there are the technical issues, and then there are the business issues. In two cases today (the other one being Aaron Forgue's GroupTweet issue), the principals took immediate and decisive action from both a technical and a business standpoint, letting their customers know that they had a problem, they were very sorry about the problem, and that they were going to make right with their customers.

    Andy is right; there's another problem-plagued company that should be taking note.

    I've written about this at http://mrontemp.blogspot.com/2008/04/theres-sup...
  • Sounds great guys, keep up the good work. No need for an apology, although the cool Disqus swag might not be too shabby =D
  • What? No comments for 30 minutes. Its all good. Everything is recovered and these growing pains are good ones to have!

    Plus, who doesn't like free schwag?
  • Hi
    How we do "dısqus comments" system translation TURKISH language
  • Hi
    How we do "dısqus comments" system translation TURKISH language
  • Aw, sorry I missed the swag offer earlier. I was, indeed, sad! But I know you're working on it and I appreciate your work.
  • Hey, guys u are doing very good.........keep up the good work........
  • Hey, guys u are doing very good.........keep up the good work........
  • Hey, guys u are doing very good.........keep up the good work........
  • It is inevitable for popular service to be experiencing downtime, but not too frequently.
  • It is inevitable for popular service to be experiencing downtime, but not too frequently.
  • SEO
    It is inevitable for popular service to be experiencing downtime, but not too frequently.
  • nice article! nice site. you're in my rss feed now ;-)
    keep it up
  • nice article! nice site. you're in my rss feed now ;-)
    keep it up
  • Sorry is the hardest word to say but then if it is easier said, then its won't be happy problem.
  • Nobody is perfect. Everybody makes mistake or fumble sometimes. Important thing is to go into damage control mode and make sure it does not happen again.
  • Nobody is perfect. Everybody makes mistake or fumble sometimes. Important thing is to go into damage control mode and make sure it does not happen again.
  • Nobody is perfect. Everybody makes mistake or fumble sometimes. Important thing is to go into damage control mode and make sure it does not happen again.
  • SEO
    Everything has some pros and cons and if people are finding any problem with Discus than it doesn’t sound new.. But I guess if they have apologized than nothing is more better than this.. I used it and find it awesome as it gives us a new and easy way to get connected with lots of people, make interaction with them and then easily post blogs in your related domain or interest..
  • Carole J Takeshita
    great article!, grats for u site :)
  • Patricia W Olson
    tu blog es excelente! te mando 454 felicitaciones!
  • Patricia W Olson
    tu blog es excelente! te mando 454 felicitaciones!
  • ezditto
    I'm glad that downtime dont' happen frequently
  • Duple Ang
    Hope that you all did manage this well!
  • replicaloh
    I think nobody will blame you all for the downtime with the bonus apology in return!
  • Jasmine Lim
    the bonus apology seems cool!
  • Annie
    We always have back up plans in case of power failures or things that might happen suddenly. I am wondering isn't there a back up plans for down server or some other issues to curb downtime issues? I am not trying to get Disqus responsible for anything but this is my thought for backups
  • Annie
    We always have back up plans in case of power failures or things that might happen suddenly. I am wondering isn't there a back up plans for down server or some other issues to curb downtime issues? I am not trying to get Disqus responsible for anything but this is my thought for backups
  • We always have back up plans in case of power failures or things that might happen suddenly. I am wondering isn't there a back up plans for down server or some other issues to curb downtime issues? I am not trying to get Disqus responsible for anything but this is my thought for backups
  • Ryan Low
    I am not caught in the down time but it seems to me that it is not an issue to most people here as from what I see from the comments. Disqus always have great supporters out there.
  • ryanlow
    I am not caught in the down time but it seems to me that it is not an issue to most people here as from what I see from the comments. Disqus always have great supporters out there.
  • jonaw
    Down time is quite serious setback to certain companies or users. As the time wasted are unpredictable. What Disqus could do is to compensate the users with more features and implementations. I believe all will fine if the down time is caused by better add-ons coming up.
  • We always have back up plans in case of power failures or things that might happen suddenly. I am wondering isn't there a back up plans for down server or some other issues to curb downtime issues? I am not trying to get Disqus responsible for anything but this is my thought for backups
  • ryanlow
    I am not caught in the down time but it seems to me that it is not an issue to most people here as from what I see from the comments. Disqus always have great supporters out there.
  • ozn
    How to use Turkish Disqus
  • ozn
    How to use Turkish Disqus
  • ozn
    How to use Turkish Disqus
  • Head to http://disqus.com/admin/settings/ and choose your Turkish
    under Languages.

    Giannii
    DISQUS
    Community Manager
    help@disqus.com
    http://twitter.com/giannii
  • Head to http://disqus.com/admin/settings/ and choose your Turkish
    under Languages.

    Giannii
    DISQUS
    Community Manager
    giannii@disqus.com
    http://twitter.com/giannii
  • ozn
    How to use Turkish Disqus
  • Head to http://disqus.com/admin/settings/ and choose your Turkish
    under Languages.

    Giannii
    DISQUS
    Community Manager
    giannii@disqus.com
    http://twitter.com/giannii
blog comments powered by Disqus