Recent Site Instability Issues - August 2011

Discussion in 'Admin Announcements' started by Bob Hubbard, Aug 9, 2011.

  1. Bob Hubbard

    Bob Hubbard Staff Member

    • Administrator
    • MartialTalk Fan
    • Founding Member
    • LifeTime Supporting Member
    Joined:
    Aug 4, 2001
    Messages:
    47,023
    Likes Received:
    608
    Trophy Points:
    248
    Location:
    Land of the Free
    Some of you may have experienced some access issues the last few days. Database errors, pages not loading, things timing out.

    Well, after some digging we figured out part of the problem. We're seeing a large spike in traffic, resulting in us maxing out the server we're on. Right now, a major server upgrade isn't financially possible. With the down state of the economy, there isn't another $200 a month in the operating budget to cover the cost increase of a new box. Right now, we're doing some intensive performance tuning. Hopefully you're already seeing some faster page loads and quicker access. I'm going to be evaluating this over the next few days and determining what other options we have. We may add additional memory to the server, which will hopefully boost things a bit as well. If we do, we'll post a notice at least a day or 2 in advance. We're also going to be performing some additional software updates in the near future so we may have some down time then as well. I'll announce that in advance.

    I apologize for the inconvenience, and want everyone to know we're working on it.
     
  2. Big Don

    Big Don Sr. Grandmaster

    • Supporting Member
    Joined:
    Sep 2, 2007
    Messages:
    10,551
    Likes Received:
    185
    Trophy Points:
    173
    Location:
    Sanger CA
    Oh...
    I thought it was on my end. I was blaming my ISP.
     
  3. Carol

    Carol Crazy like a...

    • Supporting Member
    • MartialTalk Fan
    • MartialTalk Mentor
    • LifeTime Supporting Member
    • Martial Talk Alumni
    Joined:
    Jan 16, 2006
    Messages:
    20,225
    Likes Received:
    470
    Trophy Points:
    193
    Location:
    NH
    *whew*

    I noticed it too and kept forgetting to drop you a line. I'm glad its "just" an overloaded server and not database-go-boom. :)
     
  4. Bob Hubbard

    Bob Hubbard Staff Member

    • Administrator
    • MartialTalk Fan
    • Founding Member
    • LifeTime Supporting Member
    Joined:
    Aug 4, 2001
    Messages:
    47,023
    Likes Received:
    608
    Trophy Points:
    248
    Location:
    Land of the Free
    We had over 5,000 visitors today. Ran us out of free memory (2GB) a few times which caused huge load on the drives (SCSI) and processors (AMD Athlon(tm) 64 X2 Dual Core Processor 5000+ 2.6GHZ).

    When your system (server or desktop) runs out of real memory, it'll usually use your hard drive as 'virtual memory'. This is slower and causes your processor to work harder. Solution is to add more RAM and fine tune things.

    All else fails, we feed the hamster some crack and run. :D
     
  5. The Last Legionary

    The Last Legionary All warfare is based on deception.<br><b>nemo malu

    • MartialTalk Fan
    • LifeTime Supporting Member
    • Martial Talk Alumni
    Joined:
    Oct 26, 2005
    Messages:
    1,040
    Likes Received:
    90
    Trophy Points:
    73
    Location:
    Isle de la Moros
    Seems a bit quicker. I'll go harass a few folks, see how quickly it bans me. ;)



    I kid. I kid. :roflmao:
     
  6. Bob Hubbard

    Bob Hubbard Staff Member

    • Administrator
    • MartialTalk Fan
    • Founding Member
    • LifeTime Supporting Member
    Joined:
    Aug 4, 2001
    Messages:
    47,023
    Likes Received:
    608
    Trophy Points:
    248
    Location:
    Land of the Free
    So far, no new error messages, and page loads feel faster than on KT or FMAT.

    Hows it looking on everyone elses end?
     
  7. MaxiMe

    MaxiMe Brown Belt

    Joined:
    Mar 3, 2011
    Messages:
    496
    Likes Received:
    9
    Trophy Points:
    18
    Location:
    San Diego, CA
    Seems better on my end today.
    RAM Gotta love quick and fairly inexpensive fixes :)
    Tuning the page file on the server might help a little (if you haven't already done that).
     
  8. Bob Hubbard

    Bob Hubbard Staff Member

    • Administrator
    • MartialTalk Fan
    • Founding Member
    • LifeTime Supporting Member
    Joined:
    Aug 4, 2001
    Messages:
    47,023
    Likes Received:
    608
    Trophy Points:
    248
    Location:
    Land of the Free
    we installed memcache and tuned the php config a bit. adding more ram is pending.
     
  9. Carol

    Carol Crazy like a...

    • Supporting Member
    • MartialTalk Fan
    • MartialTalk Mentor
    • LifeTime Supporting Member
    • Martial Talk Alumni
    Joined:
    Jan 16, 2006
    Messages:
    20,225
    Likes Received:
    470
    Trophy Points:
    193
    Location:
    NH
    Wasn't it your turn to buy the hampster chow? :p
     
  10. MaxiMe

    MaxiMe Brown Belt

    Joined:
    Mar 3, 2011
    Messages:
    496
    Likes Received:
    9
    Trophy Points:
    18
    Location:
    San Diego, CA
    Faint echoes of a rim shot. badda bing
     
  11. RandomPhantom700

    RandomPhantom700 Master of Arts

    Joined:
    May 19, 2004
    Messages:
    1,583
    Likes Received:
    69
    Trophy Points:
    73
    Location:
    Treasure Coast, FL
    Bob:

    I do not know if this is a related issue, or if it's a problem with my browser as opposed to the site's server.

    However, I do not have automatic login set on my account, erego I type in my information each visit. I noticed this morning, upon initially bringing the site up, the text in my login/password would remain gray when typing (it is normally black while being typed), and more importantly, my password would not be blocked as I was typing it. Upon hitting submit, the page reloaded without logging in, and things would work normally upon the second attempt, with password properly blocked by bullets.

    I've seen this twice this morning, and like I said, it could very well be a browser glitch on my end or it could be a server error. In either event, I wanted to let it be known in case anyone else has observed similar.
     
  12. Bob Hubbard

    Bob Hubbard Staff Member

    • Administrator
    • MartialTalk Fan
    • Founding Member
    • LifeTime Supporting Member
    Joined:
    Aug 4, 2001
    Messages:
    47,023
    Likes Received:
    608
    Trophy Points:
    248
    Location:
    Land of the Free
    Possibly a CSS issue, something not loading right. Happens to me sometimes on all my vB sites. Supposedly a known bug, to be fixed 'in a later release'.
     

Share This Page