Two Laptop Bag*
The moose likes Ranch Office and the fly likes Outgrowing our server(s)! Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » This Site » Ranch Office
Bookmark "Outgrowing our server(s)!" Watch "Outgrowing our server(s)!" New topic
Author

Outgrowing our server(s)!

paul wheaton
Trailboss

Joined: Dec 14, 1998
Posts: 20543
    ∞

Currently, all of JavaRanch is running on five servers all on one super-box. As many of you have pointed out to us, that one box is getting a little on the swamped side. Further, we want to expand into all sorts of new things, inluding some new forum software.
Our host (EJIP) tells us that the best quick fix is to add some more memory. 2 gigs of memory at $400 per stick is $800. They recommend that we also pop a second CPU into the box for $300.
So that's $1100 to fix our current problems.
It's well past time to set up shop on a second box. If nothing else, in case the mother board on one box blows out, there will be a second box we can run on for a while. Also, with the move to the new forum stuff, there will be a short time when we have two boxes running both forum packages. We need to measure how resource hungry the new stuff is going to be.
Here's the info from Paul Little of EJIP about a new server:
> Here's a new server hardware budget to start with, based on parts to be
> assembled by Ejip.Net.
>
> 4 GB RAM = $1600
> 3 ide disk drives = $300 - $500
> 2 2.xGHz CPUs, motherboard = $800 - $1200
> 1U Case, power = $100 - $200
>
> Total = $2800 - $3500
>
> Options:
> Hardware raid instead of software raid: + ~$400
> SCSI disks instead of IDE disks: + ~$300
> High-end XEON 3.0 GHz CPUS: + ~$500
>
> For $5000 you can have a really fast, reliable server. If you want to
> purchase this from Dell it will cost $11,000.
>
I'm thinking we should start at the $2800 option and grow from there if we need to.
So! The first order of business is to pull together $1100 to upgrade the system we have now.
Please pop on by to our donations page to chip in a buck or two!
Thanks!
[ July 30, 2003: Message edited by: Paul Wheaton ]

permaculture Wood Burning Stoves 2.0 - 4-DVD set
paul wheaton
Trailboss

Joined: Dec 14, 1998
Posts: 20543
    ∞

Maybe this is a good time to talk about what we are trying to fix.
The UBB stuff has a "regulator" in it. We've discovered that once in a while when the system gets super busy, it crashes. Paul Little of EJIP found a way to measure "busy". Apparently, the system runs at around "2" most of the time. But once in a while it rockets up to 20 and crashes. Paul tinkered a bit when we got the last memory upgrade (which fixed everything, for a while) and found that if he does a 503 every time we hit 11, the system will keep running and return to normal operation in a minute or two. At 12 it crashes. So we have the 503 regulator currently set at 11.
About six months ago (maybe it was longer) we seemed to be getting these 503 messages several times a day and the server was still crashing regularly. We then had just 500k of memory. With the upgrade to 2 gigs of memory, we instantly went to no crashes and no 503's. And then JavaRanch grew and grew and now we're starting to see some 503's again. This time we should try to add some hardware before the system starts to crash.
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
To clarify - Paul Little established 12 as the effective "crash limit" after we added the memory. The sequence as I recall was:
  • In the beginning, message volume was low, and the server was happy.
  • Message volume increased to where the server started periodically crashing in a very ugly manner.
  • Paul Little put in the regulator to convert "occasional ugly system crashes" to occasional 503's. Dunno what the setting was, but it was lower than the current 11.
  • We collected donations, and bought new memory.
  • Paul Little experimented with raising the regulator limit, and decided that 11 is about as high as we can safely go under the current hardward.
  • Now, message volume has increased enough that we're again seeing more 503's.
  • As message volume increases, if the regulator continues to work according to theory, we hopefully still won't see actual server crashes, but 503's will get more and more common.


  • Thats my understanding and recollection, anyway.
    [ July 28, 2003: Message edited by: Jim Yingst ]

    "I'm not back." - Bill Harding, Twister
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Sounds correct to me.
    Oh yeah, if any of you are thinking about putting a couple of bucks into the pot, the link to the donation page is here.
    Joe Pluta
    Ranch Hand

    Joined: Jun 23, 2003
    Posts: 1376
    Originally posted by Paul Wheaton:
    Here's the info from Paul Little of EJIP about a new server:
    > 4 GB RAM = $1600
    > 3 ide disk drives = $300 - $500
    > 2 2.xGHz CPUs, motherboard = $800 - $1200
    > 1U Case, power = $100 - $200
    >
    > Total = $2800 - $3500

    Paul, I'm going to throw in my two cents here because that's really all I have. I can't afford money because I literally just an hour ago went out and bought a replacement server for my network. Here's what I got:
    3.0GHz HT CPU (that's the two CPUs in one deal)
    1GB DDR 400 RAM
    80GB of mirrored 7200 RPM high speed disk (160GB total)
    1GB Ethernet
    370w power supply
    DVD-R drive
    $1700
    The RAM is $240/GB. The disk is superfast ATA, and $160/80GB drive. Mirroring (RAID I) is built into the motherboard. And the price includes $180 for Windows 2000 Professional.
    Oh, and the case has see-through windows and at least three fans .
    Just some comparison figures.
    Joe
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Joe,
    Any brand info? Phone numbers, etc.?
    PT LITTLE
    Greenhorn

    Joined: Dec 27, 2001
    Posts: 2
    Thanks for the info, Joe. I would like to mention that Ejip.Net needs rack-mountable servers, preferably a 1U server. We also recommend error-correcting RAM (ECC). We also purchase Kingston RAM which tends to be the most expensive (and higher quality we feel). This adds significantly to the cost.
    The on-board raid-1 that you mention may be the Promise FastTrack 100 controller. I've not been able to get it to work with Linux. I'm also told it does SCSI emulation on top of IDE, which turns out to be much slower than software raid. The Windows server 2003 drivers have not been released yet, either.
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Paul Little, good to see you here!
    We're sticking to Linux on all of our servers, right?
    Joe Pluta
    Ranch Hand

    Joined: Jun 23, 2003
    Posts: 1376
    Folks, I'm not here to start any trouble. I didn't know that this was a package deal. I have no idea how Paul generates his prices, I'm just mentioning mine. I've seen the Kingston PC3200 512MB chips with ECC for $115 online, so the $1600 just seemed high to me.
    If the RAID controller doesn't work with your system, then my prices don't matter. It's too bad that Linux forces you to more expensive hardware; I'd always thought that Linux was a cheaper solution.
    And as to the speed of the RAID, I'll only be able to tell you after it's installed. I'll eventually be moving to more drives, so the onboard is only an interim solution anyway. But I'll let you know how it works out.
    Joe
    Thomas Paul
    mister krabs
    Ranch Hand

    Joined: May 05, 2000
    Posts: 13974
    Don't worry, Joe. We know that you are just trying to be helpful. And we do appreciate it!


    Associate Instructor - Hofstra University
    Amazon Top 750 reviewer - Blog - Unresolved References - Book Review Blog
    Lance Duncan
    Ranch Hand

    Joined: Jul 28, 2003
    Posts: 39
    Originally posted by Thomas Paul:
    We know that you are just trying to be helpful. And we do appreciate it!

    we?
    Marilyn de Queiroz
    Sheriff

    Joined: Jul 22, 2000
    Posts: 9044
        
      10
    We


    JavaBeginnersFaq
    "Yesterday is history, tomorrow is a mystery, and today is a gift; that's why they call it the present." Eleanor Roosevelt
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Oui!
    Lance Duncan
    Ranch Hand

    Joined: Jul 28, 2003
    Posts: 39
    ja
    and i think there should be a hall of fame for donation...
    Barry Gaunt
    Ranch Hand

    Joined: Aug 03, 2002
    Posts: 7729
    Ain't that PayPal thing easy? Even old farts can do it!


    Ask a Meaningful Question and HowToAskQuestionsOnJavaRanch
    Getting someone to think and try something out is much more useful than just telling them the answer.
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    We're now at $160.01 !!!
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    We're now at $256.81
    Be on the lookout for people with new titles ...
    Amy Phillips
    Ranch Hand

    Joined: Apr 02, 2003
    Posts: 280
    I couldn't afford a new title and I wanted to be a cowgirl and all
    I gave what I could though and of course I will be clicking those bunkhouse links
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    We're now at $343.10!
    How would you guys feel about donations leading to having a javaranch e-mail address? Maybe I should ask: would having yourname@javaranch.com forwarded to your regular e-mail be something that you're interested in?
    Amy Phillips
    Ranch Hand

    Joined: Apr 02, 2003
    Posts: 280
    yeah that would be great
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Okay, I updated the donation page to reflect the e-mail address idea.
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Okay, I just updated the donation page to show our all-time best donators. The list might be leaving out some donators who sent a check - I need to look that info up.
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Now up to $424.23!
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    534.99!
    The memory and additional CPU have been ordered! At the rate we're going, we should have the $1100 by tomorrow night. I want the system slowness and the 503 errors to go away ASAP!
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    $732.68!
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    If you have a PayPal account with a few cents in it and you want to slide ten cents over to JavaRanch, you can use the account "pennies@javaranch.com". The regular account "donations@javaranch.com" accepts credit cards which means that all transactions get docked a bit (I think it's something like 30 cents plus 1.5%).
    I had to take the pennies stuff off of the donations page because People kept trying to use their credit card on pennies. PayPal says it's fine and then I get this big e-mail saying "Upgrade your paypal to a premium account so we can take a percentage!" and then I have to decline the funds and send an e-mail to the person trying to explain why they have to re-donate.
    So for credit cards and that sort of thing, use donations@javaranch.com. If you have a positive balance in your PayPal account, feel free to use pennies@javaranch.com.
    I'm still a bit of a greenhorn about using PayPal - is there a standard way to say all of this I just said?
    Andrew Monkhouse
    author and jackaroo
    Marshal Commander

    Joined: Mar 28, 2003
    Posts: 11437
        
      85

    Hi Paul
    You may have to wait a few days for those who dont yet have PayPal accounts to be able to donate. I have to wait 2 or 3 days for them to verify my credit card
    And now I have to choose a different signature, as yours is just too similar
    Regards, Andrew


    The Sun Certified Java Developer Exam with J2SE 5: paper version from Amazon, PDF from Apress, Online reference: Books 24x7 Personal blog
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    Your sig is soooo cool. And your member number isn't bad either!
    Verify credit card? What? I thought the whole thing would be done in under five minutes. But you're saying that PayPal is trying to make you set up a PayPal account? Does anybody else know if this is normal?
    On the bright side: Most the money stuff on the interenet is done though PayPal. Once you have it all set up, life is easier.
    Marilyn de Queiroz
    Sheriff

    Joined: Jul 22, 2000
    Posts: 9044
        
      10
    That's what happened to me also. PayPal wanted me to set up an account before I could send any money.
    Cindy Glass
    "The Hood"
    Sheriff

    Joined: Sep 29, 2000
    Posts: 8521
    Well, yeah, I had to set up an account. But the transfer took place within a few minutes.


    "JavaRanch, where the deer and the Certified play" - David O'Meara
    Thomas Paul
    mister krabs
    Ranch Hand

    Joined: May 05, 2000
    Posts: 13974
    You only have to wait if you want them to verify your bank account. You can send credit card payments within seconds.
    paul wheaton
    Trailboss

    Joined: Dec 14, 1998
    Posts: 20543
        ∞

    We're now up to $766.42
    I changed my sig to avoid confusion
    Amy Phillips
    Ranch Hand

    Joined: Apr 02, 2003
    Posts: 280
    When I set up my PayPal account I had to wait about 3 days for them to take two amounts out of my account, these showed up on the staement(which I also had to wait for) with a number which I had to enter into my account to be allowed to send money. Maybe it depends on your country?
    Andrew Monkhouse
    author and jackaroo
    Marshal Commander

    Joined: Mar 28, 2003
    Posts: 11437
        
      85

    Amy is probably right that it is a country issue. I am in Australia, and like Amy, I have to wait for them to charge my American Express card with some small amount ($2 I think) so that I can see a verification number on my bill, which I can use to activate my account.
    (Still not appearing on my electronic statment after 36 hour - may have to wait the 3 days )
    They tell me that the amount they charge will then become available to me to give to anyone I want - it is not used by PayPal except for verification purposes.
    Regards, Andrew (I also changed my sig )
    [ July 31, 2003: Message edited by: Andrew Monkhouse ]
    Andres Gonzalez
    Ranch Hand

    Joined: Nov 27, 2001
    Posts: 1561
    Where abouts in Australia are you Andrew?


    I'm not going to be a Rock Star. I'm going to be a LEGEND! --Freddie Mercury
    Andrew Monkhouse
    author and jackaroo
    Marshal Commander

    Joined: Mar 28, 2003
    Posts: 11437
        
      85

    Sent response privately: I dont think everyone here wants to hear about Sydney
    [ August 01, 2003: Message edited by: Andrew Monkhouse ]
    David O'Meara
    Rancher

    Joined: Mar 06, 2001
    Posts: 13459

    Greetings from Perth, then.
    I felt protected by my time-zone, but I'm encountering my first 503's while surfing tonight. Defintely prompted me to cough up, they are annoying.
    Mike Curwen
    Ranch Hand

    Joined: Feb 20, 2001
    Posts: 3695

    Aha! The "1-out-of-1000 gets a 503" script hack is working perfectly!

    I'm SOO just kidding.
    Erwin Bredford
    Ranch Hand

    Joined: Feb 11, 2003
    Posts: 167
    how do i get to donate
    i really want to help the ranch


    --One learns a lot during a lesson but seeing is not enough,you must do;knowing is not enough ,you must apply--<br />SCJP 1.4,SCBCD,SCEA part 1,OCM JEE Enterprise Architect.
    Marilyn de Queiroz
    Sheriff

    Joined: Jul 22, 2000
    Posts: 9044
        
      10
    Originally posted by Paul Wheaton:
    Please pop on by to our donations page to chip in a buck or two!

    Thanks!
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: Outgrowing our server(s)!