Downtime

Moderator: Site Moderator

Post Reply
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Downtime

Post by Uncle Jesse »

Sorry folks, my fault.

HD has been running out of RAM recently so I arranged for 2 more gigs from our service provider. It costs more but they gave me a deal on it since I'm a long time customer.

Got the RAM and rebooted not realizing the server was not in shape to do it!

Had to spend hours doing what is known in FreeBSD land as "making world" which means rebuilding the entire system from source. Couldn't bring HD up while doing this.

Still finishing up with this upgrade. Went from FreeBSD 11.0 to 11.1 which includes a lot of good enhancements and security fixes but... sorry for the downtime!

Again, not quite finished.
If only the best birds sang, the woods would be silent.
cob
Master of Distillation
Posts: 2691
Joined: Sun Jul 26, 2009 4:38 pm
Location: little puffs of dust where my feet used to be

Re: Downtime

Post by cob »

Uncle Jesse; I'm going to nominate you for sainthood, ya I know rule #5,

but your getting nominated regardless
be water my friend
User avatar
jon1163
Site Donor
Site Donor
Posts: 502
Joined: Wed Jul 05, 2017 8:57 pm

Re: Downtime

Post by jon1163 »

Thanks brother for all the work you put into the site so that we can all meet up and bull#&*-
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

For those of you who speak tech geek, we were averaging 44-50% swap use. Now we aren't swapping. I've done some intensive MySQL operations and we're currently sitting at 683MB free RAM.

Much better.
If only the best birds sang, the woods would be silent.
User avatar
Still Life
Site Donor
Site Donor
Posts: 1545
Joined: Fri Apr 22, 2016 4:24 pm
Location: Great State Of Missouri

Re: Downtime

Post by Still Life »

Thought the reading finally sent me blind when the site went dark.
Glad the site is back up and healthier than ever.
You're the best, Uncle Jesse.
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

Sorry folks. Site went down again sometime last night - actually powered off! I'm not sure why. I've fixed it and will contact the ISP to see if they had a power out... but this one was strange. I'll be keeping an eye on it today.
If only the best birds sang, the woods would be silent.
User avatar
nuntius01
Rumrunner
Posts: 705
Joined: Thu Jun 08, 2017 4:51 pm
Location: upper midwest

Re: Downtime

Post by nuntius01 »

great work :clap:
I'm just the bank and the mule

post your still pics here
http://homedistiller.org/forum/viewtopi ... 16&t=66917
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

Still haven't swapped. Looking good!
If only the best birds sang, the woods would be silent.
Worm_Drippinz
Site Donor
Site Donor
Posts: 319
Joined: Thu Jan 28, 2016 6:35 am

Re: Downtime

Post by Worm_Drippinz »

Thank you UJ. ...and to the rest of the lot fer puttin' up with me.
User avatar
Canuckwoods
Swill Maker
Posts: 222
Joined: Mon Oct 23, 2017 2:18 am

Re: Downtime

Post by Canuckwoods »

Thanks, Uncle Jesse
I don't think most of us realize how much time and effort goes into sites like this.
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

Second time in a few days the system was powered off. I'm in touch with the ISP and looking into causes.
If only the best birds sang, the woods would be silent.
RedwoodHillBilly

Re: Downtime

Post by RedwoodHillBilly »

I've found that "shutdown -h now" generally isn't a good idea in the daily cron job :P
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

Hopefully found the issue. Rebooted a bit ago which some of you may have noticed.

Keeping a close eye on the server.
If only the best birds sang, the woods would be silent.
User avatar
Coyotey
Novice
Posts: 73
Joined: Sun Nov 19, 2017 3:39 am
Location: Over the hills and far away.

Re: Downtime

Post by Coyotey »

"Bridge to Engineering"...."Scotty, status on the new reactor" ...."You'll have warp capability in a jiffy, running diagnostics now...stand by"

Thanks for your efforts and a great platform Uncle Jesse! Make sure you have enough time for stillin too though!

Skål!
By three methods we may learn wisdom: First, by reflection, which is the noblest; Second, by imitation, which is the easiest; and Third, by experience, which is the bitterest. - Confucius
User avatar
Chixter
Site Donor
Site Donor
Posts: 135
Joined: Sat Sep 09, 2017 1:39 pm

Re: Downtime

Post by Chixter »

Thanks for all your work and dedication.
When I was a boy, I prayed for a bicycle. I soon learned G~d doesn't work like this so I stole a bike and prayed for forgiveness.
User avatar
fizzix
Site Donor
Site Donor
Posts: 3698
Joined: Tue Dec 05, 2017 4:08 pm

Re: Downtime

Post by fizzix »

Like the others, I appreciate all you do Uncle Jesse, and thanks for allowing my sorry butt to join.
I thought maybe someone kicked the extension cord out of the socket this morning when the site was down.
I know you're on it though.
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

I've got the ISP scratching their heads. Meanwhile I have checked, double-checked, triple-checked everything on the server and I can't find any reason for it to power down every day. I don't get it. I'll keep trying until it is fixed for good.
If only the best birds sang, the woods would be silent.
User avatar
fizzix
Site Donor
Site Donor
Posts: 3698
Joined: Tue Dec 05, 2017 4:08 pm

Re: Downtime

Post by fizzix »

If it's any help, it's been going down around 4am thru 6am Central U.S. time.
Out at 6-ish CST this morning (Dec. 9th).
Anything power related at that time or alien abduction schedule?
StillerBoy
Master of Distillation
Posts: 3387
Joined: Thu Dec 01, 2016 6:27 pm
Location: Ontario

Re: Downtime

Post by StillerBoy »

This maybe of help..
http://www.tomshardware.com/answers/id- ... maged.html" onclick="window.open(this.href);return false;" rel="nofollow

https://redmondmag.com/articles/2011/03 ... tages.aspx" onclick="window.open(this.href);return false;" rel="nofollow

Mars
" I know quite certainly that I myself have no special talent. Curiosity, Obsession and dogged endurance, combined with self-criticism, have brought me to my knowledge and understanding "

– Albert Einstein
User avatar
Kareltje
Distiller
Posts: 2176
Joined: Tue Feb 09, 2016 4:29 pm

Re: Downtime

Post by Kareltje »

Maybe if we all donate our fores and heads to use as fuel? :twisted:
It feels desolated, if I can not reach HD.
But all my respect for your attempts, Uncle Jesse! Thank you! :thumbup: :clap:
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

ISP dropped us back to 4 gigs of RAM and no power off weirdness last night.

Huh?
If only the best birds sang, the woods would be silent.
User avatar
Expat
Site Donor
Site Donor
Posts: 2245
Joined: Fri Feb 24, 2012 3:58 pm

Re: Downtime

Post by Expat »

Uncle Jesse wrote:ISP dropped us back to 4 gigs of RAM and no power off weirdness last night.

Huh?
Sounds like the VM host is having issues, over allocated perhaps.

Side note, Thank you for all you do to keep the site online!
_____________________
EXPAT

Current boiler and pot head
Cross flow condenser
Modular 3" Boka - pics tbd
___________________
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

ExpatLad wrote:
Uncle Jesse wrote:ISP dropped us back to 4 gigs of RAM and no power off weirdness last night.

Huh?
Sounds like the VM host is having issues, over allocated perhaps.

Side note, Thank you for all you do to keep the site online!
Yet another reason FreeBSD jails are better than Xen or VMWare.
If only the best birds sang, the woods would be silent.
Uncle Jesse
Site Admin
Posts: 3924
Joined: Wed Apr 21, 2004 3:00 pm

Re: Downtime

Post by Uncle Jesse »

Still running. I really believe the bump to 6 gigs of RAM from 4 gigs somehow caused those crashes.

I did a ton of work on the unix side trying to tighten things up and couldn't find anything which would cause the server to power down.

Unix servers don't normally power down. They core dump or panic or reboot.

Still keeping an eye on it.
If only the best birds sang, the woods would be silent.
User avatar
Single Malt Yinzer
Trainee
Posts: 973
Joined: Fri Sep 23, 2016 3:20 pm

Re: Downtime

Post by Single Malt Yinzer »

Put SNMP on it & setup a monitoring server locally. Solarwinds, tons of other that you can run free for a small number of nodes. I doubt it will give you the root cause but it might point you in the right direction. Plus : STATS AND GRAPHS! Who can resist them?!?!?!

And thanks as always!
User avatar
fizzix
Site Donor
Site Donor
Posts: 3698
Joined: Tue Dec 05, 2017 4:08 pm

Re: Downtime

Post by fizzix »

Uncle Jesse, don't forget to reset The Parent Site.
hellbilly007
Site Donor
Site Donor
Posts: 581
Joined: Fri Aug 03, 2012 10:59 am
Location: Never one place very long

Re: Downtime

Post by hellbilly007 »

UJ, I know you're a busy man. Just wanted to let ya know the parent site is down today at 1612 Eastern time. Thanks for providing a place for hooligan to conspire and learn from one another
Post Reply