System Updates Thread ...7/21/20 . Security and Pictures
 

*** go to my last post for most current updates


I will update this post very soon.

We have had a major database crash and it's being worked on.





.

For the last 3 1/2 hours I have been on and off the phone with our hosting company. There was a MYSQL error (a database) and some files were corrupted. The last backup was done at around noon today. Data after that was not able to be retrieved. My apologies.

Due to this I am in talks at this minute with someone to upgrade our main s/w so this won't happen again. It probably should have been done but the shit to equity ratio wasn't there yet, now it is.

We will be going to the latest version of our software and there might be some planned downtime...which is much better than unplanned.

And they tell me the database isn't perfectly stable right now (fingers crossed).

.

You Da Man Leon!

We might have lost some data that will really upset me. It's being worked on and I don't know if it can be retrieved or not.

There will need to be some threads redone I am afraid.

As a result of this outage, the next Goldin Auction end date will be postponed indefiintely.

I am having a database person look at this tomorrow at noon time (but possibly later tonight). There is a chance we get stuff back but I am always a bit pessimistic.

There are some spots on the forum that are shaky right now, database wise. All I can say is sorry for the issues and it's being worked on as quickly as possible.

Leon - I really appreciate all the work you do to to keep this site going!

I lost my post and a PM someone sent me. I’ll repost when the site seems stable.


Sent from my iPhone using Tapatalk

No problem....there is some money made (now) so I feel a responsibility to have it be the best it can be.

What a lot of folks don't know is that when I started doing this there was no monetization whatsoever, for close to 2 yrs.
I just wanted to have a good place to talk about vintage baseball cards. So I take it personally when this kind of stuff happens. At minimum the s/w will have a total upgrade...same software, just the newest version. I don't think (not positive) there will be aesthetic changes. The general look and feel won't change.

I know the search database is corrupt and not working right now. But honestly, it never worked great in the first place. Maybe the latest version will be better...

.

test

Knock on wood, crossing my fingers we are back up and running. More to come later....been a bit stressful over here. Sorry fo the huge issues.

We don't fault you Leon, we appreciate all your hard work trying to restore the little world you created for us.

Thank you for all of your effort, Leon.

+1

Thanks guys but I can't tell you how horrible I feel about this. It is partly my fault and I am doing everything possible so it doesn't happen again. In ALL OF THIS MESS, we will end up losing a total amount of 4 hours of data. That is from yesterday at about noon to 4 to 430.....because the backup was done at noon and that was what was recovered.

test

Leon, you and your great team are a cute reason why I enjoy theirs website so much. I want to thank you guys for everything!

Shake it off, Leon. Nothing or no one is perfect. I really appreciate having N54.

It's all good Leon, we really appreciate all you do to keep Net 54 going!

I still cannot re-post scans on my auctions..........

It is continuing to be a bit of a problem. It is up and ok at the moment and being monitored. FYI, this started with a BruteForce attack on our site. So it was a malicious attack that started these issues. Then they compounded due to some antiquated parts of the s/w....and that is being addressed but I want the system to be stable and up for a day or two before the upgrade. Thanks for everyone's patience.

I see that. I will open the ticket back up......

The internal attachment (*pic) system is not working at the moment but if you host somewhere else it does...I* just posted a pic in the auction BST test thread...

And I thought it was my fault for creating yesterday's thread. I think I'll go back to Facebook for controversial posts :)

Rich

I just replied to a party's post for a card, I attached the scans as usual but they are not showing up. I will refresh in a while and see if they appear.

It certainly wasn't your fault and the thread was fine.

We are still working on the system and hope to get it fixed enough so we can upgrade it. It will happen but could be a little more pain getting there...

I know on that I was just trying to lighten your mood ;) I know I get frustrated when I have computer issues at work so I'm with you on that front.

See you Friday :)

Rich

At this moment the attachment pics are not working. It is being looked at now. Until then you can host on another site as I did in the Auction BST section test thread, if possible.... ....the Bender shows from my own personal site but the one I tried to upload to this site didn't work. Again, it is being looked at now....

Thanks just wanted to let you know. I contacted the poster and emailed him the pics.

Don't worry, it's just a virus....

The system still isn't perfect and to me it looks some old attachments aren't being shown, at least yet. We are still working on it and it seems to be getting better for this moment :) .

... I will keep updating this thread. The goal is to get the system fixed and working and then upgrade immediately after that. I don't want to say much else because before I hit the submit reply button below here, things could change.

Here is a new one uploaded through our system. And I am aware that our community photo albums are not working right now. This too is being worked on and is part of the same attachment issue, though this could be a php issue on top of it.

Leon,
Appreciate all your efforts and your patience through all this.

Jim

YES Leon---
 

--your desire to keep a top-notch website is GREATLY appreciated! Net 54 is a JOY to belong to & visit on a daily basis!

Being somewhat familiar with sw upgrades/patches etc and the possibilities of how bad they can go, I think the results could be a lot worse.
Thanks for what you do!

I did lose some pics from a BST listing.
But I just uploaded them again with no issue.
Thanks.

Thx leon no issues bst

Thanks for doing a great job running the board Leon. Not sure what Bruteforce is, as I am not enet savy. Did they go after other chat boards, on the same platform to? Keep up the good work.

I am aware the site just went down for a few minutes. Sorry once again. This was a Windows SVR 2008 issue. Another upgrade we are doing asap.....

The tech didn't even ask my name :)

.

Assuming the tech knows your voice by now... :p

Oh Yeah...he picks up the phone and I say hello...he says Hi Leon....yeah, they know me.

Concerning the major attachment loss that has happened including our whole community photo album, the hosting company is doing a restore from 24 hours before these problems started. All of this is being done with the offline backups they keep. I had to pick a time and that was the least loss....and it isn't for sure either. The files could be majorly corrupted beyond repair. I have a DBA person that might be able to help some too. We really need to get squared away before we upgrade. More updates as I know them. And to reiterate, any downtime will be scheduled at very off hours. And hopefully our next downtime is scheduled. I wouldn't bet on it though., :o


.

I'm confused why someone would want to send an attack on your site. We are all cardboard nerds. What's the point?

Possibly a disgruntle member or someone with an interest in seeing this site go down. I hope you don't lose all your relatively with the seach engine spiders. Whenever you do a keyword search with baseball cards, this platform seems to always pop up in top 10 with former discussions. Now who would benfit if net54 loss decades of these threads?

True...though it would make a pretty good satirical mockumentary on the crazy cardboard nerds and their life and death feuding over baseball cards. I want to be played by George Clooney...but I suspect it would be Jeff Garlin.

System going down at 3AM EST 7/10/2020 for maintenance
 

The hosting company retrieved some picture/attachment data from a backup from one day before this stuff started happening. I am not sure how much but they are going to install it in our system at 3AM EST 7/10/2020 and we will taken off line (DOWN) at that time. It should be for about an hour. Hopefully this works or we can try another day back. Once we get this current system in good enough shape, get rid/fix corrupted files etc... we will do the upgrades. My intention is as soon as possible. The system has been a little shaky today as many will have seen it go down for a few minutes here or there. Each time I made a call and asked them to kick the server to get it going again. But that is getting old real quick. That is where we are at...thanks for everyone's patience.

PS...hosting company has a very good online tech support ticket system. But calling and getting someone to help "now" is quicker. :)


.

Crazy isn't it? As long as we don't say uncle we should get through this okay.

Thanks Leon for everything you're doing to keep the site stable!

As we head into the two minute warning, I will cross my fingers and toes and wish the forum the best of luck!

Brian

It looks like we got a lot of pictures back. WE did not get the community picture albums yet. I will look into getting that section back too. Other than community picture album not working (yet, hopefully) I would like to hear other issues being seen? And the system is still a bit wobbly and might be until we get these upgrades done. I have someone lined up this weekend to do the upgrades, if the outstanding issues get resolved first . OR if they won't be able to be resolved. Still some unknowns....

Possibly, but it could also easily be someone trying out an attack based on the software that is used. I would think it would be more likely they were going after the username/password combinations. Just a guess because the information on this site is important to such a small group of folks so why target it?

As for the search, you will still be able to search fine. Especially since most of the searches you do are relatively esoteric and isn't really found many places on the web. Despite the site become overly T206 heavy in recent years there is a great deal of interesting information held in these pages since Elliot started the board.

Hey Folks
I think I am going to shoot myself :).....Can't really say how this is going to go. I am sorry. I am constantly talking to folks. There are corrupted files that just aren't coming back.

Weloome Back Folks! No shooting of ourselves allowed.

Brian

I am becoming a skeleton from stress. :) ..... I was on the phone with the hosting company at 445 this morning...and lots of other times before and after....Even my prescription trazodone (for sleeping) isn't working.... :eek:
A lot of pictures won't be up right now but we are still hopeful in getting them back...

.

Thanks for all the efforts Leon, but stay away from the 'Stress Diet'. It is probably just a passing fad, and could have serious ramifications as well.

Brian

Maybe injecting bleach will help.

It didn't.

.

As a 36 year Telecom Tech I feel your pain. Sorry can't help you technically but hang in there and fight the good fight.

Net54 Rocks!!

Leon,

Before you shoot, can you please make sure that I'm in your will. I wouldn't mind having a card in my collection with LL provenance. 20 years from now I'll be able to say that I have a LL card in my collection. :p :D


I think most of the board members understand, and if they don't, then the only advice to be given is to take a deep breath, count to 10 and hit the refresh button or start a Leon "go fund me" page to pay for the mental health bill that is coming...

Maybe Corn Pop has a database background.

Thanks for the sympathy, guys.

To give a tiny explanation the situation is compounded by old s/w and a brute force attack. And as anyone vaguely familiar with this stuff knows, one thing leads to another and another and so on. Now we have some badly corrupted attacment files. I am hopeful the system stays up at this point and that's about all I can say right now.

And hopefully sooner than later we recover the other files. Just can't tell right now.

So far so good. Thanks for your hard work, Leon.


We appreciate you...and love this site!



.

I think Weird Al would make a good "Doug"...

Keep your chin up Leon!!
 

Anyone who's been on NET54 for any length of time knows how much effort you put into this site.

I thank you for giving us a place to come to where we can enjoy our hobby with folks that are passionate about collecting. Stuff can always be rebuilt/replaced. Stay healthy and sane!

Best,

Steve

Learned a long time ago....
 

Don't sweat the small stuff........and


It's ALL small stuff!


A day at a time my friend - do the best you can. Take a deep breath and know that you have built something appreciated by many.

Thanks Leon
 

Thanks for busting your ass Leon!!...a lot of us use this site as our archive...especially of cards we no longer have...and scans on computers we no longer have...Jerry

Hang in there Leon, we all appreciate the effort and sympathize with the stress. If there is anything we can do, just ask - we are a community of diverse talents and abilities, who will likely do whatever they/we can to help. Just ask and I would bet many would be happy to help.

And I just saw the sticky post about systems administrators, nice! Unfortunately, I am horrible with technology.

Hi, Leon.

I echo the sentiments expressed by many. We truly appreciate all that you do for us through your excellent stewardship of this site.

You provide us with a corner of the Internet where we can share our love for the hobby...with people who don't consider it odd to chase down century-old cardboard.

We understand you're doing everything possible to get the technology issues resolved. Keep up the the great work.

Maybe.......
 

.....a very powerful light...you know...like inside.
But very seriously Thank You very much for making this place available to all of us. Certainly we can live with the occasional SNAFU. And I am sure everyone understands the torture and work you go through to keep this ship right side up !! Thank you Leon and moderators !!!

We have an OBC person, Wayne, looking at the DB Files right now. OBC guys rock. :)
It is not an easy process but he is going to try to pinpoint the corrupted files and manually fix them. There are no guarantees.
And if anyone can tell me if they see anything wacky except some pictures and the whole community picture album missing, I would appreciate it? Again. we have to get the system fixed and stable before we upgrade. And keeping the system up is my main goal....

,

Leon,

I'm late to the news. I don't check in here every day. But I'm reading that our picture albums are wiped out?

Should I try to reload and build it again or wait until everything is fixed?

I appreciate this isn't your priority. I hope all goes well.

Cheers

I think I would wait for now. It isn't working properly at all. It is the main thing we are trying to fix along with some other pictures. Should know in the next day or two if there will be success or not....

.

site
 

Appreciate you keeping the site afloat !

System update 11:40 am 7/15/2020
 

There has been another attack a little while ago.....

We are addressing it. It is not just at us.....it is widespread.....

and ways are being investigated to fix these attacks for good too.. Wish me luck :)

Dang, Leon, this is just rubbish. I certainly do appreciate all your efforts in keeping us up and running!!!

Thanks for all you do, Leon. Sorry to hear this has been such a headache!

Quick update

Still looking into fixing the current issues. As far as I am aware the issues are only some pictures (maybe quite a few) and the community tab, photo album pictures. if there are other things that are missing please PM me or post it here.

With the help of a few members quite a few database issues have been fixed. The system seems quite a bit faster today.
There was a conference call today with the hosting company for some more more reliable and secure solutions. Probably our own private physical server and more secure firewall. And we are still looking at minimally upgrading our current s/w version ..or maybe go a whole two iterations newer to the most current s/w. The look of the forum should stay the same. That is the goal.
Thanks for everyone's patience.


ps...so when the page reloaded for this particular post it was sketchy. Needless to say everything is still being looked at and worked on as we go..... (fingers crossed)
.

It was a few days ago. I had some PMs disappear and didn't get email notifications on a few. Been working good for me since.

It's a bit surprising the ISP isn't more helpful. But then failure to keep hosted data secure as an upsell opportunity is sooo software industry.

Had a conference call with the hosting company yesterday. We will most likely be moving to our own physical server. That should make us almost immune from the Brute force and other attacks.

We are trying to retrieve pictures between 7/6/2020 and 7/12?/2020.

The Community photo album might have been damaged beyond repair or erased altogether. It is still being looked at and by Monday or Tuesday we should know for sure.

That said, the Community photo album does look like it is working correctly at the moment, as far as loading new pics. I just loaded one and it worked. I don't want to promise anything but that is it for now.

And of course right after this post the system went down for a few minutes. I was back on the phone with them :).

Gosh. Leon, I sure do thank you for all your efforts in getting this right.

I do hope you are able to retrieve the Community "Pictures and Albums". I must have had hundreds of images to support information sharing with forum over the years. Without the images many of my posts have limited value (some might argue the same even with the images).

At any rate, hoping you are able to retrieve all the images for those who took the time to post them and those who take the time to view them.

Good Luck!

Leon,

We're Luckey to have you. Thans!

Sounds like they weren't really doing backups.

Not what I'd continue with.

I will do what I think is best for the forum and my sanity. They were doing backups but it was automated and they backed up a crashed site. Those things are being addressed.

.