Two years of threads, then gone? / Purging threads (from Stats & SABR forum)

[Tango Tiger]

It looks like threads around here last two years and then disappear into oblivion. Is this true?

I'd like to propose that this forum, and perhaps the hall of fame among others, have a longer lifespan. Alot of the commentary in our forum is of a timeless nature.

[AutographCollector]

It looks like threads around here last two years and then disappear into oblivion. Is this true?

I'd like to propose that this forum, and perhaps the hall of fame among others, have a longer lifespan. Alot of the commentary in our forum is of a timeless nature.

Mid last month the size of the data base of this website was exceeded. Webmaster Sean had no choice but to purge a lot of different threads in a few forums. I assure you (as can Sean) that he did not "aim" at a specific forum, he just picked and choosed (if that's even a word).

I agree that a lot of folks would concur with your statement to "long live" the threads in this section among others. But if everyone chimed in to save this forum and that forum, then what choice would we have the next time this issue pops up?

[leecemark]

--The purge deleted only threads with no activity in two years, not those started 2 or more years ago. So if there is a continuing interest they remained in place. I guess if there is a thread you want to make sure doesn't disappear the thing to do is occassionally make a post to keep it out of the elimination zone.

[Tango Tiger]

The threads in this forum that are old are not discussion pieces, but historical. So, they would not be "active" in the sense you are describing. Nor would I want them to be "bumped" to keep its active status.

Even if a purge were necessary, being advised of a purge would have been desirable, so that we could have taken backups if necessary. I went through this with Fanhome, and did manage to save several hundred threads.

In any case, what's done is done. In the future, I suggest one of the following:
1. Do an EXPORT of each forum (or the entire database, minus the user profile table) into a SQL file. I imagine Sean is doing daily backups? I know I do it for my blog and wiki. Post that SQL file on an FTP site, xdrive.com, or Sourceforge for a limited time, so that those who want it can have it.

OR

2. Advise the members that a purge is imminent. Provide the names of all the thread numbers in each forum. This existing thread is numbered t=73464. Provide all these numbers. Then, those of us interested will be able to create a script to download all these threads, and store them on an FTP site for archive purposes.

Does this seem reasonable to you?

[digglahhh]

You may want to take this to the Web Improvement forum. Those whose opinions on this issue matter will probably have a better likelihood of reading it there.

Or, you can contact Sean directly, I know you guys speak.

[Tango Tiger]

Purging threads

http://baseball-fever.com/showthread.php?t=73464

[Mattingly]

I'm moving this to Web Improvements, since it's about the forum itself, not SABR discussion. I've also merged it with the thread started by Tango Tiger, whose link now points to this merged thread.

[Yankeebiscuitfan]

The strange thing here is that I don't see any reactions of the people who opposed against the Brooklyn Dodgers fans.

A couple of weeks ago they (the Brooklyn fans) had the same question. Maybe their reaction was a bit more fierce than yours, but still...

Just my :twocents:

[VTSoxFan]

I may be wrong, but my understanding of what happened is that it was an emergency situation, and Sean was faced with something like triage - what can and must be saved v. what can and must be let go, with no time to pick and choose. In order to keep the entire site from crashing, Sean had to choose a cut-off date for older threads to consign for deletion. No forum was singled out; we all lost threads we would rather have kept.

I'm sure he hated having to make that decision, but he had little choice.

[Tango Tiger]

Can you point me to a thread that has the details? Was there a general announcement somewhere?

In any case, doesn't the site perform weekly (daily?) backups, be it from the webhost or Sean himself? So, even if the database was at its limit, what happened to those backups? Why not put those backups up on an FTP site?

[Macker]

I don't think any advance notice of threads being purged is necessary. These are merely discussion boards. If the discussion has ended, there isn't a reason to keep it. If a thread is particularly valuable to you, why wouldn't you already have kept a copy?

The strange thing here is that I don't see any reactions of the people who opposed against the Brooklyn Dodgers fans.

A couple of weeks ago they (the Brooklyn fans) had the same question. Maybe their reaction was a bit more fierce than yours, but still...

I'm not sure what you are saying, but I'm guessing you are questioning why the Brooklyn fans didn't have much response from moderators. I didn't respond, because I was unaware of their questions until seeing your post above. Turns out they asked those discussions in a Brooklyn forum thread about whether fans would have supported the team in Queens. That's what happens when threads go off-topic.


As far as space issues, I have long advocated that mods clean up their forums by eliminating the excess quoting, interspersed chit-chat, me-too posts, duplicate threads and off-topic posts. Members could be more diligent in how they use the quote feature as well. However, I have always been in a very small minority.

I have never suggested it as a requirement to being a mod, just that mods should do this if they have the time and inclination. Nonetheless, some are horrified at the thought that their post may be deleted or edited.

In Trivia, I occasionally go in and wipe out wrong guesses. If the trivia question is where did Ruth hit his only minor league homer? and their are 10 wrong guesses, why keep them there after the thread has run its course? There is some good information in the old trivia threads, but removing the wrong guesses saves more than half the space in that one forum.

There are a lot of posts where someone quotes a 50-line post above and is obviously replying to just one or two points of that post. There is no reason to repeat the entire 50 lines. Others will quote a long post and just post, "I agree." If you look at some threads, you will see each post repeated as a quote in the subsequent post. One could edit out the excess quoting without losing any content. Who needs to look at a two-year-old thread and see a dozen posts of an off-topic argument?

I estimate that if we were to remove the me-too posts, off-topic posts, and excess quoting, we could probably free up 30% of the space being used. Unless people are on board with me, and that means the mods of your particular forum, the day will come again when all threads meeting a certain criteria will be purged.

A few days ago, I deleted about 20 threads from within the past two years. They were things like, "Happy Thanksgiving" and "Merry Christmas." Keep in mind, those threads about holidays remained after some of your older, more useful, threads were purged.

Visit your favorite forum. Click to go to the last page of the forum and view a few threads. Ask yourself how many of those threads really need to be there. How many posts within a thread need to be there?

[Yankeebiscuitfan]

I'm not sure what you are saying, but I'm guessing you are questioning why the Brooklyn fans didn't have much response from moderators. I didn't respond, because I was unaware of their questions until seeing your post above. Turns out they asked those discussions in a Brooklyn forum thread about whether fans would have supported the team in Queens. That's what happens when threads go off-topic.

When the Brooklyn fans opposed to the deletion of their threads, there were a lot of forumers (anti Brooklynites?) who attacked them. They said the same as you do, that this a message board and no thread is save forever.

But now someone else is asking the same question, maybe a bit less fierce, and I see none of the guys who attacked the Brooklyn fans making any comments.

Weird isn't it? Or is it just me?

[leecemark]

--The difference is that this thread asks for information and suggests some possible solutions. In the Brooklyn forum they suggested it was an attack on their forum and/or that their forum should have been exempted from the site wide reduction. It wasn't and there is no reason that froum should have received special treatment.

[Macker]

In the Brooklyn forum they suggested it was an attack on their forum and/or that their forum should have been exempted from the site wide reduction.

Exactly. Also, when members sink to the point of attacking the webmaster, most mods will stay out of it.

Examples from that thread in the Brooklyn forum:

We now see a website that practices censorship and erases the past. Both practices indicate that the webmaster has lost control of what he is doing. And by callously and clumsily erasing the past, the webmaster is telling all posters that what they are posting will disappear forever when space runs out: space, not content, is what matters. This entire enterprise has become meaningless.

And since this massacre occurred, there has been no explanation and no apology. It's as if we never existed...as if the webmaster doesn't give a damn.

The webmaster has scuttled his entire ship.

================================================== ====

But now someone else is asking the same question, maybe a bit less fierce, and I see none of the guys who attacked the Brooklyn fans making any comments.

Weird isn't it? Or is it just me?

No, I don't think it's you. The issue with the Brooklyn fans was the high percentage of threads they lost. But take a look at the Brooklyn forum. I see an inordinate number of threads wishing a Dodger a happy birthday. I would never suggest those threads shouldn't exist, but should the birthday wish from each year be preserved? There is also a thread about Nathan's hot dogs that isn't even about the Dodgers.

There is a thread from July 2007 wishing Pee Wee Reese a happy birthday. There is another birthday thread for the team captain in July 2006. Among the threads that were purged recently, there were probably a few more Pee Wee birthday threads.

I know there were a number of well-written, clever posts among those that were purged. A periodic trimming as I suggested in a previous post likely would have allowed those threads to stay as years-old birthday threads would have been deleted.

[Ubiquitous]

Quoting the webmaster:

This was not a matter of storage space, it was a matter of physical table size. We have enough storage space to keep 10,000 more photos as we keep those physically on the server's drive. Same with the 70+ tables we have. The post table itself can't exceed a certain size - we exceed that size which prevented the system from functioning any more. This forum is made up of more than 70 tables, and only that one was exceeded the envelope which isn't bad considering it has been running strong for years now. I am already looking into other ways we can ever prevent this from happening in the future and don't like making promises, but rest assured I'm trying to find every way possible to tweak the system even more than it already has been!

And for all believe me nobody not even the webmaster is happy this happened. Nobody got singled out and nobody wanted this to happened. That being said it happened and now we must move on. The webmaster is looking for ways to prevent this and I think his track record is pretty darn good when it comes to things like this. He has been running this place for over 6 years now, much of the time out of his own pocket if not still out of his own pocket. When something happens that can be termed a "mistake" he learns from it and tries to make sure it doesn't happen again. He has upgraded his system numerous times, bought bigger space several times, increased our own abilities numerous times. I think we should give him the benefit of the doubt on this and not accuse him of being Lt. Commander Queeg.