Options

Where did Alpha Go? (read this)

ÄlphämönkëyÄlphämönkëy Registered User regular
Hey all.
The forums have been picking up in usage recently, and I've been hiding in the background working on keeping all the features afloat. Turns out I can't. phpBB just plain doesn't scale for searching without adding hardware or doing massive code rewrites. So searching is going to be removed from our phpBB powered forum permanently. Before the lync mob comes to my appartment, atleast hear my second announcement. I finally got the new forum code base in a place where I like it. Cheifly, someone else did a lot of the hardwork. I need to make a few more phone calls, but I think the new forum software will be a rails app by the name of Opinion.
Let me explain why I chose opinion. Tobi and the guys over at leetsoft, are very talented developers. Infact, these guys write the code that was used as a framework for the penny arcade main page. This also provides people with an outlet, as if you want to see a feature on the forums, write a patch. If it doesn't suck, I'll throw it in. Opinion isn't perfect, I have to roll in a lot of changes to their source code. Likewise, Opinion's default template was designed to be an "interactive blog" and we definatly aren't an interactive blog. So that will need to change. But the core of it is a good app, that will scale really well, and will be more responsive in general.

I'm going to try and get a demo application hosted so you guys can see what it looks like and beat on it a bit.

Okay, now you can send the lynch mob to my appartment.
-Alpha

Älphämönkëy on
«134

Posts

  • Options
    TDLTDL ClubPA, __BANNED USERS regular
    edited March 2006
    Oh man, how am I going to ego search now?

    But seriously, I'm sure we'll all get over it.

    TDL on
    Meet me on my vast veranda
    My sweet, untouched Miranda
    And while the seagulls are crying
    We fall but our souls are flying
  • Options
    OrikaeshigitaeOrikaeshigitae Registered User, ClubPA regular
    edited March 2006
    Yay, stability forthcoming!

    <3 Alpha.

    Orikaeshigitae on
  • Options
    EchoEcho ski-bap ba-dapModerator mod
    edited March 2006
    <3<3<3 Rails <3<3<3

    Echo on
  • Options
    redxredx I(x)=2(x)+1 whole numbersRegistered User regular
    edited March 2006
    duh... forum not break so much. good.

    me happy.

    no understad rails.

    love alpha.


    :edit: hey alpha got off his ass and is finaly inserting word matches again! Yay editing abounds.

    redx on
    They moistly come out at night, moistly.
  • Options
    ThanatosThanatos Registered User regular
    edited March 2006
    Hmmmm... Nonworking forum with searches, or working forum without searches... tough call. :P

    You kick ass, Alpha.

    Thanatos on
  • Options
    contrabandcontraband Registered User regular
    edited March 2006
    redx wrote:
    duh... forum not break so much. good.

    me happy.

    no understad rails.

    love alpha.

    you know we would all be on our asses over at gamefaqs or something if it wasn't for you ;)

    contraband on
    sigxw0.jpg
  • Options
    ToastToast Registered User regular
    edited March 2006
    :^:

    Toast on
  • Options
    Rear Admiral ChocoRear Admiral Choco I wanna be an owl, Jerry! Owl York CityRegistered User regular
    edited March 2006
    Hell, I barely use the search function anyway.

    Thanks. :^:

    Rear Admiral Choco on
  • Options
    elecgutselecguts Registered User regular
    edited March 2006
    The absence of a search function may hurt Linksville a bit, although people still posted old links even with the search function. It just might happen tad more often now.

    elecguts on
    wait, what?
  • Options
    redxredx I(x)=2(x)+1 whole numbersRegistered User regular
    edited March 2006
    so... mods can still search, right?

    People can't get away with doing naughy things without being found out.

    redx on
    They moistly come out at night, moistly.
  • Options
    Bionic MonkeyBionic Monkey Registered User, ClubPA regular
    edited March 2006
    Burn Him!!
    :wink:

    Seriously though, thanks for all the hard work Alpha, and uh... guys at Leetsoft.

    Bionic Monkey on
    sig_megas_armed.jpg
  • Options
    HenroidHenroid Mexican kicked from Immigration Thread Centrism is Racism :3Registered User regular
    edited March 2006
    I never used the search function anyhow. :wink:

    I think it is a good call. Someone, somewhere, is bound to know where a thread is floating if a duplicate is made.

    Additionally, maybe I should start up the Game On Reference Desk again.

    Thanks for the work you do Alpha.

    Edit - I went ahead and made it. I'd like to think this is for all forumers, not just G&T (it's there because it fits).

    Henroid on
  • Options
    DeusfauxDeusfaux Registered User regular
    edited March 2006
    what are some of the other major forums around the net using? how do they get on?

    edit: er, and how long between disabling search (and getting it back via the new code?)

    Deusfaux on
  • Options
    PheezerPheezer Registered User, ClubPA regular
    edited March 2006
    Deusfaux wrote:
    what are some of the other major forums around the net using? how do they get on?

    edit: er, and how long between disabling search (and getting it back via the new code?)

    I think the more common answer to load issues is to throw more and more hardware at the problem.

    The forums are a pretty small segment of PA's readership though, and accordingly, don't really warrant the whole "invest whatever is necessary to keep the current codebase living" approach.

    The largest webforum around, based on http://rankings.big-boards.com/ has 3400867 members in total, but gets by with phpBB 2.

    IGN's forums software is indeterminate, at least to me, but according to the credits they're paying someone to run it off of an Oracle database for them.

    Most of the really big forums derive significant revenue from their forums. PA is pretty big for a forum that probably does not recoup its own costs directly (ignoring ad revenue stemming from traffic that would visit the front page only because that given user visits the forums because that is not realistic).

    Pheezer on
    IT'S GOT ME REACHING IN MY POCKET IT'S GOT ME FORKING OVER CASH
    CUZ THERE'S SOMETHING IN THE MIDDLE AND IT'S GIVING ME A RASH
  • Options
    DeusfauxDeusfaux Registered User regular
    edited March 2006
    haha i wonder who wrote the little blurbs for us:

    Hardware: A shared web cluster powers the webserving front end to the site, and is matched by a small, private MySQL 4 database server on the backend.

    Software: Running a very hacked version of phpBB. Most of the code has been optimized by hand, and several mods were developed solely for use with the penny arcade forums. Additionally, the developers have come up with several security enhancements which they have re-released to the public.

    Number of moderators: 26



    edit: that site has neat stats. while we're only around 170th for posts (seems kind of a silly thing to rank by first and foremost) and we drop to like 600 for members, we shoot way up for traffic (40-some) and daily page views - though those are probably related to the front page somehow?

    we have a high average online users count too at 22nd with (500)

    so many lurkers!!!

    Deusfaux on
  • Options
    PheezerPheezer Registered User, ClubPA regular
    edited March 2006
    Deusfaux wrote:
    haha i wonder who wrote the little blurbs for us:

    Hardware: A shared web cluster powers the webserving front end to the site, and is matched by a small, private MySQL 4 database server on the backend.

    Software: Running a very hacked version of phpBB. Most of the code has been optimized by hand, and several mods were developed solely for use with the penny arcade forums. Additionally, the developers have come up with several security enhancements which they have re-released to the public.

    Number of moderators: 26



    edit: that site has neat stats. while we're only around 170th for posts (seems kind of a silly thing to rank by first and foremost) and we drop to like 600 for members, we shoot way up for traffic (40-some) and daily page views - though those are probably related to the front page somehow?

    we have a high average online users count too at 22nd with (500)

    so many lurkers!!!

    I'm pretty sure they're seperating the non-forum hits from the forum hits. Actually I'm absolutely sure. PA is a huge, huge site.

    That members stat is the telling part though. You can expect ad revenue to be related most directly to the number of members, meaning that on average, we make as much in ad revenue as forums with around 600 users. But there's no way in hell any other forum with 600 or so users puts as much load on their server hardware as we do.

    Pheezer on
    IT'S GOT ME REACHING IN MY POCKET IT'S GOT ME FORKING OVER CASH
    CUZ THERE'S SOMETHING IN THE MIDDLE AND IT'S GIVING ME A RASH
  • Options
    DeusfauxDeusfaux Registered User regular
    edited March 2006
    for anyone whom it wasnt also immediately clear: our auto pruning kills a fair amount of our posts

    certainly more than back in the day when you could go back like through 30 pages of material or more.

    so these other forums with piles of servers can probably afford to house massive amounts of history better than we could (and rank higher according to dimensions that use postcount)

    I think Alpha still has the on-going stats for total forum posts and things though, maybe he could tell us for curiousity's sake.

    Deusfaux on
  • Options
    Just_Bri_ThanksJust_Bri_Thanks
    lurks out

    Seething with rage from a handbasket.Registered User, ClubPA regular
    edited March 2006
    lurks out

    Just_Bri_Thanks on
    ...and when you are done with that; take a folding
    chair to Creation and then suplex the Void.
  • Options
    PheezerPheezer Registered User, ClubPA regular
    edited March 2006
    Deusfaux wrote:
    for anyone whom it wasnt also immediately clear: our auto pruning kills a fair amount of our posts

    certainly more than back in the day when you could go back like through 30 pages of material or more.

    so these other forums with piles of servers can probably afford to house massive amounts of history better than we could (and rank higher according to dimensions that use postcount)

    I think Alpha still has the on-going stats for total forum posts and things though, maybe he could tell us for curiousity's sake.

    This is a good point. As of this moment there are only approximately 559,000 posts in total in the database. I don't think the current pruning schema allows it to really get much higher.

    I don't know if alpha keeps "lifetime" stats, though.

    Pheezer on
    IT'S GOT ME REACHING IN MY POCKET IT'S GOT ME FORKING OVER CASH
    CUZ THERE'S SOMETHING IN THE MIDDLE AND IT'S GIVING ME A RASH
  • Options
    DeusfauxDeusfaux Registered User regular
    edited March 2006
    it was him or Ramius who once spouted off a few stats that as far as I remember, included stuff from before the big wipe, even...

    Deusfaux on
  • Options
    contrabandcontraband Registered User regular
    edited March 2006
    contraband on
    sigxw0.jpg
  • Options
    scarlet st.scarlet st. Registered User regular
    edited March 2006
    Alpha, I will take you out to lunch someday

    :^: Minnesota

    scarlet st. on
    japsig.jpg
  • Options
    mccmcc glitch Registered User, ClubPA regular
    edited March 2006
    Thanks for the update.

    Will the Opinion forum support searching once it is ready?

    mcc on
  • Options
    HenroidHenroid Mexican kicked from Immigration Thread Centrism is Racism :3Registered User regular
    edited March 2006
    Hey, uh, this kinda just occured to me, but seeing as adminship has changed to being Alpha and Whippy, will the Monkey Den be renamed to reflect this?

    Didn't think this deserved it's own thread.

    Henroid on
  • Options
    DogDog Registered User, Administrator, Vanilla Staff admin
    edited March 2006
    No.

    Unknown User on
  • Options
    HenroidHenroid Mexican kicked from Immigration Thread Centrism is Racism :3Registered User regular
    edited March 2006
    No.
    But it isn't Alpha Monkey and denihilist!

    Henroid on
  • Options
    LegacyLegacy Stuck Somewhere In Cyberspace The Grid(Seattle)Registered User, ClubPA regular
    edited March 2006
    Henroid wrote:
    No.
    But it isn't Alpha Monkey and denihilist!

    It still works.

    Legacy on
    Can we get the chemicals in. 'Cause anything's better than this.
  • Options
    DogDog Registered User, Administrator, Vanilla Staff admin
    edited March 2006
    The topic has come up a few times already, Whippy said the name will stay.

    Unknown User on
  • Options
    jothkijothki Registered User regular
    edited March 2006
    As long as everything works more or less in the same way on the user side, I'll be happy. I love the current feel of the forums.

    What will we be seeing when the shift happens?

    jothki on
  • Options
    HenroidHenroid Mexican kicked from Immigration Thread Centrism is Racism :3Registered User regular
    edited March 2006
    The topic has come up a few times already, Whippy said the name will stay.
    Ah okay. Just wonderin'. :P

    Henroid on
  • Options
    WhippyWhippy Moderator, Admin Emeritus Admin Emeritus
    edited March 2006
    Basically I just can't think of anything witty.

    :-p

    Whippy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    pheezer FD wrote:
    Deusfaux wrote:
    what are some of the other major forums around the net using? how do they get on?

    edit: er, and how long between disabling search (and getting it back via the new code?)

    I think the more common answer to load issues is to throw more and more hardware at the problem.
    Yup. The expensive part of a forum isn't the PHP part (once you optimize things) it is the database part. So most people introduce some form of replication. Extra hardware is where we would be able to save ourselves, but I would rather see extra hardware spent on things that would help take stress off the mainsite, maybe some dedicated power for serving ads quicker.
    The forums are a pretty small segment of PA's readership though, and accordingly, don't really warrant the whole "invest whatever is necessary to keep the current codebase living" approach.
    Yup. Robert has offered to buy us software and even a bit of hardware if it would make things easier, but again, my priority in terms of speed is the mainsite.
    The largest webforum around, based on http://rankings.big-boards.com/ has 3400867 members in total, but gets by with phpBB 2.
    That's gaiaonline. To say that they use phpBB2 is somewhat of an exaggeration. They started with phpBB2, and basically rewrote every SQL query. They also use a server farm to host their stuff. They have replication of databases like you wouldn't believe and a host of linux webservers up front. Last time I talked with Lanzer they basically had an entire set of racks devoted to them. A lot of the optimizations we use he developed, and I'm fairly confident he is using a few of my hacks as well.
    IGN's forums software is indeterminate, at least to me, but according to the credits they're paying someone to run it off of an Oracle database for them.
    Yeah. If I had a dedicated Oracle 9i enterprise server and a full-time DBA, you wouldn't see any database issues. MySQL corrupting is the problem, and row level locks in a database is the solution. The problem is, MySQL sucks with row level locks in 4, and MySQL 5 seems like a little kid that is now wearing Pull-Ups. Yeah, you don't shit yourself as much anymore, but when you do, its a mess to clean up. PostgreSQL will be our saving grace in the next iteration of the forums, and it will use some cool features for accomplishing things like permissions and such. But, that is a story for another time...
    Most of the really big forums derive significant revenue from their forums. PA is pretty big for a forum that probably does not recoup its own costs directly (ignoring ad revenue stemming from traffic that would visit the front page only because that given user visits the forums because that is not realistic).
    Exactly. We are fairly unique in that aspect. I mean, we don't even have the google adsense ads anymore, so the forums provide no revenue for Penny Arcade.

    Älphämönkëy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    Deusfaux wrote:
    haha i wonder who wrote the little blurbs for us:

    Hardware: A shared web cluster powers the webserving front end to the site, and is matched by a small, private MySQL 4 database server on the backend.

    Software: Running a very hacked version of phpBB. Most of the code has been optimized by hand, and several mods were developed solely for use with the penny arcade forums. Additionally, the developers have come up with several security enhancements which they have re-released to the public.

    Number of moderators: 26



    edit: that site has neat stats. while we're only around 170th for posts (seems kind of a silly thing to rank by first and foremost) and we drop to like 600 for members, we shoot way up for traffic (40-some) and daily page views - though those are probably related to the front page somehow?

    we have a high average online users count too at 22nd with (500)

    so many lurkers!!!
    Heh. That was me a long time ago. Back in the HomeLAN era actually. Those big-boards stats are a bit messed up as I turned on some protection code that prevents spiders from crawling the wrong info. Problem is, that prevents them from crawling as well. So we are 600th in members using really old data. I bet if I contacted the guys at big-boards, I could get them access to a raw numbers page, and we would shoot up considerably. I think our posts are also semi-protected. I don't know, I don't keep track of these things very well.

    Älphämönkëy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    mcc wrote:
    Thanks for the update.

    Will the Opinion forum support searching once it is ready?
    Yup. We will be able to have a lot more "fancy" effects since we will be freeing up the DB from constantly being beaten on. For those who enjoy buzzwords, we will be using fun AJAXy bits to make it feel more like an application.

    Älphämönkëy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    Alpha, I will take you out to lunch someday

    :^: Minnesota
    I appreciate the offer, but I'm only in Minnesota in the summers, during the school year I hide elsewhere. Even so, outside of PAX, I don't meet up with many people. The one exception being Snowcone, I met him and his wife in Texas a few years back.

    Älphämönkëy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    Deusfaux wrote:
    it was him or Ramius who once spouted off a few stats that as far as I remember, included stuff from before the big wipe, even...
    That was me. I can't get at those stats any more due to pruning. I can give you some estimates though. This post here will be #22655487 in thread #216077. Those numbers started at 1 when we relaunched.

    Älphämönkëy on
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    jothki wrote:
    As long as everything works more or less in the same way on the user side, I'll be happy. I love the current feel of the forums.

    What will we be seeing when the shift happens?
    The forums will look and feel a bit different. I would like to port BBCode over to opinion, but I don't know if I will have the time. I might make a list of code bounties, for kind developers to tackle. If so, that would be high up there.

    Älphämönkëy on
  • Options
    FinsterFinster Registered User regular
    edited March 2006
    I've been wondering this since a few days ago when I landed a Web programming (PHP) job with a local company here. They do colocation and all that other stuff as well. The whole place runs on FreeBSD/PHP/MySQL.

    So, I was wondering what (basic) configuration is used to host the forums. My understanding is that it is a single server using some of the software noted in Alphamonkey's sig, but I guess I'm more curious about what actually goes into setting up the server as far as backend stuff.

    I think there was a thread on this very subject many moons ago, but I have no idea where that would be now.

    Finster on
    herpderp
  • Options
    ÄlphämönkëyÄlphämönkëy Registered User regular
    edited March 2006
    Finster wrote:
    I've been wondering this since a few days ago when I landed a Web programming (PHP) job with a local company here. They do colocation and all that other stuff as well. The whole place runs on FreeBSD/PHP/MySQL.

    So, I was wondering what (basic) configuration is used to host the forums. My understanding is that it is a single server using some of the software noted in Alphamonkey's sig, but I guess I'm more curious about what actually goes into setting up the server as far as backend stuff.

    I think there was a thread on this very subject many moons ago, but I have no idea where that would be now.
    Normally I'd get into it. But I don't have the time. I'd say you could search for it, but... well... yeah.

    Älphämönkëy on
  • Options
    FyreWulffFyreWulff YouRegistered User, ClubPA regular
    edited March 2006
    I'm looking forward to a forumized version of this software to use on my site.. that is, unless I get the money for vBB first.

    My patience has run out with the phpBB developers.


    I imagine the jail system will come over, or will that be forgotten? (or added later on)

    FyreWulff on
Sign In or Register to comment.