!Corgi Posted August 30, 2012 Share Posted August 30, 2012 Just to correct the record, there are two servers: One for the web pages and content, and one for the database. I thought we could have a server discussion topic on ideas on how to make the SFO server more efficient and faster. I am really high right meow and will come back later and post my theories. For meow, Discuss the issues related to the website and other topics. We should work together and pool our knowledge to fix the website as a community :-) Link to comment Share on other sites More sharing options...
"User" Posted August 30, 2012 Share Posted August 30, 2012 Nova, you know I had to overhaul your OP. >_> Please don't call people out like that again. I support we could have a general roundtable discussion on the matter. What exactly would you like to see improved on? Link to comment Share on other sites More sharing options...
!Corgi Posted August 30, 2012 Author Share Posted August 30, 2012 We need a new host and possibly some hardware upgrades, Specifically looking at a faster RPM HardDisk for the Data. Usually what happens when a website goes down (Without an attack) is because the hardware pooped on its shoe and needs to be rebooted. Servers are made to have on average 10,000 hours of uptime before any maintenance is even thought of. Link to comment Share on other sites More sharing options...
sroberson Posted August 30, 2012 Share Posted August 30, 2012 Servers are made to have on average 10,000 hours of uptime before any maintenance is even thought of. ...Man...apparently that's a lot to ask for in the job I have. I think they count themselves lucky if nothing randomly goes down in the span of a year, let alone a few months (switches failing, power supplies failing, co-location connection issues, random Windows updates crashing file sharing resources, the myriad of hardware issues that can occur on the server itself, etc.). I think the simplest way to make sure that the SFO website doesn't go down, and possibly even increase speed for some is to have simply multiple servers that operate in a high availability situation. Don't get me wrong I am not begging for DZ to go and shell out a ton of money for this to happen, but from my angle and experience that's really the only scaleable solution guaranteed to provide a benefit. Asking for hardware upgrades on a server farm that DZ doesn't own in any way (just the space) just isn't really an option unless that is in the license agreement with the hosting agent - doubtful. I might be wrong, but didn't we also single out software issues with why the server was having issues just recently? I didn't keep up with exactly what was going on. In conclusion, just to make sure nobody misunderstands. I appreciate all the work that goes into this site, if it goes down every once in a while due to a "whoops"...that's ok, I can live. I need to go outside and be in the sunlight once in a while . Update: I just sat back re-reading what I posted and realized that actually a lot of work would have to be done to set up a high availability system regarding how it communicates with databases and the establishment of reliable healthchecks for servers possibly across a continent that won't suffer from dropped packets and false positive failover situations...Anyways! Link to comment Share on other sites More sharing options...
!Corgi Posted August 31, 2012 Author Share Posted August 31, 2012 Another thing that User and I discussed is the crawlers coming from Chinese IP's. We need to monitor what traffic is actually beneficial to the website and what is simply just eating up the bandwidth. Understanding that DZ isn't made of money, This is where it becomes difficult because he does indeed have a popular site with a very decent amount of traffic, But in order to solve the problem, We really only have a few options that wouldn't cost too much although a LOT of volunteer man hours. 1. IPB - The CMS that this website runs on IPB isn't really the best CMS and I remember when we made the switch, There was a whole ton of problems. We could take a look at the extensions and features that are enabled and possibly make the website run more lean without unused features. - The other thing we could do is possibly look for a new CMS or have a group of us volunteer and code a fresh CMS/HTML/Java site. 2. The actual hosting plan/payment The server(s) that dz is renting aren't exactly affordable as it is. He runs this website out of pocket as a hobby. Notice that there is no adds on the site, This means that there is no revenue coming in other than donations (Which I think are slowed to nothing because everyone has already donated.) We'd have to increase the bandwidth/hardware of the site for it to run more smoothly even with the fat CMS, In theory if you had enough power, It could work. But at what expense? 3. The User Created Content (UCC) This site is run by the community and made FOR the community, Everyone has images, signatures, Private messages, profiles ect... This all adds to the list that the data server has to look through for pulling content. (Back onto making the site more lean) It is possible to have all the data compressed server side (On the Data Server) then add a little bit more load onto the Webpage/CMS Server to unpack the information on the End-User's side. This although would probably require a large amount of cookie storage and would be slow on accessing the website after having a cache clear. It's a bit tricky to lean out any website, But I'm just continuing to add my thoughts and theories on the website. Hopefully once we add enough information into the pile, We'll be able to work something out and have a solution. Link to comment Share on other sites More sharing options...
sroberson Posted August 31, 2012 Share Posted August 31, 2012 Define "crawlers" please. Also, does DZ physically own the machines? I made an assumption that he rents server space from a farm that he has no interaction with. If he has full physical administrative power over the hardware and networking we might be able to at least figure out an eloquent means of trying to filter abusers of the website by means of a firewall (if I'm wrong could you explain the environment and situation we are working with?)...I actually had a fun idea a while ago of making a dynamic log inspecting firewall application that would locate network abusers and temporarily blacklist them (mind you I'm not a strong programmer, more so visionary). I get the feeling a hardware upgrade or update could be in the works, budget and donations permitting...also on the note of donations, last I saw DZ said there was no real need for donations, but maybe given the issues lately that could change. I have never been informed on how to donate. Just for funzies, has there been consideration of a caching reverse proxy server? Or some system of high performance SSD caches in the server? Performance RAID arrays? Just kind of spitballing a few fun sounding ideas If we start to undertake a project of recoding the website, ill offer whatever help I can give. I'm not too bad with html and scripting. Link to comment Share on other sites More sharing options...
DZComposer Posted August 31, 2012 Share Posted August 31, 2012 Firstly, I am not in favor of switching software at this time. I agree that IP.Content isn't the best, but I've made it work thus far, save for a few CSS bugs that pop-up every update. I do like IP.Board, however, and I find the integration and unified look between IP.Content and IP.Board worth it. One of my projects I am working on is planning a new skin for the site to honor Star Fox's 20th anniversary next year. While that theme is live, I will develop a new site theme for 2014 and beyond. This theme will be based on the celebratory one, which I am building from the ground-up with the ultimate goal being usability and css stability. The reason for the CSS bugs is that the 3-column layout I have going is based on hacking-up currently existing styles. I have gotten better at web development since I built that and I will rebuild it the right way using unique CSS classes that will make it look right without having to hack the theme (to see what I mean, activate the IP.Board theme and look at the main site. It looks like shit.). This way, as IPS is updated, it won't break something on the site every time. 2. Servers. SFO's web service is hosted on a managed VPS, and the database is hosted on a second one. The VPSes are on a shared machine with other VPSes, so there is some resource competition at times. Dedicated servers at Dreamhost start at around $120/mo. While I think the lowest server is adequate given SFO's volume, it is not financially feasible for me at this time. I despise ads, so I wouldn't put that on others. So, what can be done is based upon what I can afford plus what is donated. 3. Traffic monitoring. I am running Google Analytics and IPB's own logs. My current problem is whack-a-mole with Baiduspiders. Every time I .htaccess block a Baidu subnet, Baidu bots on another subnet start crawling the site. Also, Bounce Rate (percentage of traffic that views one page and leaves) is higher than I would like, though I need to get the site content flowing to improve that likely. Link to comment Share on other sites More sharing options...
!Corgi Posted September 1, 2012 Author Share Posted September 1, 2012 Also, Bounce Rate (percentage of traffic that views one page and leaves) is higher than I would like, though I need to get the site content flowing to improve that likely. I've noticed any time I post a link to facebook, It seems to have 2 facebook crawlers on the page for one link. Is there a way to limit what information is pulled when some page is linked? Link to comment Share on other sites More sharing options...
sroberson Posted September 1, 2012 Share Posted September 1, 2012 I've noticed any time I post a link to facebook, It seems to have 2 facebook crawlers on the page for one link. Is there a way to limit what information is pulled when some page is linked? Maybe I'm misunderstanding the issue, but the only way I can imagine this happening is if we limited content that guests are allowed to access which could have the effect of souring the feelings of potentially new members. What affect does the crawlers have on the server? Are they excessively increasing the network activity? Also, about how much is being spent on the current hosting facility? Link to comment Share on other sites More sharing options...
Hidi Posted October 13, 2012 Share Posted October 13, 2012 I was having a "server is busy" error today. Thought I'd add this here than to the older "Server is Busy Errors" thread which is in Rules and Announcements. Link to comment Share on other sites More sharing options...
Owner/Technical Admin Sideways Posted October 18, 2012 Owner/Technical Admin Share Posted October 18, 2012 I don't see issues on my side. SFO runs fast and everything works well for me. At least on my pc here. Link to comment Share on other sites More sharing options...
Hidi Posted October 19, 2012 Share Posted October 19, 2012 Well I'm not expecting the speed to be stellar considering that this isn't top-grade and I'm not particularly close to the server, but I think there is still a standard of performance that we should meet in order to attract and keep new members. Site performance and speed is a big thing. Especially if we may be expecting a lot of traffic in the distant future. Link to comment Share on other sites More sharing options...
DZComposer Posted October 19, 2012 Share Posted October 19, 2012 I do not need individual reports of server issues. I know things are slow. I am looking into solutions. I can't change anything in the immediacy, but I will fix this. Link to comment Share on other sites More sharing options...
Owner/Technical Admin Sideways Posted October 19, 2012 Owner/Technical Admin Share Posted October 19, 2012 Thank you for all you do, DZ. Link to comment Share on other sites More sharing options...
Hidi Posted October 20, 2012 Share Posted October 20, 2012 Thank you for all you do, DZ. Ditto. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now